System

Abstract

A system is provided.SOLUTION: A system comprising: a default setting unit configured to allow a user to register; a unit configured to collect and store conversation data with the user; a unit configured to analyze the collected conversation data and learn a manner of speaking and a context of the user; a unit configured to train a user-specific model based on the learned data; a unit configured to generate a reply of the conversation using the generated user-specific model; and a unit configured to transmit the generated reply to a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, with the spread of messaging apps as a communication tool, users are increasingly seeking unique and interesting conversations. However, current automated conversation systems and bots can only provide generic responses, making it difficult to create conversations that reflect a user's unique speaking style and way of thinking. As a result, the unnatural and impersonal nature of automated conversations detracts from the user experience. Therefore, there is a demand for conversational bots that are like clones of the user, reflecting their individuality. [Means for solving the problem]

[0005] The present invention relates to a system that learns a user's speaking style and way of thinking and creates a clone bot dedicated to that user. Specifically, the system includes an initial setup means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data to learn the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, and a means for sending the generated replies to the user's terminal. The system also includes a means for periodically analyzing the collected conversation data and updating the model, and a means for the user to invite the clone bot to a group chat, which automatically generates and sends replies to other users' messages that mimic the user's speaking style. This allows for a natural, interactive conversation experience unique to the user, improving the user experience.

[0006] "User" means an individual or organization that uses the system and interacts with and configures the clonebot.

[0007] "Initial setting means" is a function that allows users to register a clonebot and create basic settings and profiles.

[0008] "Conversation data" refers to information about messages and dialogue exchanged between the user and the clonebot, and is used to learn about the user's characteristics.

[0009] "Means of collection and storage" refers to the function of acquiring conversation data with users in real time or after a certain period of time and storing it in a database or storage.

[0010] "Means for analysis" refers to a function for analyzing collected conversation data and extracting characteristics such as the user's speaking style, grammar, and specific expressions.

[0011] "Means for learning" is a function that uses machine learning algorithms to learn the user's speaking style and context based on analyzed data.

[0012] "Means for training" refers to the functionality for generating or updating a user-specific AI model based on learned data.

[0013] "Generation means" refers to a feature that uses a trained AI model to create reply messages that mimic the user's speaking style.

[0014] "Means for sending" is a function for sending the generated reply message to the user's device or group chat.

[0015] "Means to update the model" refers to the ability to re-analyze conversational data periodically or as needed and re-train the AI ​​model to keep it up to date.

[0016] A "group chat" is a chat room within a messaging app that multiple users join, and is a place where clonebots can participate in the conversations.

[0017] "Means to imitate" is a function that automatically generates a reply message in response to a message from another user by replicating the user's speaking style and expressions. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention relates to a dialogue system using a clonebot personalized for each user. This system learns the user's speaking style and way of thinking, generates an AI model dedicated to the user, and realizes natural dialogue that mimics the user's speaking style.

[0040] System Configuration

[0041] This system consists of the following main components:

[0042] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[0043] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[0044] 3. Database: Stores and manages user conversation data and trained models.

[0045] Initial setting method

[0046] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[0047] Server: Receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[0048] Conversation data collection and storage method

[0049] User: Interacts with the clonebot on a daily basis, for example sending a message saying "I was busy this morning."

[0050] Device: Sends a message to the server via the LINE API.

[0051] Server: Analyzes received messages and stores conversation data (content, sender information, timestamp, etc.) in a database.

[0052] Analysis and learning tools

[0053] Server: Periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0054] Server: Trains a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[0055] Server: After training is complete, save the updated model to the database.

[0056] Interacting with Clonebots

[0057] Users: Invite your clonebot to join a group chat and have it participate in conversations with other users.

[0058] Friends: Asking in a group chat, "What did you do over the weekend?"

[0059] Device: Sends the received message to the server via the LINE API.

[0060] Server: Analyzes the received message and generates a reply using a user-specific AI model. Specifically, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[0061] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[0062] Specific examples

[0063] For example, if Tanaka were to use CloneBot, the following conversations would be possible: If Tanaka sends a message to CloneBot on LINE saying, "Last night's movie was great," CloneBot would reply, "Yeah, I enjoyed it too. What shall we see next?" Also, if Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on Tanaka's behalf, saying, "I went camping over the weekend."

[0064] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[0068] Step 2:

[0069] Device: Sends the user's message to the server via the LINE API.

[0070] Step 3:

[0071] Server: Receives the initialization message and generates a new user identification (user ID).

[0072] Step 4:

[0073] Server: Save the new user's profile (user ID, initial settings information) in the database.

[0074] Step 5:

[0075] Server: Generates the initial setup completion message and sends it to the user.

[0076] Step 6:

[0077] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[0078] Step 7:

[0079] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0080] Step 8:

[0081] Device: Sends messages from users to the server via the LINE API.

[0082] Step 9:

[0083] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[0084] Step 10:

[0085] Server: Save the extracted conversation data in a database.

[0086] Step 11:

[0087] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style and context.

[0088] Step 12:

[0089] Server: Generates or updates a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[0090] Step 13:

[0091] Server: After training is complete, the updated AI model is saved in the database.

[0092] Step 14:

[0093] User: Invite your clonebot to join a group chat and have it interact with other users.

[0094] Step 15:

[0095] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0096] Step 16:

[0097] Device: Send this message to the server via the LINE API.

[0098] Step 17:

[0099] Server: Analyzes the received message and generates an appropriate reply. Creates natural responses based on the user's past conversation data and training results.

[0100] Step 18:

[0101] Server: The generated reply message is sent to the user's device via the LINE API.

[0102] Step 19:

[0103] Terminal: Clonebot's reply message appears in group chat.

[0104] For example, if Mr. Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the clonebot will generate a reply saying, "Yes, it was really interesting. What shall we see next?" In this way, natural dialogue that reflects the user's characteristics is realized.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional dialogue systems have struggled to achieve natural dialogue that takes into account the individual speaking style and context of each user. Furthermore, no dialogue system to date has used a generative AI model that can accurately learn and mimic a user's speaking style and way of thinking. This has made it difficult for users to enjoy dialogue that reflects their own individuality, and the system's responses can feel unnatural.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific generative AI model based on the learned data, a means for generating conversation replies using the generated user-specific generative AI model, and a means for transmitting the generated replies to the user terminal, thereby enabling the user to enjoy natural conversations that reflect their own speaking style and way of thinking.

[0110] "Initialization means" refers to the process or function by which a user registers with the system.

[0111] "Conversation data" refers to the messages and dialogue exchanged between the user and the clonebot.

[0112] "Means of collection and storage" refers to the functions and processes for collecting conversation data with users and storing it in a database.

[0113] "Means of analysis" refers to the technologies and algorithms used to analyze collected conversational data and understand the user's speaking style and context.

[0114] "Means of learning" refers to the process of training users' speaking and expressing themselves based on analytical data.

[0115] A "generative AI model" refers to an artificial intelligence model that is trained to mimic a user's speech style and context.

[0116] "Training method" refers to the process by which a generative AI model is trained based on user conversation data.

[0117] "Means for generating replies" refers to the ability to use a generative AI model to automatically create replies that mimic the way a user would speak.

[0118] "User Device" refers to the device a user uses to interact with a clonebot, including a smartphone or computer.

[0119] "Means for sending" refers to the function or process for sending the generated reply message to the user terminal.

[0120] "Means of updating" refers to the process of periodically retraining the generative AI model based on collected conversational data and updating the model based on the latest information.

[0121] A "clonebot" is an AI assistant that is trained to mimic the way a user speaks and thinks.

[0122] "Group chat" refers to a chat function that allows multiple users to interact at the same time.

[0123] "Other users' messages" refers to messages sent by users other than the CloneBot user within a group chat.

[0124] This invention relates to a dialogue system using a user-specific clonebot, which learns the user's speaking style and way of thinking, generates a user-specific generative AI model, and realizes natural dialogue that mimics the user's speaking style.

[0125] System Configuration

[0126] This system consists of the following main components:

[0127] 1. User device: A device such as a smartphone or PC on which the user interacts with the clonebot using a communication app.

[0128] 2. Server: Hosts the clonebot generative AI model and performs data processing, analysis, and training.

[0129] 3. Database: Stores and manages user conversation data and trained models.

[0130] Initial setting method

[0131] User: The user adds the clone bot as a friend in the communication app and performs the initial setup. Specifically, the user adds the clone bot by scanning the QR code and sends a message saying, "Make my clone bot."

[0132] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[0133] Conversation data collection and storage method

[0134] User: The user interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0135] Device: The user device sends this message to the server through the API.

[0136] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[0137] Analysis and learning tools

[0138] Server: The server periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0139] Server: The server trains a generative AI model specific to the user based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[0140] Server: After training is complete, save the updated model to the database.

[0141] Interacting with Clonebots

[0142] User: Users can invite clonebots to group chats and have them participate in conversations with other users.

[0143] Friends: Friends ask in group chats, "What did you do over the weekend?"

[0144] Device: The user device sends the received message to the server via the API.

[0145] Server: The server analyzes the received message and generates a reply using a generative AI model specific to the user. For example, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[0146] Server: The generated reply message is sent to the user's device via the API and displayed in the group chat.

[0147] Specific examples

[0148] For example, when a user uses this system, the following dialogue is possible:

[0149] If a user sends a message to CloneBot in a communication app saying, "The movie last night was great," CloneBot will reply, "I enjoyed it too. What shall we watch next?" If a user's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on the user's behalf, saying, "I went camping over the weekend."

[0150] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[0151] Examples of prompt statements

[0152] For example, "Generate a scenario in which you talk to a clonebot about a movie you saw yesterday."

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1:

[0155] Initial Setup

[0156] User: The user adds the clone bot as a friend in a communication app and sends a message saying, "Make a clone bot of me."

[0157] Input: Default message from the user

[0158] Output: Initialization request sent to server

[0159] What it does: The user scans the QR code in the app, adds the clonebot as a friend, and then types and sends a message saying, "Make a clonebot of me."

[0160] Step 2:

[0161] User Registration

[0162] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to a database.

[0163] Input: Initial Setup Request

[0164] Output: User ID and profile are saved in the database.

[0165] What happens: The server analyzes the received message and stores the new user's information in the database. It generates a user ID using an ID generation algorithm and stores the profile information.

[0166] Step 3:

[0167] Conversation data collection

[0168] User: The user interacts with the clonebot on a daily basis, sending messages such as "I was busy this morning."

[0169] Input: Conversational message from user

[0170] Output: The message is sent to the server

[0171] Specific operation: The user interacts with the clonebot and sends messages, which the device forwards to the server via API.

[0172] Step 4:

[0173] Saving conversation data

[0174] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[0175] Input: Received message

[0176] Output: Parsed conversation data is saved in a database

[0177] Specific operation: The server analyzes the contents of the message, adds sender information and a timestamp, and stores the message in a database.

[0178] Step 5:

[0179] Analysis of conversation data

[0180] Server: The server periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0181] Input: Conversation data retrieved from the database

[0182] Output: Analysis results

[0183] How it works: The server periodically retrieves conversation data stored in the database and analyzes it using machine learning algorithms. The analysis results include the user's unique speaking style and expressions.

[0184] Step 6:

[0185] Training a generative AI model

[0186] Server: The server trains a user-specific generative AI model based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[0187] Input: Analysis results

[0188] Output: A trained generative AI model

[0189] Specific operation: The server uses the acquired analysis results to train the generative AI model. It uses OpenAI's Fine tuning API to optimize the model for the user.

[0190] Step 7:

[0191] Saving the generated AI model

[0192] Server: Stores the trained generative AI model in a database.

[0193] Input: A trained generative AI model

[0194] Output: Generative AI model stored in a database

[0195] What happens: The trained model is saved in a database and ready to be used for future dialogue generation.

[0196] Step 8:

[0197] Interacting with Clonebots

[0198] User: Users can invite clonebots to group chats to participate in conversations with other users.

[0199] Input: Message from other users

[0200] Output: A reply message is sent by the clonebot

[0201] What it does: A friend asks, "What did you do over the weekend?" The device sends this message to a server via an API. The server analyzes the message and uses a trained generative AI model to generate a natural-sounding response, such as "I went camping over the weekend." The server then sends the generated reply message to the user's device, where it appears in the group chat.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] Current food delivery services lack personalized recommendations and interactions based on users' individual preferences, which makes the ordering process cumbersome, as users are presented with the same operations and options every time. Furthermore, suggestions do not take into account users' food and beverage preferences or eating history, which makes it time-consuming to find more suitable options.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a means for learning the user's preferences using the above means and recommending meals and drinks, and a means for transmitting the generated recommendations to the user terminal, thereby enabling the user to receive personalized dialogue and recommendations based on their individual preferences.

[0207] "User-registered initial setting means" is a function that allows users to input initial information into the system and customize the system individually.

[0208] "Means for collecting and storing conversation data with users" refers to the function of recording and storing the content of conversations users have with the system.

[0209] "Means of analyzing collected conversation data and learning the user's speaking style and context" refers to a function that understands the user's speaking style, unique expressions, and context based on recorded conversation data.

[0210] "Means to train user-specific models based on learned data" refers to the ability to create individually personalized AI models based on the user's learned speech patterns and context.

[0211] "Means for generating conversation replies using a generated user-specific model" is a function that uses a user-specific AI model created to generate replies tailored to the user's style.

[0212] "Means of learning the user's preferences using the above means and recommending meals and drinks" refers to a function that analyzes the user's favorite dishes and drinks based on the user's conversation data and past ordering history, and makes recommendations accordingly.

[0213] "Means for sending the generated recommendation content to the user's device" refers to a function that notifies the user's device of the recommendation content created by the AI ​​model.

[0214] The present invention relates to a system that provides a food delivery assistant customized for each user. The system collects and analyzes user conversation data, generates a user-specific AI model, and recommends meals and beverages based on the user's preferences.

[0215] System Configuration

[0216] The system mainly consists of the following components:

[0217] 1. User device: The user interacts with the clonebot on a device such as a smartphone or computer.

[0218] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[0219] 3. Database: Stores and manages user conversation data and trained models.

[0220] Hardware and software used

[0221] Hardware: Smartphones, cloud servers

[0222] Software: LINE API, OpenAI Fine-tuning API

[0223] User device behavior

[0224] On the user's device, the user adds the clonebot as a friend through LINE or a dedicated app and starts the initial setup process by scanning the QR code and sending a message saying, "Help me deliver food."

[0225] Server Operation

[0226] The server receives the initial setup request sent from the device, generates a user ID, and saves the profile in a database. It then collects and saves daily conversation data and periodically analyzes it.

[0227] Analysis and Training

[0228] The server analyzes the user's speaking style and preferences based on the collected conversation data, and uses the OpenAI Fine-tuning API to train a user-specific AI model, which is then used to generate natural-sounding dialogue based on the user's preferences and context.

[0229] Dietary and Food Recommendations

[0230] The server learns the user's preferences based on their conversation data and past ordering history, so that when the user orders food and drink, it can make recommendations that are tailored to the user's preferences.

[0231] Interacting with Clonebots

[0232] If a user wants to order a specific meal, they can ask the clonebot, "What would you like to eat today?" and the clonebot can make a suggestion based on the user's preferences, such as, "I feel like eating Italian today. That pasta with meat sauce is especially delicious." Also, if the user wants to order the same dish again, the order details are saved, so they can be reordered with a simple command, such as, "Order the pasta with meat sauce again."

[0233] Specific examples

[0234] For example, if a user asks the clonebot, "What kind of food do you recommend today?", it will suggest, "You seem to be coming down with a cold today. How about some stomach-friendly Japanese food?" If the user asks, "Which restaurant specifically?", it will respond, "What's the nearest Japanese restaurant? It was delicious the last time I ordered."

[0235] Prompt Sentence Examples

[0236] "What dish do you recommend today?"

[0237] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[0238] In this way, the system of the present invention can provide personalized interactions and recommendations based on the user's individual preferences, significantly improving the food delivery ordering process.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Step 1:

[0241] Users add the clone bot as a friend via LINE or a dedicated app and send it a message saying, "Help with food delivery."

[0242] Input: User's initial setup request

[0243] Output: Notification of the start of the initialization process

[0244] How it works: The user scans the QR code, becomes friends with the clonebot in the LINE app, sends an initial setup message, and sends a request to the server.

[0245] Step 2:

[0246] The server receives the initial setup request, generates a user ID and saves the profile in a database.

[0247] Input: Initial Setup Request

[0248] Output: User ID and profile creation and saving

[0249] Behavior: The server analyzes the initial configuration request data, generates a new user ID, and stores the generated user ID and profile in the database.

[0250] Step 3:

[0251] Collect and store daily conversation data with users.

[0252] Input: User's daily conversation data

[0253] Output: Saved conversation data

[0254] Operation: The device sends the user's conversation data to the server via the LINE API. The server stores the received data in a database.

[0255] Step 4:

[0256] The server analyzes the collected conversation data and learns the user's speaking style and preferences.

[0257] Input: Saved conversation data

[0258] Output: Analyzed speaking style and preference data

[0259] How it works: The server periodically retrieves conversation data from the database and uses natural language processing techniques to analyze the user's speaking style, context, and preferences.

[0260] Step 5:

[0261] The server trains a user-specific AI model based on the learned data.

[0262] Input: Parsed data

[0263] Output: A trained, user-specific AI model

[0264] How it works: The server uses the analyzed data to train a user-specific CloneBot model using the OpenAI Fine-tuning API.

[0265] Step 6:

[0266] The server generates conversations and recommendation replies using the generated user-specific model.

[0267] Input: Trained AI model, user request

[0268] Output: Natural conversation and recommendation replies

[0269] How it works: The server receives a user request and uses the trained model to generate a natural-sounding response based on the user's speaking style and preferences.

[0270] Step 7:

[0271] The server sends the generated reply to the user terminal.

[0272] Input: Generated reply

[0273] Output: Reply displayed on the user's terminal

[0274] Behavior: The server sends the generated reply to the user's device via the LINE API, and the reply is displayed on the user's device.

[0275] Specific examples

[0276] Example prompt sentence:

[0277] "What dish do you recommend today?"

[0278] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[0279] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0280] The present invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining it with an emotion engine.

[0281] System Configuration

[0282] This system consists of the following main components:

[0283] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[0284] 2. Server: Hosts the CloneBot AI model and emotion engine, and performs data processing, analysis, and training.

[0285] 3. Database: Stores and manages user conversation data and trained models.

[0286] 4. Emotion Engine: An engine that analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[0287] Initial setting method

[0288] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[0289] Server: Receives the initial setup request and generates a new user identification (User ID), which stores the new user's profile in the database.

[0290] Conversation data collection and storage method

[0291] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0292] Device: Sends a message to the server via the LINE API.

[0293] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.). Additionally, it uses an emotion engine to analyze the emotional state of the message.

[0294] Server: Stores the extracted conversation data and emotion data in a database.

[0295] Analysis and learning tools

[0296] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[0297] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[0298] Server: After training is complete, save the updated model to the database.

[0299] Interacting with Clonebots

[0300] User: Invite your clonebot to join a group chat and have it interact with other users.

[0301] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0302] Device: Send this message to the server via the LINE API.

[0303] Server: Analyzes received messages and generates appropriate replies. Creates natural responses based on the user's past conversation data, training results, and emotional state. Using an emotion engine, it recognizes emotions such as "fun" or "difficult" and generates appropriate replies.

[0304] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[0305] Specific examples

[0306] For example, if Tanaka sends a message to CloneBot saying, "The movie yesterday was great," CloneBot will recognize the emotion as "enjoyment" and reply, "Yeah, it was really great. What shall we watch next?" If Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can generate a response that includes an emotional expression, such as, "I went camping over the weekend. It was so refreshing!"

[0307] In this way, the present invention allows users to enjoy a natural and interactive dialogue experience that reflects their own emotions and speaking style. Furthermore, by combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[0308] The processing flow will be explained below.

[0309] Step 1:

[0310] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[0311] Step 2:

[0312] Device: Sends the user's message to the server via the LINE API.

[0313] Step 3:

[0314] Server: Receives the initialization message and generates a new user identification (user ID).

[0315] Step 4:

[0316] Server: Save the new user's profile (user ID, initial settings information) in the database.

[0317] Step 5:

[0318] Server: Generates the initial setup completion message and sends it to the user.

[0319] Step 6:

[0320] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[0321] Step 7:

[0322] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0323] Step 8:

[0324] Device: Sends messages from users to the server via the LINE API.

[0325] Step 9:

[0326] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[0327] Step 10:

[0328] Server: Analyzes the user's emotional state from the received message using the emotion engine. For example, recognizes the emotion "stress" from the keyword "busy."

[0329] Step 11:

[0330] Server: Stores the extracted conversation data and emotion data in a database.

[0331] Step 12:

[0332] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[0333] Step 13:

[0334] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[0335] Step 14:

[0336] Server: After training is complete, the updated AI model is saved in the database.

[0337] Step 15:

[0338] User: Invite your clonebot to join a group chat and have it interact with other users.

[0339] Step 16:

[0340] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0341] Step 17:

[0342] Device: Send this message to the server via the LINE API.

[0343] Step 18:

[0344] Server: Analyzes the received message and generates an appropriate reply. It creates natural responses based on the user's past conversation data, training results, and emotional state. For example, it uses the emotion engine to recognize the emotion "fun" and generates a reply such as, "I went camping this weekend. It was so refreshing!"

[0345] Step 19:

[0346] Server: The generated reply message is sent to the user's device via the LINE API.

[0347] Step 20:

[0348] Terminal: Clonebot's reply message appears in group chat.

[0349] As a concrete example, if Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the emotion engine will recognize the emotion of "enjoyment" from the keyword "interesting." The server will generate a reply that reflects this emotion: "Yes, it was really interesting. What shall we watch next?" and send it to Tanaka's device via the LINE API. In this way, natural dialogue that reflects the user's emotions and speaking style is realized.

[0350] Example 2

[0351] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0352] Conventional dialogue systems have struggled to achieve natural dialogue that reflects the user's emotional state. Furthermore, there are limited means to individually learn a user's speaking style and context and generate a model dedicated to that user. Furthermore, mechanisms for effectively analyzing and storing collected conversation data and regularly updating models are inadequate.

[0353] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0354] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific artificial intelligence model based on the learned data, a means for generating conversation replies using the generated user-specific artificial intelligence model, a means for sending the generated replies to the user terminal, and a means for analyzing the emotional state of the message using an emotion engine. This makes it possible to realize natural, interactive dialogue that reflects the user's emotions and speaking style, and to continuously generate and update a user-specific model.

[0355] "User" refers to an individual or end user of an interactive system.

[0356] The "initial setting means" includes a setting procedure that a user must perform before using the dialogue system.

[0357] "Conversation Data" refers to information, including text messages, that a user sends and receives through an interactive system.

[0358] "Means for collection and storage" refers to the function by which the server receives conversation data, analyzes it, and stores it in a database.

[0359] "Means for analyzing" refers to the methods and technologies that use collected conversational data to analyze a user's speaking style, context, and emotional state.

[0360] "Learning means" refers to the process of accumulating information about users based on the analysis results and using it to improve the quality of future interactions.

[0361] "Artificial intelligence model" refers to a computer program that uses machine learning and natural language processing techniques to mimic a user's speech patterns and context.

[0362] "Training" refers to the process of using collected and learned data to improve and optimize an artificial intelligence model.

[0363] "Means for generating replies" refers to the ability to automatically generate replies to users' messages using a trained artificial intelligence model.

[0364] An "emotion engine" is a software tool that analyzes emotions from users' messages and uses that information to reflect them in the content of the conversation.

[0365] "Communications application" refers to application software that allows a user to send and receive messages.

[0366] This invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining an emotion engine. The main components of this system include a user terminal, a server, a database, and an emotion engine.

[0367] System Configuration

[0368] 1. User terminal: The user terminal of this system is a device such as a smartphone or PC, which the user uses to interact with the clonebot through the LINE app.

[0369] Example: A user sends a message to a clone bot on the LINE app saying, "The movie yesterday was great."

[0370] 2. Server: The server hosts the CloneBot's artificial intelligence model and emotion engine, and performs data processing, analysis, and training.

[0371] Software used: Node.js, Python, NLTK, TensorFlow, PyTorch, OpenAI API

[0372] Example: A server parses messages received from users and analyzes their emotional state using Python's NLTK.

[0373] 3. Database: The database stores and manages user conversation data and trained models.

[0374] Software used: MySQL, SQLAlchemy

[0375] 4. Emotion Engine: The emotion engine analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[0376] Initial setting method

[0377] Users add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, users add the clone bot by scanning the QR code and send a message saying, "Make my clone bot."

[0378] The server receives the initial setup request and generates a new user identity (user ID), which saves the new user's profile in the database. It uses the UUID library to generate a unique ID and saves it in the MySQL database.

[0379] Conversation data collection and storage method

[0380] Users exchange messages with the clonebot on a daily basis, for example, sending a message saying, "I was busy this morning."

[0381] The device forwards the sent message to the server via the LINE API.

[0382] The server parses the received messages and extracts metadata such as content, sender information, and timestamps. It also analyzes the sentiment of the messages using Python's NLTK. The analyzed data is stored in a database using SQLAlchemy.

[0383] Analysis and learning tools

[0384] The server periodically retrieves conversation data from the database. This is automated using a scheduler. The retrieved data is analyzed using TensorFlow or PyTorch to generate and update an AI model specific to the user. OpenAI's Fine Tuning API is used for training. The generated and updated model is stored in the database.

[0385] Interacting with Clonebots

[0386] Users can invite clonebots to join group chats in communication applications to interact with other users.

[0387] A friend sends a question in a group chat: "What did you do over the weekend?"

[0388] The device forwards this message to the server via the LINE API.

[0389] The server uses OpenAI's GPT-3 API to generate appropriate replies, taking into account past conversation data and emotional states. The replies are then sent to the user's device via the LINE API and displayed in the group chat.

[0390] Examples of specific examples and prompts

[0391] For example, if a user sends a message to a clonebot on the LINE app saying, "Last night's movie was great," the server will recognize the emotion as "enjoyment" and generate a reply saying, "Yeah, it was really great. What should we watch next?"

[0392] Example prompt sentence:

[0393] "Make a clone bot of me."

[0394] "The movie yesterday was interesting."

[0395] "What did you do over the weekend?"

[0396] In this way, the present invention can realize natural and interactive dialogue that reflects the user's emotions and speaking style. By combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[0397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0398] Step 1:

[0399] The user adds the clone bot to their friends on the LINE app and sets up the initial settings. At this stage, the user scans the QR code and sends a message saying, "Make my clone bot."

[0400] Input: QR code scan and initial message

[0401] Output: Initialization request

[0402] Step 2:

[0403] The server receives an initial setup request through the LINE API. It uses Node.js to parse this request and generate a new user ID. It uses the UUID library to generate a unique ID.

[0404] Input: Initial Setup Request

[0405] Output: User ID

[0406] Step 3:

[0407] The server stores the generated user ID in a MySQL database and creates a profile for the user, using SQLAlchemy for database operations.

[0408] Input: User ID

[0409] Output: User profile stored in the database

[0410] Step 4:

[0411] Users can exchange messages with the clone bot on a daily basis using the LINE app, for example, sending a message like "I was busy this morning."

[0412] Input: Message from user

[0413] Output: None

[0414] Step 5:

[0415] The device forwards the message to the server via the LINE API, and Node.js is used to receive and analyze the message.

[0416] Input: User's message

[0417] Output: Message received by the server

[0418] Step 6:

[0419] The server parses the received messages, extracting message content and metadata (sender information, timestamp, etc.), and analyzes the emotional state using Python's NLTK library.

[0420] Input: The message received by the server

[0421] Output: Parsed message content and sentiment data

[0422] Step 7:

[0423] The server extracts conversation data and emotion data and stores it in a database. Data is added to the database using SQLAlchemy.

[0424] Input: Parsed message content and sentiment data

[0425] Output: Conversation data and emotion data stored in a database

[0426] Step 8:

[0427] The server periodically retrieves conversation data from the database. This process is automated by a scheduler, and the data is analyzed using TensorFlow and PyTorch.

[0428] Input: Conversation data retrieved from the database

[0429] Output: Analysis results

[0430] Step 9:

[0431] The server uses OpenAI's Fine Tuning API to train a user-specific AI model, then optimizes the model based on the analysis results.

[0432] Input: Analysis results

[0433] Output: A trained artificial intelligence model

[0434] Step 10:

[0435] After the server completes the training, it stores the updated artificial intelligence model in a database.

[0436] Input: A trained artificial intelligence model

[0437] Output: Artificial intelligence model stored in a database

[0438] Step 11:

[0439] A user invites a clonebot to join a group chat in a communication application and allows it to interact with other users.

[0440] Input: None

[0441] Output: None

[0442] Step 12:

[0443] A friend sends a question in a group chat: "What did you do over the weekend?"

[0444] Input: Message from a friend

[0445] Output: None

[0446] Step 13:

[0447] The device forwards this message to the server via the LINE API, and receives the message using Node.js.

[0448] Input: Message from a friend

[0449] Output: Message received by the server

[0450] Step 14:

[0451] The server takes into account past conversation data and emotional state and generates appropriate replies using OpenAI's GPT-3 API.

[0452] Input: Received messages, past conversation data, emotion data

[0453] Output: The generated reply

[0454] Step 15:

[0455] The server generates a reply and sends it to the user's device via the LINE API, where it is displayed in the group chat.

[0456] Input: Generated reply

[0457] Output: Reply sent to the user's device

[0458] This step allows users to enjoy a natural, interactive dialogue experience that reflects their own emotions and speaking style.

[0459] (Application example 2)

[0460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0461] In modern autonomous vehicles, users want a more comfortable and stress-free driving experience. However, conventional autonomous driving systems lack the ability to engage in natural dialogue with users and provide emotionally appropriate responses. This can lead to distrust and frustration among users. Furthermore, the lack of a dialogue model tailored to individual users results in repetitive, uniform responses, leading to lower user satisfaction.

[0462] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0463] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a sentiment analysis means for analyzing the user's sentiment, a means for adjusting the reply content based on the sentiment analysis, and a means for sending the generated replies to the user terminal. This enables recognition of the user's sentiment and a natural dialogue appropriate to that sentiment. Furthermore, training a user-specific dialogue model enables dialogue optimized for each individual user, improving the user experience.

[0464] "Initial setting means" refers to the means by which a user accesses the system and makes the necessary settings.

[0465] "Conversation data" refers to information such as messages and voices exchanged between a user and a system.

[0466] "Collection and storage means" refers to the means for accumulating and managing conversation data obtained from users.

[0467] "Analysis means" refers to a means for analyzing collected conversation data and learning the user's speaking style and context.

[0468] A "training means" is a means for generating a user-specific dialogue model based on the analyzed data.

[0469] The "generation means" is a means for generating conversation replies using the generated user-specific model.

[0470] "Emotion analysis means" is a means for estimating and analyzing emotions from user messages.

[0471] The "reply adjustment means" is a means for appropriately adjusting the reply content generated based on the result of emotion analysis.

[0472] The "transmitting means" is a means for transmitting the generated reply to the user terminal.

[0473] "Virtual driver" refers to a virtual driver bot that interacts with the user inside an autonomous vehicle and provides information about the driving situation and destination.

[0474] This invention relates to the construction of a system for use in autonomous vehicles, specifically for analyzing user emotions and providing natural dialogue accordingly. The system consists of the following main components:

[0475] User Device

[0476] The user terminal is a smartphone or an in-vehicle infotainment system, and is the device through which the user accesses the system and performs initial setup and daily interaction. Initial setup includes installing applications and setting up a user-specific profile. The "initial setup means" allows the user to register the necessary information for the system.

[0477] server

[0478] The server is the main device that stores and manages the collected conversation data, and is responsible for analysis and training. Specifically, it performs the following processes:

[0479] Conversation data collection and storage: Conversation data with the user is acquired and stored in a database. For example, a message such as "The roads are crowded" is stored.

[0480] Analysis method: Analyzes collected conversation data to learn about the user's speaking style and context. This analysis is done using machine learning algorithms and natural language processing techniques.

[0481] Training method: A dialogue model specific to the user is generated based on the analyzed data. This training is performed using OpenAI's Fine tuning API, among other tools.

[0482] Sentiment analysis: The Sentimental library is used to estimate and analyze the sentiment of users' messages. For example, if a user says "I'm a little tired," the sentiment is analyzed as negative.

[0483] Reply adjustment: The generated reply content is adjusted based on the results of sentiment analysis, allowing for the generation of appropriate reply content according to the user's emotions.

[0484] Transmission method: The generated reply is sent to the user's device. For example, a message such as "Are you tired? Shall we take a break at the next service area?" is sent to the user's device.

[0485] Specific examples

[0486] For example, if a user in an autonomous vehicle says, "Could you please speed up a bit?", the system will analyze the message using its "emotion analysis means" and generate an appropriate reply if necessary. The reply generated will be, "We can't speed up, but we are ensuring a safe route," and sent to the user's device. In this way, users can experience natural dialogue that reflects their emotions.

[0487] Examples of prompts include:

[0488] "If a user says they're tired, suggest an appropriate place to rest."

[0489] "If the user's sentiment is positive, recommend them the next tourist attraction they should visit."

[0490] This invention allows users to travel more comfortably and stress-free in self-driving vehicles, and is expected to improve user satisfaction by providing emotion-based dialogue.

[0491] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0492] Step 1:

[0493] Initial Setup

[0494] The user installs the application on their own device and performs initial setup. Specifically, they start the application and enter the necessary information. The entered information is sent to the server via the "initial setup means." The server uses this information to generate a user-specific profile and stores it in a database. Input: User information, Output: User profile.

[0495] Step 2:

[0496] Conversation data collection

[0497] The user starts a conversation on the device. The device sends the user's message to the server via the LINE API or the in-car infotainment system. The server stores the received message in a database using the "conversation data collection and storage means." Input: User's message, Output: Stored conversation data.

[0498] Step 3:

[0499] Analysis of conversation data

[0500] The server periodically retrieves conversation data from the database. The "analysis means" analyzes the collected conversation data and learns the user's speaking style and context. In this process, machine learning algorithms (e.g., natural language processing technology) are used. Input: saved conversation data, Output: analysis results.

[0501] Step 4:

[0502] Training the dialogue model

[0503] A "training method" generates a user-specific dialogue model based on the analyzed data. This training is performed using OpenAI's Fine tuning API, etc. After training, the generated model is saved in a database. Input: Analysis results, Output: User-specific dialogue model.

[0504] Step 5:

[0505] Emotion analysis

[0506] The server analyzes new messages received from users using the "sentiment analysis means." This analysis uses the Sentimental library to determine whether the user's emotion is "positive" or "negative." For example, if the message is "tired," it is determined to be negative. Input: User's message, Output: Emotion determination result.

[0507] Step 6:

[0508] Generate and moderate replies

[0509] The "generation means" generates a conversational reply using a dialogue model dedicated to the user. The "reply adjustment means" then adjusts the reply content based on the results of the emotion analysis. For example, if the user says "I'm a little tired," the reply content is adjusted to "Shall we take a break at the next service area?" Input: Emotion determination result and user-specific dialogue model, Output: Adjusted reply.

[0510] Step 7:

[0511] Sending a reply

[0512] The "transmission means" sends the generated and adjusted reply to the user's terminal. The user can check the reply through the terminal. Input: adjusted reply, Output: message displayed on the user's terminal.

[0513] Step 8:

[0514] User experience updates

[0515] The collected new conversation data and the results of sentiment analysis are periodically learned and updated by the "analysis means" and "training means." This ensures that the user-specific dialogue model always reflects the latest conversation patterns and emotional state. Input: New conversation data and sentiment analysis results, Output: Updated dialogue model.

[0516] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0517] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0518] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0519] [Second embodiment]

[0520] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0521] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0522] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0523] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0524] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0526] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0527] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0528] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0529] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0530] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0531] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0532] This invention relates to a dialogue system using a clonebot personalized for each user. This system learns the user's speaking style and way of thinking, generates an AI model dedicated to the user, and realizes natural dialogue that mimics the user's speaking style.

[0533] System Configuration

[0534] This system consists of the following main components:

[0535] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[0536] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[0537] 3. Database: Stores and manages user conversation data and trained models.

[0538] Initial setting method

[0539] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[0540] Server: Receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[0541] Conversation data collection and storage method

[0542] User: Interacts with the clonebot on a daily basis, for example sending a message saying "I was busy this morning."

[0543] Device: Sends a message to the server via the LINE API.

[0544] Server: Analyzes received messages and stores conversation data (content, sender information, timestamp, etc.) in a database.

[0545] Analysis and learning tools

[0546] Server: Periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0547] Server: Trains a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[0548] Server: After training is complete, save the updated model to the database.

[0549] Interacting with Clonebots

[0550] Users: Invite your clonebot to join a group chat and have it participate in conversations with other users.

[0551] Friends: Asking in a group chat, "What did you do over the weekend?"

[0552] Device: Sends the received message to the server via the LINE API.

[0553] Server: Analyzes the received message and generates a reply using a user-specific AI model. Specifically, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[0554] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[0555] Specific examples

[0556] For example, if Tanaka were to use CloneBot, the following conversations would be possible: If Tanaka sends a message to CloneBot on LINE saying, "Last night's movie was great," CloneBot would reply, "Yeah, I enjoyed it too. What shall we see next?" Also, if Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on Tanaka's behalf, saying, "I went camping over the weekend."

[0557] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[0558] The processing flow will be explained below.

[0559] Step 1:

[0560] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[0561] Step 2:

[0562] Device: Sends the user's message to the server via the LINE API.

[0563] Step 3:

[0564] Server: Receives the initialization message and generates a new user identification (user ID).

[0565] Step 4:

[0566] Server: Save the new user's profile (user ID, initial settings information) in the database.

[0567] Step 5:

[0568] Server: Generates the initial setup completion message and sends it to the user.

[0569] Step 6:

[0570] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[0571] Step 7:

[0572] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0573] Step 8:

[0574] Device: Sends messages from users to the server via the LINE API.

[0575] Step 9:

[0576] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[0577] Step 10:

[0578] Server: Save the extracted conversation data in a database.

[0579] Step 11:

[0580] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style and context.

[0581] Step 12:

[0582] Server: Generates or updates a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[0583] Step 13:

[0584] Server: After training is complete, the updated AI model is saved in the database.

[0585] Step 14:

[0586] User: Invite your clonebot to join a group chat and have it interact with other users.

[0587] Step 15:

[0588] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0589] Step 16:

[0590] Device: Send this message to the server via the LINE API.

[0591] Step 17:

[0592] Server: Analyzes the received message and generates an appropriate reply. Creates natural responses based on the user's past conversation data and training results.

[0593] Step 18:

[0594] Server: The generated reply message is sent to the user's device via the LINE API.

[0595] Step 19:

[0596] Terminal: Clonebot's reply message appears in group chat.

[0597] For example, if Mr. Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the clonebot will generate a reply saying, "Yes, it was really interesting. What shall we see next?" In this way, natural dialogue that reflects the user's characteristics is realized.

[0598] Example 1

[0599] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0600] Conventional dialogue systems have struggled to achieve natural dialogue that takes into account the individual speaking style and context of each user. Furthermore, no dialogue system to date has used a generative AI model that can accurately learn and mimic a user's speaking style and way of thinking. This has made it difficult for users to enjoy dialogue that reflects their own individuality, and the system's responses can feel unnatural.

[0601] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0602] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific generative AI model based on the learned data, a means for generating conversation replies using the generated user-specific generative AI model, and a means for transmitting the generated replies to the user terminal, thereby enabling the user to enjoy natural conversations that reflect their own speaking style and way of thinking.

[0603] "Initialization means" refers to the process or function by which a user registers with the system.

[0604] "Conversation data" refers to the messages and dialogue exchanged between the user and the clonebot.

[0605] "Means of collection and storage" refers to the functions and processes for collecting conversation data with users and storing it in a database.

[0606] "Means of analysis" refers to the technologies and algorithms used to analyze collected conversational data and understand the user's speaking style and context.

[0607] "Means of learning" refers to the process of training users' speaking and expressing themselves based on analytical data.

[0608] A "generative AI model" refers to an artificial intelligence model that is trained to mimic a user's speech style and context.

[0609] "Training method" refers to the process by which a generative AI model is trained based on user conversation data.

[0610] "Means for generating replies" refers to the ability to use a generative AI model to automatically create replies that mimic the way a user would speak.

[0611] "User Device" refers to the device a user uses to interact with a clonebot, including a smartphone or computer.

[0612] "Means for sending" refers to the function or process for sending the generated reply message to the user terminal.

[0613] "Means of updating" refers to the process of periodically retraining the generative AI model based on collected conversational data and updating the model based on the latest information.

[0614] A "clonebot" is an AI assistant that is trained to mimic the way a user speaks and thinks.

[0615] "Group chat" refers to a chat function that allows multiple users to interact at the same time.

[0616] "Other users' messages" refers to messages sent by users other than the CloneBot user within a group chat.

[0617] This invention relates to a dialogue system using a user-specific clonebot, which learns the user's speaking style and way of thinking, generates a user-specific generative AI model, and realizes natural dialogue that mimics the user's speaking style.

[0618] System Configuration

[0619] This system consists of the following main components:

[0620] 1. User device: A device such as a smartphone or PC on which the user interacts with the clonebot using a communication app.

[0621] 2. Server: Hosts the clonebot generative AI model and performs data processing, analysis, and training.

[0622] 3. Database: Stores and manages user conversation data and trained models.

[0623] Initial setting method

[0624] User: The user adds the clone bot as a friend in the communication app and performs the initial setup. Specifically, the user adds the clone bot by scanning the QR code and sends a message saying, "Make my clone bot."

[0625] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[0626] Conversation data collection and storage method

[0627] User: The user interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0628] Device: The user device sends this message to the server through the API.

[0629] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[0630] Analysis and learning tools

[0631] Server: The server periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0632] Server: The server trains a generative AI model specific to the user based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[0633] Server: After training is complete, save the updated model to the database.

[0634] Interacting with Clonebots

[0635] User: Users can invite clonebots to group chats and have them participate in conversations with other users.

[0636] Friends: Friends ask in group chats, "What did you do over the weekend?"

[0637] Device: The user device sends the received message to the server via the API.

[0638] Server: The server analyzes the received message and generates a reply using a generative AI model specific to the user. For example, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[0639] Server: The generated reply message is sent to the user's device via the API and displayed in the group chat.

[0640] Specific examples

[0641] For example, when a user uses this system, the following dialogue is possible:

[0642] If a user sends a message to CloneBot in a communication app saying, "The movie last night was great," CloneBot will reply, "I enjoyed it too. What shall we watch next?" If a user's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on the user's behalf, saying, "I went camping over the weekend."

[0643] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[0644] Examples of prompt statements

[0645] For example, "Generate a scenario in which you talk to a clonebot about a movie you saw yesterday."

[0646] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0647] Step 1:

[0648] Initial Setup

[0649] User: The user adds the clone bot as a friend in a communication app and sends a message saying, "Make a clone bot of me."

[0650] Input: Default message from the user

[0651] Output: Initialization request sent to server

[0652] What it does: The user scans the QR code in the app, adds the clonebot as a friend, and then types and sends a message saying, "Make a clonebot of me."

[0653] Step 2:

[0654] User Registration

[0655] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to a database.

[0656] Input: Initial Setup Request

[0657] Output: User ID and profile are saved in the database.

[0658] What happens: The server analyzes the received message and stores the new user's information in the database. It generates a user ID using an ID generation algorithm and stores the profile information.

[0659] Step 3:

[0660] Conversation data collection

[0661] User: The user interacts with the clonebot on a daily basis, sending messages such as "I was busy this morning."

[0662] Input: Conversational message from user

[0663] Output: The message is sent to the server

[0664] Specific operation: The user interacts with the clonebot and sends messages, which the device forwards to the server via API.

[0665] Step 4:

[0666] Saving conversation data

[0667] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[0668] Input: Received message

[0669] Output: Parsed conversation data is saved in a database

[0670] Specific operation: The server analyzes the contents of the message, adds sender information and a timestamp, and stores the message in a database.

[0671] Step 5:

[0672] Analysis of conversation data

[0673] Server: The server periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[0674] Input: Conversation data retrieved from the database

[0675] Output: Analysis results

[0676] How it works: The server periodically retrieves conversation data stored in the database and analyzes it using machine learning algorithms. The analysis results include the user's unique speaking style and expressions.

[0677] Step 6:

[0678] Training a generative AI model

[0679] Server: The server trains a user-specific generative AI model based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[0680] Input: Analysis results

[0681] Output: A trained generative AI model

[0682] Specific operation: The server uses the acquired analysis results to train the generative AI model. It uses OpenAI's Fine tuning API to optimize the model for the user.

[0683] Step 7:

[0684] Saving the generated AI model

[0685] Server: Stores the trained generative AI model in a database.

[0686] Input: A trained generative AI model

[0687] Output: Generative AI model stored in a database

[0688] What happens: The trained model is saved in a database and ready to be used for future dialogue generation.

[0689] Step 8:

[0690] Interacting with Clonebots

[0691] User: Users can invite clonebots to group chats to participate in conversations with other users.

[0692] Input: Message from other users

[0693] Output: A reply message is sent by the clonebot

[0694] What it does: A friend asks, "What did you do over the weekend?" The device sends this message to a server via an API. The server analyzes the message and uses a trained generative AI model to generate a natural-sounding response, such as "I went camping over the weekend." The server then sends the generated reply message to the user's device, where it appears in the group chat.

[0695] (Application example 1)

[0696] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] Current food delivery services lack personalized recommendations and interactions based on users' individual preferences, which makes the ordering process cumbersome, as users are presented with the same operations and options every time. Furthermore, suggestions do not take into account users' food and beverage preferences or eating history, which makes it time-consuming to find more suitable options.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0699] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a means for learning the user's preferences using the above means and recommending meals and drinks, and a means for transmitting the generated recommendations to the user terminal, thereby enabling the user to receive personalized dialogue and recommendations based on their individual preferences.

[0700] "User-registered initial setting means" is a function that allows users to input initial information into the system and customize the system individually.

[0701] "Means for collecting and storing conversation data with users" refers to the function of recording and storing the content of conversations users have with the system.

[0702] "Means of analyzing collected conversation data and learning the user's speaking style and context" refers to a function that understands the user's speaking style, unique expressions, and context based on recorded conversation data.

[0703] "Means to train user-specific models based on learned data" refers to the ability to create individually personalized AI models based on the user's learned speech patterns and context.

[0704] "Means for generating conversation replies using a generated user-specific model" is a function that uses a user-specific AI model created to generate replies tailored to the user's style.

[0705] "Means of learning the user's preferences using the above means and recommending meals and drinks" refers to a function that analyzes the user's favorite dishes and drinks based on the user's conversation data and past ordering history, and makes recommendations accordingly.

[0706] "Means for sending the generated recommendation content to the user's device" refers to a function that notifies the user's device of the recommendation content created by the AI ​​model.

[0707] The present invention relates to a system that provides a food delivery assistant customized for each user. The system collects and analyzes user conversation data, generates a user-specific AI model, and recommends meals and beverages based on the user's preferences.

[0708] System Configuration

[0709] The system mainly consists of the following components:

[0710] 1. User device: The user interacts with the clonebot on a device such as a smartphone or computer.

[0711] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[0712] 3. Database: Stores and manages user conversation data and trained models.

[0713] Hardware and software used

[0714] Hardware: Smartphones, cloud servers

[0715] Software: LINE API, OpenAI Fine-tuning API

[0716] User device behavior

[0717] On the user's device, the user adds the clonebot as a friend through LINE or a dedicated app and starts the initial setup process by scanning the QR code and sending a message saying, "Help me deliver food."

[0718] Server Operation

[0719] The server receives the initial setup request sent from the device, generates a user ID, and saves the profile in a database. It then collects and saves daily conversation data and periodically analyzes it.

[0720] Analysis and Training

[0721] The server analyzes the user's speaking style and preferences based on the collected conversation data, and uses the OpenAI Fine-tuning API to train a user-specific AI model, which is then used to generate natural-sounding dialogue based on the user's preferences and context.

[0722] Dietary and Food Recommendations

[0723] The server learns the user's preferences based on their conversation data and past ordering history, so that when the user orders food and drink, it can make recommendations that are tailored to the user's preferences.

[0724] Interacting with Clonebots

[0725] If a user wants to order a specific meal, they can ask the clonebot, "What would you like to eat today?" and the clonebot can make a suggestion based on the user's preferences, such as, "I feel like eating Italian today. That pasta with meat sauce is especially delicious." Also, if the user wants to order the same dish again, the order details are saved, so they can be reordered with a simple command, such as, "Order the pasta with meat sauce again."

[0726] Specific examples

[0727] For example, if a user asks the clonebot, "What kind of food do you recommend today?", it will suggest, "You seem to be coming down with a cold today. How about some stomach-friendly Japanese food?" If the user asks, "Which restaurant specifically?", it will respond, "What's the nearest Japanese restaurant? It was delicious the last time I ordered."

[0728] Prompt Sentence Examples

[0729] "What dish do you recommend today?"

[0730] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[0731] In this way, the system of the present invention can provide personalized interactions and recommendations based on the user's individual preferences, significantly improving the food delivery ordering process.

[0732] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0733] Step 1:

[0734] Users add the clone bot as a friend via LINE or a dedicated app and send it a message saying, "Help with food delivery."

[0735] Input: User's initial setup request

[0736] Output: Notification of the start of the initialization process

[0737] How it works: The user scans the QR code, becomes friends with the clonebot in the LINE app, sends an initial setup message, and sends a request to the server.

[0738] Step 2:

[0739] The server receives the initial setup request, generates a user ID and saves the profile in a database.

[0740] Input: Initial Setup Request

[0741] Output: User ID and profile creation and saving

[0742] Behavior: The server analyzes the initial configuration request data, generates a new user ID, and stores the generated user ID and profile in the database.

[0743] Step 3:

[0744] Collect and store daily conversation data with users.

[0745] Input: User's daily conversation data

[0746] Output: Saved conversation data

[0747] Operation: The device sends the user's conversation data to the server via the LINE API. The server stores the received data in a database.

[0748] Step 4:

[0749] The server analyzes the collected conversation data and learns the user's speaking style and preferences.

[0750] Input: Saved conversation data

[0751] Output: Analyzed speaking style and preference data

[0752] How it works: The server periodically retrieves conversation data from the database and uses natural language processing techniques to analyze the user's speaking style, context, and preferences.

[0753] Step 5:

[0754] The server trains a user-specific AI model based on the learned data.

[0755] Input: Parsed data

[0756] Output: A trained, user-specific AI model

[0757] How it works: The server uses the analyzed data to train a user-specific CloneBot model using the OpenAI Fine-tuning API.

[0758] Step 6:

[0759] The server generates conversations and recommendation replies using the generated user-specific model.

[0760] Input: Trained AI model, user request

[0761] Output: Natural conversation and recommendation replies

[0762] How it works: The server receives a user request and uses the trained model to generate a natural-sounding response based on the user's speaking style and preferences.

[0763] Step 7:

[0764] The server sends the generated reply to the user terminal.

[0765] Input: Generated reply

[0766] Output: Reply displayed on the user's terminal

[0767] Behavior: The server sends the generated reply to the user's device via the LINE API, and the reply is displayed on the user's device.

[0768] Specific examples

[0769] Example prompt sentence:

[0770] "What dish do you recommend today?"

[0771] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[0772] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0773] The present invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining it with an emotion engine.

[0774] System Configuration

[0775] This system consists of the following main components:

[0776] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[0777] 2. Server: Hosts the CloneBot AI model and emotion engine, and performs data processing, analysis, and training.

[0778] 3. Database: Stores and manages user conversation data and trained models.

[0779] 4. Emotion Engine: An engine that analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[0780] Initial setting method

[0781] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[0782] Server: Receives the initial setup request and generates a new user identification (User ID), which stores the new user's profile in the database.

[0783] Conversation data collection and storage method

[0784] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0785] Device: Sends a message to the server via the LINE API.

[0786] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.). Additionally, it uses an emotion engine to analyze the emotional state of the message.

[0787] Server: Stores the extracted conversation data and emotion data in a database.

[0788] Analysis and learning tools

[0789] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[0790] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[0791] Server: After training is complete, save the updated model to the database.

[0792] Interacting with Clonebots

[0793] User: Invite your clonebot to join a group chat and have it interact with other users.

[0794] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0795] Device: Send this message to the server via the LINE API.

[0796] Server: Analyzes received messages and generates appropriate replies. Creates natural responses based on the user's past conversation data, training results, and emotional state. Using an emotion engine, it recognizes emotions such as "fun" or "difficult" and generates appropriate replies.

[0797] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[0798] Specific examples

[0799] For example, if Tanaka sends a message to CloneBot saying, "The movie yesterday was great," CloneBot will recognize the emotion as "enjoyment" and reply, "Yeah, it was really great. What shall we watch next?" If Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can generate a response that includes an emotional expression, such as, "I went camping over the weekend. It was so refreshing!"

[0800] In this way, the present invention allows users to enjoy a natural and interactive dialogue experience that reflects their own emotions and speaking style. Furthermore, by combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[0801] The processing flow will be explained below.

[0802] Step 1:

[0803] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[0804] Step 2:

[0805] Device: Sends the user's message to the server via the LINE API.

[0806] Step 3:

[0807] Server: Receives the initialization message and generates a new user identification (user ID).

[0808] Step 4:

[0809] Server: Save the new user's profile (user ID, initial settings information) in the database.

[0810] Step 5:

[0811] Server: Generates the initial setup completion message and sends it to the user.

[0812] Step 6:

[0813] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[0814] Step 7:

[0815] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[0816] Step 8:

[0817] Device: Sends messages from users to the server via the LINE API.

[0818] Step 9:

[0819] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[0820] Step 10:

[0821] Server: Analyzes the user's emotional state from the received message using the emotion engine. For example, recognizes the emotion "stress" from the keyword "busy."

[0822] Step 11:

[0823] Server: Stores the extracted conversation data and emotion data in a database.

[0824] Step 12:

[0825] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[0826] Step 13:

[0827] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[0828] Step 14:

[0829] Server: After training is complete, the updated AI model is saved in the database.

[0830] Step 15:

[0831] User: Invite your clonebot to join a group chat and have it interact with other users.

[0832] Step 16:

[0833] Friends: Send a question in a group chat: "What did you do over the weekend?"

[0834] Step 17:

[0835] Device: Send this message to the server via the LINE API.

[0836] Step 18:

[0837] Server: Analyzes the received message and generates an appropriate reply. It creates natural responses based on the user's past conversation data, training results, and emotional state. For example, it uses the emotion engine to recognize the emotion "fun" and generates a reply such as, "I went camping this weekend. It was so refreshing!"

[0838] Step 19:

[0839] Server: The generated reply message is sent to the user's device via the LINE API.

[0840] Step 20:

[0841] Terminal: Clonebot's reply message appears in group chat.

[0842] As a concrete example, if Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the emotion engine will recognize the emotion of "enjoyment" from the keyword "interesting." The server will generate a reply that reflects this emotion: "Yes, it was really interesting. What shall we watch next?" and send it to Tanaka's device via the LINE API. In this way, natural dialogue that reflects the user's emotions and speaking style is realized.

[0843] Example 2

[0844] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] Conventional dialogue systems have struggled to achieve natural dialogue that reflects the user's emotional state. Furthermore, there are limited means to individually learn a user's speaking style and context and generate a model dedicated to that user. Furthermore, mechanisms for effectively analyzing and storing collected conversation data and regularly updating models are inadequate.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0847] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific artificial intelligence model based on the learned data, a means for generating conversation replies using the generated user-specific artificial intelligence model, a means for sending the generated replies to the user terminal, and a means for analyzing the emotional state of the message using an emotion engine. This makes it possible to realize natural, interactive dialogue that reflects the user's emotions and speaking style, and to continuously generate and update a user-specific model.

[0848] "User" refers to an individual or end user of an interactive system.

[0849] The "initial setting means" includes a setting procedure that a user must perform before using the dialogue system.

[0850] "Conversation Data" refers to information, including text messages, that a user sends and receives through an interactive system.

[0851] "Means for collection and storage" refers to the function by which the server receives conversation data, analyzes it, and stores it in a database.

[0852] "Means for analyzing" refers to the methods and technologies that use collected conversational data to analyze a user's speaking style, context, and emotional state.

[0853] "Learning means" refers to the process of accumulating information about users based on the analysis results and using it to improve the quality of future interactions.

[0854] "Artificial intelligence model" refers to a computer program that uses machine learning and natural language processing techniques to mimic a user's speech patterns and context.

[0855] "Training" refers to the process of using collected and learned data to improve and optimize an artificial intelligence model.

[0856] "Means for generating replies" refers to the ability to automatically generate replies to users' messages using a trained artificial intelligence model.

[0857] An "emotion engine" is a software tool that analyzes emotions from users' messages and uses that information to reflect them in the content of the conversation.

[0858] "Communications application" refers to application software that allows a user to send and receive messages.

[0859] This invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining an emotion engine. The main components of this system include a user terminal, a server, a database, and an emotion engine.

[0860] System Configuration

[0861] 1. User terminal: The user terminal of this system is a device such as a smartphone or PC, which the user uses to interact with the clonebot through the LINE app.

[0862] Example: A user sends a message to a clone bot on the LINE app saying, "The movie yesterday was great."

[0863] 2. Server: The server hosts the CloneBot's artificial intelligence model and emotion engine, and performs data processing, analysis, and training.

[0864] Software used: Node.js, Python, NLTK, TensorFlow, PyTorch, OpenAI API

[0865] Example: A server parses messages received from users and analyzes their emotional state using Python's NLTK.

[0866] 3. Database: The database stores and manages user conversation data and trained models.

[0867] Software used: MySQL, SQLAlchemy

[0868] 4. Emotion Engine: The emotion engine analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[0869] Initial setting method

[0870] Users add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, users add the clone bot by scanning the QR code and send a message saying, "Make my clone bot."

[0871] The server receives the initial setup request and generates a new user identity (user ID), which saves the new user's profile in the database. It uses the UUID library to generate a unique ID and saves it in the MySQL database.

[0872] Conversation data collection and storage method

[0873] Users exchange messages with the clonebot on a daily basis, for example, sending a message saying, "I was busy this morning."

[0874] The device forwards the sent message to the server via the LINE API.

[0875] The server parses the received messages and extracts metadata such as content, sender information, and timestamps. It also analyzes the sentiment of the messages using Python's NLTK. The analyzed data is stored in a database using SQLAlchemy.

[0876] Analysis and learning tools

[0877] The server periodically retrieves conversation data from the database. This is automated using a scheduler. The retrieved data is analyzed using TensorFlow or PyTorch to generate and update an AI model specific to the user. OpenAI's Fine Tuning API is used for training. The generated and updated model is stored in the database.

[0878] Interacting with Clonebots

[0879] Users can invite clonebots to join group chats in communication applications to interact with other users.

[0880] A friend sends a question in a group chat: "What did you do over the weekend?"

[0881] The device forwards this message to the server via the LINE API.

[0882] The server uses OpenAI's GPT-3 API to generate appropriate replies, taking into account past conversation data and emotional states. The replies are then sent to the user's device via the LINE API and displayed in the group chat.

[0883] Examples of specific examples and prompts

[0884] For example, if a user sends a message to a clonebot on the LINE app saying, "Last night's movie was great," the server will recognize the emotion as "enjoyment" and generate a reply saying, "Yeah, it was really great. What should we watch next?"

[0885] Example prompt sentence:

[0886] "Make a clone bot of me."

[0887] "The movie yesterday was interesting."

[0888] "What did you do over the weekend?"

[0889] In this way, the present invention can realize natural and interactive dialogue that reflects the user's emotions and speaking style. By combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[0890] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0891] Step 1:

[0892] The user adds the clone bot to their friends on the LINE app and sets up the initial settings. At this stage, the user scans the QR code and sends a message saying, "Make my clone bot."

[0893] Input: QR code scan and initial message

[0894] Output: Initialization request

[0895] Step 2:

[0896] The server receives an initial setup request through the LINE API. It uses Node.js to parse this request and generate a new user ID. It uses the UUID library to generate a unique ID.

[0897] Input: Initial Setup Request

[0898] Output: User ID

[0899] Step 3:

[0900] The server stores the generated user ID in a MySQL database and creates a profile for the user, using SQLAlchemy for database operations.

[0901] Input: User ID

[0902] Output: User profile stored in the database

[0903] Step 4:

[0904] Users can exchange messages with the clone bot on a daily basis using the LINE app, for example, sending a message like "I was busy this morning."

[0905] Input: Message from user

[0906] Output: None

[0907] Step 5:

[0908] The device forwards the message to the server via the LINE API, and Node.js is used to receive and analyze the message.

[0909] Input: User's message

[0910] Output: Message received by the server

[0911] Step 6:

[0912] The server parses the received messages, extracting message content and metadata (sender information, timestamp, etc.), and analyzes the emotional state using Python's NLTK library.

[0913] Input: The message received by the server

[0914] Output: Parsed message content and sentiment data

[0915] Step 7:

[0916] The server extracts conversation data and emotion data and stores it in a database. Data is added to the database using SQLAlchemy.

[0917] Input: Parsed message content and sentiment data

[0918] Output: Conversation data and emotion data stored in a database

[0919] Step 8:

[0920] The server periodically retrieves conversation data from the database. This process is automated by a scheduler, and the data is analyzed using TensorFlow and PyTorch.

[0921] Input: Conversation data retrieved from the database

[0922] Output: Analysis results

[0923] Step 9:

[0924] The server uses OpenAI's Fine Tuning API to train a user-specific AI model, then optimizes the model based on the analysis results.

[0925] Input: Analysis results

[0926] Output: A trained artificial intelligence model

[0927] Step 10:

[0928] After the server completes the training, it stores the updated artificial intelligence model in a database.

[0929] Input: A trained artificial intelligence model

[0930] Output: Artificial intelligence model stored in a database

[0931] Step 11:

[0932] A user invites a clonebot to join a group chat in a communication application and allows it to interact with other users.

[0933] Input: None

[0934] Output: None

[0935] Step 12:

[0936] A friend sends a question in a group chat: "What did you do over the weekend?"

[0937] Input: Message from a friend

[0938] Output: None

[0939] Step 13:

[0940] The device forwards this message to the server via the LINE API, and receives the message using Node.js.

[0941] Input: Message from a friend

[0942] Output: Message received by the server

[0943] Step 14:

[0944] The server takes into account past conversation data and emotional state and generates appropriate replies using OpenAI's GPT-3 API.

[0945] Input: Received messages, past conversation data, emotion data

[0946] Output: The generated reply

[0947] Step 15:

[0948] The server generates a reply and sends it to the user's device via the LINE API, where it is displayed in the group chat.

[0949] Input: Generated reply

[0950] Output: Reply sent to the user's device

[0951] This step allows users to enjoy a natural, interactive dialogue experience that reflects their own emotions and speaking style.

[0952] (Application example 2)

[0953] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0954] In modern autonomous vehicles, users want a more comfortable and stress-free driving experience. However, conventional autonomous driving systems lack the ability to engage in natural dialogue with users and provide emotionally appropriate responses. This can lead to distrust and frustration among users. Furthermore, the lack of a dialogue model tailored to individual users results in repetitive, uniform responses, leading to lower user satisfaction.

[0955] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0956] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a sentiment analysis means for analyzing the user's sentiment, a means for adjusting the reply content based on the sentiment analysis, and a means for sending the generated replies to the user terminal. This enables recognition of the user's sentiment and a natural dialogue appropriate to that sentiment. Furthermore, training a user-specific dialogue model enables dialogue optimized for each individual user, improving the user experience.

[0957] "Initial setting means" refers to the means by which a user accesses the system and makes the necessary settings.

[0958] "Conversation data" refers to information such as messages and voices exchanged between a user and a system.

[0959] "Collection and storage means" refers to the means for accumulating and managing conversation data obtained from users.

[0960] "Analysis means" refers to a means for analyzing collected conversation data and learning the user's speaking style and context.

[0961] A "training means" is a means for generating a user-specific dialogue model based on the analyzed data.

[0962] The "generation means" is a means for generating conversation replies using the generated user-specific model.

[0963] "Emotion analysis means" is a means for estimating and analyzing emotions from user messages.

[0964] The "reply adjustment means" is a means for appropriately adjusting the reply content generated based on the result of emotion analysis.

[0965] The "transmitting means" is a means for transmitting the generated reply to the user terminal.

[0966] "Virtual driver" refers to a virtual driver bot that interacts with the user inside an autonomous vehicle and provides information about the driving situation and destination.

[0967] This invention relates to the construction of a system for use in autonomous vehicles, specifically for analyzing user emotions and providing natural dialogue accordingly. The system consists of the following main components:

[0968] User Device

[0969] The user terminal is a smartphone or an in-vehicle infotainment system, and is the device through which the user accesses the system and performs initial setup and daily interaction. Initial setup includes installing applications and setting up a user-specific profile. The "initial setup means" allows the user to register the necessary information for the system.

[0970] server

[0971] The server is the main device that stores and manages the collected conversation data, and is responsible for analysis and training. Specifically, it performs the following processes:

[0972] Conversation data collection and storage: Conversation data with the user is acquired and stored in a database. For example, a message such as "The roads are crowded" is stored.

[0973] Analysis method: Analyzes collected conversation data to learn about the user's speaking style and context. This analysis is done using machine learning algorithms and natural language processing techniques.

[0974] Training method: A dialogue model specific to the user is generated based on the analyzed data. This training is performed using OpenAI's Fine tuning API, among other tools.

[0975] Sentiment analysis: The Sentimental library is used to estimate and analyze the sentiment of users' messages. For example, if a user says "I'm a little tired," the sentiment is analyzed as negative.

[0976] Reply adjustment: The generated reply content is adjusted based on the results of sentiment analysis, allowing for the generation of appropriate reply content according to the user's emotions.

[0977] Transmission method: The generated reply is sent to the user's device. For example, a message such as "Are you tired? Shall we take a break at the next service area?" is sent to the user's device.

[0978] Specific examples

[0979] For example, if a user in an autonomous vehicle says, "Could you please speed up a bit?", the system will analyze the message using its "emotion analysis means" and generate an appropriate reply if necessary. The reply generated will be, "We can't speed up, but we are ensuring a safe route," and sent to the user's device. In this way, users can experience natural dialogue that reflects their emotions.

[0980] Examples of prompts include:

[0981] "If a user says they're tired, suggest an appropriate place to rest."

[0982] "If the user's sentiment is positive, recommend them the next tourist attraction they should visit."

[0983] This invention allows users to travel more comfortably and stress-free in self-driving vehicles, and is expected to improve user satisfaction by providing emotion-based dialogue.

[0984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0985] Step 1:

[0986] Initial Setup

[0987] The user installs the application on their own device and performs initial setup. Specifically, they start the application and enter the necessary information. The entered information is sent to the server via the "initial setup means." The server uses this information to generate a user-specific profile and stores it in a database. Input: User information, Output: User profile.

[0988] Step 2:

[0989] Conversation data collection

[0990] The user starts a conversation on the device. The device sends the user's message to the server via the LINE API or the in-car infotainment system. The server stores the received message in a database using the "conversation data collection and storage means." Input: User's message, Output: Stored conversation data.

[0991] Step 3:

[0992] Analysis of conversation data

[0993] The server periodically retrieves conversation data from the database. The "analysis means" analyzes the collected conversation data and learns the user's speaking style and context. In this process, machine learning algorithms (e.g., natural language processing technology) are used. Input: saved conversation data, Output: analysis results.

[0994] Step 4:

[0995] Training the dialogue model

[0996] A "training method" generates a user-specific dialogue model based on the analyzed data. This training is performed using OpenAI's Fine tuning API, etc. After training, the generated model is saved in a database. Input: Analysis results, Output: User-specific dialogue model.

[0997] Step 5:

[0998] Emotion analysis

[0999] The server analyzes new messages received from users using the "sentiment analysis means." This analysis uses the Sentimental library to determine whether the user's emotion is "positive" or "negative." For example, if the message is "tired," it is determined to be negative. Input: User's message, Output: Emotion determination result.

[1000] Step 6:

[1001] Generate and moderate replies

[1002] The "generation means" generates a conversational reply using a dialogue model dedicated to the user. The "reply adjustment means" then adjusts the reply content based on the results of the emotion analysis. For example, if the user says "I'm a little tired," the reply content is adjusted to "Shall we take a break at the next service area?" Input: Emotion determination result and user-specific dialogue model, Output: Adjusted reply.

[1003] Step 7:

[1004] Sending a reply

[1005] The "transmission means" sends the generated and adjusted reply to the user's terminal. The user can check the reply through the terminal. Input: adjusted reply, Output: message displayed on the user's terminal.

[1006] Step 8:

[1007] User experience updates

[1008] The collected new conversation data and the results of sentiment analysis are periodically learned and updated by the "analysis means" and "training means." This ensures that the user-specific dialogue model always reflects the latest conversation patterns and emotional state. Input: New conversation data and sentiment analysis results, Output: Updated dialogue model.

[1009] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1010] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1011] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1012] [Third embodiment]

[1013] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1014] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1015] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1016] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1017] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1018] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1019] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1020] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1021] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1022] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1023] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1024] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1025] This invention relates to a dialogue system using a clonebot personalized for each user. This system learns the user's speaking style and way of thinking, generates an AI model dedicated to the user, and realizes natural dialogue that mimics the user's speaking style.

[1026] System Configuration

[1027] This system consists of the following main components:

[1028] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[1029] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[1030] 3. Database: Stores and manages user conversation data and trained models.

[1031] Initial setting method

[1032] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[1033] Server: Receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[1034] Conversation data collection and storage method

[1035] User: Interacts with the clonebot on a daily basis, for example sending a message saying "I was busy this morning."

[1036] Device: Sends a message to the server via the LINE API.

[1037] Server: Analyzes received messages and stores conversation data (content, sender information, timestamp, etc.) in a database.

[1038] Analysis and learning tools

[1039] Server: Periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1040] Server: Trains a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[1041] Server: After training is complete, save the updated model to the database.

[1042] Interacting with Clonebots

[1043] Users: Invite your clonebot to join a group chat and have it participate in conversations with other users.

[1044] Friends: Asking in a group chat, "What did you do over the weekend?"

[1045] Device: Sends the received message to the server via the LINE API.

[1046] Server: Analyzes the received message and generates a reply using a user-specific AI model. Specifically, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[1047] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[1048] Specific examples

[1049] For example, if Tanaka were to use CloneBot, the following conversations would be possible: If Tanaka sends a message to CloneBot on LINE saying, "Last night's movie was great," CloneBot would reply, "Yeah, I enjoyed it too. What shall we see next?" Also, if Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on Tanaka's behalf, saying, "I went camping over the weekend."

[1050] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[1051] The processing flow will be explained below.

[1052] Step 1:

[1053] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[1054] Step 2:

[1055] Device: Sends the user's message to the server via the LINE API.

[1056] Step 3:

[1057] Server: Receives the initialization message and generates a new user identification (user ID).

[1058] Step 4:

[1059] Server: Save the new user's profile (user ID, initial settings information) in the database.

[1060] Step 5:

[1061] Server: Generates the initial setup completion message and sends it to the user.

[1062] Step 6:

[1063] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[1064] Step 7:

[1065] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1066] Step 8:

[1067] Device: Sends messages from users to the server via the LINE API.

[1068] Step 9:

[1069] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[1070] Step 10:

[1071] Server: Save the extracted conversation data in a database.

[1072] Step 11:

[1073] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style and context.

[1074] Step 12:

[1075] Server: Generates or updates a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[1076] Step 13:

[1077] Server: After training is complete, the updated AI model is saved in the database.

[1078] Step 14:

[1079] User: Invite your clonebot to join a group chat and have it interact with other users.

[1080] Step 15:

[1081] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1082] Step 16:

[1083] Device: Send this message to the server via the LINE API.

[1084] Step 17:

[1085] Server: Analyzes the received message and generates an appropriate reply. Creates natural responses based on the user's past conversation data and training results.

[1086] Step 18:

[1087] Server: The generated reply message is sent to the user's device via the LINE API.

[1088] Step 19:

[1089] Terminal: Clonebot's reply message appears in group chat.

[1090] For example, if Mr. Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the clonebot will generate a reply saying, "Yes, it was really interesting. What shall we see next?" In this way, natural dialogue that reflects the user's characteristics is realized.

[1091] Example 1

[1092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1093] Conventional dialogue systems have struggled to achieve natural dialogue that takes into account the individual speaking style and context of each user. Furthermore, no dialogue system to date has used a generative AI model that can accurately learn and mimic a user's speaking style and way of thinking. This has made it difficult for users to enjoy dialogue that reflects their own individuality, and the system's responses can feel unnatural.

[1094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1095] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific generative AI model based on the learned data, a means for generating conversation replies using the generated user-specific generative AI model, and a means for transmitting the generated replies to the user terminal, thereby enabling the user to enjoy natural conversations that reflect their own speaking style and way of thinking.

[1096] "Initialization means" refers to the process or function by which a user registers with the system.

[1097] "Conversation data" refers to the messages and dialogue exchanged between the user and the clonebot.

[1098] "Means of collection and storage" refers to the functions and processes for collecting conversation data with users and storing it in a database.

[1099] "Means of analysis" refers to the technologies and algorithms used to analyze collected conversational data and understand the user's speaking style and context.

[1100] "Means of learning" refers to the process of training users' speaking and expressing themselves based on analytical data.

[1101] A "generative AI model" refers to an artificial intelligence model that is trained to mimic a user's speech style and context.

[1102] "Training method" refers to the process by which a generative AI model is trained based on user conversation data.

[1103] "Means for generating replies" refers to the ability to use a generative AI model to automatically create replies that mimic the way a user would speak.

[1104] "User Device" refers to the device a user uses to interact with a clonebot, including a smartphone or computer.

[1105] "Means for sending" refers to the function or process for sending the generated reply message to the user terminal.

[1106] "Means of updating" refers to the process of periodically retraining the generative AI model based on collected conversational data and updating the model based on the latest information.

[1107] A "clonebot" is an AI assistant that is trained to mimic the way a user speaks and thinks.

[1108] "Group chat" refers to a chat function that allows multiple users to interact at the same time.

[1109] "Other users' messages" refers to messages sent by users other than the CloneBot user within a group chat.

[1110] This invention relates to a dialogue system using a user-specific clonebot, which learns the user's speaking style and way of thinking, generates a user-specific generative AI model, and realizes natural dialogue that mimics the user's speaking style.

[1111] System Configuration

[1112] This system consists of the following main components:

[1113] 1. User device: A device such as a smartphone or PC on which the user interacts with the clonebot using a communication app.

[1114] 2. Server: Hosts the clonebot generative AI model and performs data processing, analysis, and training.

[1115] 3. Database: Stores and manages user conversation data and trained models.

[1116] Initial setting method

[1117] User: The user adds the clone bot as a friend in the communication app and performs the initial setup. Specifically, the user adds the clone bot by scanning the QR code and sends a message saying, "Make my clone bot."

[1118] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[1119] Conversation data collection and storage method

[1120] User: The user interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1121] Device: The user device sends this message to the server through the API.

[1122] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[1123] Analysis and learning tools

[1124] Server: The server periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1125] Server: The server trains a generative AI model specific to the user based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[1126] Server: After training is complete, save the updated model to the database.

[1127] Interacting with Clonebots

[1128] User: Users can invite clonebots to group chats and have them participate in conversations with other users.

[1129] Friends: Friends ask in group chats, "What did you do over the weekend?"

[1130] Device: The user device sends the received message to the server via the API.

[1131] Server: The server analyzes the received message and generates a reply using a generative AI model specific to the user. For example, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[1132] Server: The generated reply message is sent to the user's device via the API and displayed in the group chat.

[1133] Specific examples

[1134] For example, when a user uses this system, the following dialogue is possible:

[1135] If a user sends a message to CloneBot in a communication app saying, "The movie last night was great," CloneBot will reply, "I enjoyed it too. What shall we watch next?" If a user's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on the user's behalf, saying, "I went camping over the weekend."

[1136] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[1137] Examples of prompt statements

[1138] For example, "Generate a scenario in which you talk to a clonebot about a movie you saw yesterday."

[1139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1140] Step 1:

[1141] Initial Setup

[1142] User: The user adds the clone bot as a friend in a communication app and sends a message saying, "Make a clone bot of me."

[1143] Input: Default message from the user

[1144] Output: Initialization request sent to server

[1145] What it does: The user scans the QR code in the app, adds the clonebot as a friend, and then types and sends a message saying, "Make a clonebot of me."

[1146] Step 2:

[1147] User Registration

[1148] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to a database.

[1149] Input: Initial Setup Request

[1150] Output: User ID and profile are saved in the database.

[1151] What happens: The server analyzes the received message and stores the new user's information in the database. It generates a user ID using an ID generation algorithm and stores the profile information.

[1152] Step 3:

[1153] Conversation data collection

[1154] User: The user interacts with the clonebot on a daily basis, sending messages such as "I was busy this morning."

[1155] Input: Conversational message from user

[1156] Output: The message is sent to the server

[1157] Specific operation: The user interacts with the clonebot and sends messages, which the device forwards to the server via API.

[1158] Step 4:

[1159] Saving conversation data

[1160] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[1161] Input: Received message

[1162] Output: Parsed conversation data is saved in a database

[1163] Specific operation: The server analyzes the contents of the message, adds sender information and a timestamp, and stores the message in a database.

[1164] Step 5:

[1165] Analysis of conversation data

[1166] Server: The server periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1167] Input: Conversation data retrieved from the database

[1168] Output: Analysis results

[1169] How it works: The server periodically retrieves conversation data stored in the database and analyzes it using machine learning algorithms. The analysis results include the user's unique speaking style and expressions.

[1170] Step 6:

[1171] Training a generative AI model

[1172] Server: The server trains a user-specific generative AI model based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[1173] Input: Analysis results

[1174] Output: A trained generative AI model

[1175] Specific operation: The server uses the acquired analysis results to train the generative AI model. It uses OpenAI's Fine tuning API to optimize the model for the user.

[1176] Step 7:

[1177] Saving the generated AI model

[1178] Server: Stores the trained generative AI model in a database.

[1179] Input: A trained generative AI model

[1180] Output: Generative AI model stored in a database

[1181] What happens: The trained model is saved in a database and ready to be used for future dialogue generation.

[1182] Step 8:

[1183] Interacting with Clonebots

[1184] User: Users can invite clonebots to group chats to participate in conversations with other users.

[1185] Input: Message from other users

[1186] Output: A reply message is sent by the clonebot

[1187] What it does: A friend asks, "What did you do over the weekend?" The device sends this message to a server via an API. The server analyzes the message and uses a trained generative AI model to generate a natural-sounding response, such as "I went camping over the weekend." The server then sends the generated reply message to the user's device, where it appears in the group chat.

[1188] (Application example 1)

[1189] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1190] Current food delivery services lack personalized recommendations and interactions based on users' individual preferences, which makes the ordering process cumbersome, as users are presented with the same operations and options every time. Furthermore, suggestions do not take into account users' food and beverage preferences or eating history, which makes it time-consuming to find more suitable options.

[1191] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1192] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a means for learning the user's preferences using the above means and recommending meals and drinks, and a means for transmitting the generated recommendations to the user terminal, thereby enabling the user to receive personalized dialogue and recommendations based on their individual preferences.

[1193] "User-registered initial setting means" is a function that allows users to input initial information into the system and customize the system individually.

[1194] "Means for collecting and storing conversation data with users" refers to the function of recording and storing the content of conversations users have with the system.

[1195] "Means of analyzing collected conversation data and learning the user's speaking style and context" refers to a function that understands the user's speaking style, unique expressions, and context based on recorded conversation data.

[1196] "Means to train user-specific models based on learned data" refers to the ability to create individually personalized AI models based on the user's learned speech patterns and context.

[1197] "Means for generating conversation replies using a generated user-specific model" is a function that uses a user-specific AI model created to generate replies tailored to the user's style.

[1198] "Means of learning the user's preferences using the above means and recommending meals and drinks" refers to a function that analyzes the user's favorite dishes and drinks based on the user's conversation data and past ordering history, and makes recommendations accordingly.

[1199] "Means for sending the generated recommendation content to the user's device" refers to a function that notifies the user's device of the recommendation content created by the AI ​​model.

[1200] The present invention relates to a system that provides a food delivery assistant customized for each user. The system collects and analyzes user conversation data, generates a user-specific AI model, and recommends meals and beverages based on the user's preferences.

[1201] System Configuration

[1202] The system mainly consists of the following components:

[1203] 1. User device: The user interacts with the clonebot on a device such as a smartphone or computer.

[1204] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[1205] 3. Database: Stores and manages user conversation data and trained models.

[1206] Hardware and software used

[1207] Hardware: Smartphones, cloud servers

[1208] Software: LINE API, OpenAI Fine-tuning API

[1209] User device behavior

[1210] On the user's device, the user adds the clonebot as a friend through LINE or a dedicated app and starts the initial setup process by scanning the QR code and sending a message saying, "Help me deliver food."

[1211] Server Operation

[1212] The server receives the initial setup request sent from the device, generates a user ID, and saves the profile in a database. It then collects and saves daily conversation data and periodically analyzes it.

[1213] Analysis and Training

[1214] The server analyzes the user's speaking style and preferences based on the collected conversation data, and uses the OpenAI Fine-tuning API to train a user-specific AI model, which is then used to generate natural-sounding dialogue based on the user's preferences and context.

[1215] Dietary and Food Recommendations

[1216] The server learns the user's preferences based on their conversation data and past ordering history, so that when the user orders food and drink, it can make recommendations that are tailored to the user's preferences.

[1217] Interacting with Clonebots

[1218] If a user wants to order a specific meal, they can ask the clonebot, "What would you like to eat today?" and the clonebot can make a suggestion based on the user's preferences, such as, "I feel like eating Italian today. That pasta with meat sauce is especially delicious." Also, if the user wants to order the same dish again, the order details are saved, so they can be reordered with a simple command, such as, "Order the pasta with meat sauce again."

[1219] Specific examples

[1220] For example, if a user asks the clonebot, "What kind of food do you recommend today?", it will suggest, "You seem to be coming down with a cold today. How about some stomach-friendly Japanese food?" If the user asks, "Which restaurant specifically?", it will respond, "What's the nearest Japanese restaurant? It was delicious the last time I ordered."

[1221] Prompt Sentence Examples

[1222] "What dish do you recommend today?"

[1223] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[1224] In this way, the system of the present invention can provide personalized interactions and recommendations based on the user's individual preferences, significantly improving the food delivery ordering process.

[1225] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1226] Step 1:

[1227] Users add the clone bot as a friend via LINE or a dedicated app and send it a message saying, "Help with food delivery."

[1228] Input: User's initial setup request

[1229] Output: Notification of the start of the initialization process

[1230] How it works: The user scans the QR code, becomes friends with the clonebot in the LINE app, sends an initial setup message, and sends a request to the server.

[1231] Step 2:

[1232] The server receives the initial setup request, generates a user ID and saves the profile in a database.

[1233] Input: Initial Setup Request

[1234] Output: User ID and profile creation and saving

[1235] Behavior: The server analyzes the initial configuration request data, generates a new user ID, and stores the generated user ID and profile in the database.

[1236] Step 3:

[1237] Collect and store daily conversation data with users.

[1238] Input: User's daily conversation data

[1239] Output: Saved conversation data

[1240] Operation: The device sends the user's conversation data to the server via the LINE API. The server stores the received data in a database.

[1241] Step 4:

[1242] The server analyzes the collected conversation data and learns the user's speaking style and preferences.

[1243] Input: Saved conversation data

[1244] Output: Analyzed speaking style and preference data

[1245] How it works: The server periodically retrieves conversation data from the database and uses natural language processing techniques to analyze the user's speaking style, context, and preferences.

[1246] Step 5:

[1247] The server trains a user-specific AI model based on the learned data.

[1248] Input: Parsed data

[1249] Output: A trained, user-specific AI model

[1250] How it works: The server uses the analyzed data to train a user-specific CloneBot model using the OpenAI Fine-tuning API.

[1251] Step 6:

[1252] The server generates conversations and recommendation replies using the generated user-specific model.

[1253] Input: Trained AI model, user request

[1254] Output: Natural conversation and recommendation replies

[1255] How it works: The server receives a user request and uses the trained model to generate a natural-sounding response based on the user's speaking style and preferences.

[1256] Step 7:

[1257] The server sends the generated reply to the user terminal.

[1258] Input: Generated reply

[1259] Output: Reply displayed on the user's terminal

[1260] Behavior: The server sends the generated reply to the user's device via the LINE API, and the reply is displayed on the user's device.

[1261] Specific examples

[1262] Example prompt sentence:

[1263] "What dish do you recommend today?"

[1264] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[1265] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1266] The present invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining it with an emotion engine.

[1267] System Configuration

[1268] This system consists of the following main components:

[1269] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[1270] 2. Server: Hosts the CloneBot AI model and emotion engine, and performs data processing, analysis, and training.

[1271] 3. Database: Stores and manages user conversation data and trained models.

[1272] 4. Emotion Engine: An engine that analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[1273] Initial setting method

[1274] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[1275] Server: Receives the initial setup request and generates a new user identification (User ID), which stores the new user's profile in the database.

[1276] Conversation data collection and storage method

[1277] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1278] Device: Sends a message to the server via the LINE API.

[1279] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.). Additionally, it uses an emotion engine to analyze the emotional state of the message.

[1280] Server: Stores the extracted conversation data and emotion data in a database.

[1281] Analysis and learning tools

[1282] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[1283] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[1284] Server: After training is complete, save the updated model to the database.

[1285] Interacting with Clonebots

[1286] User: Invite your clonebot to join a group chat and have it interact with other users.

[1287] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1288] Device: Send this message to the server via the LINE API.

[1289] Server: Analyzes received messages and generates appropriate replies. Creates natural responses based on the user's past conversation data, training results, and emotional state. Using an emotion engine, it recognizes emotions such as "fun" or "difficult" and generates appropriate replies.

[1290] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[1291] Specific examples

[1292] For example, if Tanaka sends a message to CloneBot saying, "The movie yesterday was great," CloneBot will recognize the emotion as "enjoyment" and reply, "Yeah, it was really great. What shall we watch next?" If Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can generate a response that includes an emotional expression, such as, "I went camping over the weekend. It was so refreshing!"

[1293] In this way, the present invention allows users to enjoy a natural and interactive dialogue experience that reflects their own emotions and speaking style. Furthermore, by combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[1294] The processing flow will be explained below.

[1295] Step 1:

[1296] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[1297] Step 2:

[1298] Device: Sends the user's message to the server via the LINE API.

[1299] Step 3:

[1300] Server: Receives the initialization message and generates a new user identification (user ID).

[1301] Step 4:

[1302] Server: Save the new user's profile (user ID, initial settings information) in the database.

[1303] Step 5:

[1304] Server: Generates the initial setup completion message and sends it to the user.

[1305] Step 6:

[1306] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[1307] Step 7:

[1308] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1309] Step 8:

[1310] Device: Sends messages from users to the server via the LINE API.

[1311] Step 9:

[1312] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[1313] Step 10:

[1314] Server: Analyzes the user's emotional state from the received message using the emotion engine. For example, recognizes the emotion "stress" from the keyword "busy."

[1315] Step 11:

[1316] Server: Stores the extracted conversation data and emotion data in a database.

[1317] Step 12:

[1318] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[1319] Step 13:

[1320] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[1321] Step 14:

[1322] Server: After training is complete, the updated AI model is saved in the database.

[1323] Step 15:

[1324] User: Invite your clonebot to join a group chat and have it interact with other users.

[1325] Step 16:

[1326] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1327] Step 17:

[1328] Device: Send this message to the server via the LINE API.

[1329] Step 18:

[1330] Server: Analyzes the received message and generates an appropriate reply. It creates natural responses based on the user's past conversation data, training results, and emotional state. For example, it uses the emotion engine to recognize the emotion "fun" and generates a reply such as, "I went camping this weekend. It was so refreshing!"

[1331] Step 19:

[1332] Server: The generated reply message is sent to the user's device via the LINE API.

[1333] Step 20:

[1334] Terminal: Clonebot's reply message appears in group chat.

[1335] As a concrete example, if Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the emotion engine will recognize the emotion of "enjoyment" from the keyword "interesting." The server will generate a reply that reflects this emotion: "Yes, it was really interesting. What shall we watch next?" and send it to Tanaka's device via the LINE API. In this way, natural dialogue that reflects the user's emotions and speaking style is realized.

[1336] Example 2

[1337] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1338] Conventional dialogue systems have struggled to achieve natural dialogue that reflects the user's emotional state. Furthermore, there are limited means to individually learn a user's speaking style and context and generate a model dedicated to that user. Furthermore, mechanisms for effectively analyzing and storing collected conversation data and regularly updating models are inadequate.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1340] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific artificial intelligence model based on the learned data, a means for generating conversation replies using the generated user-specific artificial intelligence model, a means for sending the generated replies to the user terminal, and a means for analyzing the emotional state of the message using an emotion engine. This makes it possible to realize natural, interactive dialogue that reflects the user's emotions and speaking style, and to continuously generate and update a user-specific model.

[1341] "User" refers to an individual or end user of an interactive system.

[1342] The "initial setting means" includes a setting procedure that a user must perform before using the dialogue system.

[1343] "Conversation Data" refers to information, including text messages, that a user sends and receives through an interactive system.

[1344] "Means for collection and storage" refers to the function by which the server receives conversation data, analyzes it, and stores it in a database.

[1345] "Means for analyzing" refers to the methods and technologies that use collected conversational data to analyze a user's speaking style, context, and emotional state.

[1346] "Learning means" refers to the process of accumulating information about users based on the analysis results and using it to improve the quality of future interactions.

[1347] "Artificial intelligence model" refers to a computer program that uses machine learning and natural language processing techniques to mimic a user's speech patterns and context.

[1348] "Training" refers to the process of using collected and learned data to improve and optimize an artificial intelligence model.

[1349] "Means for generating replies" refers to the ability to automatically generate replies to users' messages using a trained artificial intelligence model.

[1350] An "emotion engine" is a software tool that analyzes emotions from users' messages and uses that information to reflect them in the content of the conversation.

[1351] "Communications application" refers to application software that allows a user to send and receive messages.

[1352] This invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining an emotion engine. The main components of this system include a user terminal, a server, a database, and an emotion engine.

[1353] System Configuration

[1354] 1. User terminal: The user terminal of this system is a device such as a smartphone or PC, which the user uses to interact with the clonebot through the LINE app.

[1355] Example: A user sends a message to a clone bot on the LINE app saying, "The movie yesterday was great."

[1356] 2. Server: The server hosts the CloneBot's artificial intelligence model and emotion engine, and performs data processing, analysis, and training.

[1357] Software used: Node.js, Python, NLTK, TensorFlow, PyTorch, OpenAI API

[1358] Example: A server parses messages received from users and analyzes their emotional state using Python's NLTK.

[1359] 3. Database: The database stores and manages user conversation data and trained models.

[1360] Software used: MySQL, SQLAlchemy

[1361] 4. Emotion Engine: The emotion engine analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[1362] Initial setting method

[1363] Users add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, users add the clone bot by scanning the QR code and send a message saying, "Make my clone bot."

[1364] The server receives the initial setup request and generates a new user identity (user ID), which saves the new user's profile in the database. It uses the UUID library to generate a unique ID and saves it in the MySQL database.

[1365] Conversation data collection and storage method

[1366] Users exchange messages with the clonebot on a daily basis, for example, sending a message saying, "I was busy this morning."

[1367] The device forwards the sent message to the server via the LINE API.

[1368] The server parses the received messages and extracts metadata such as content, sender information, and timestamps. It also analyzes the sentiment of the messages using Python's NLTK. The analyzed data is stored in a database using SQLAlchemy.

[1369] Analysis and learning tools

[1370] The server periodically retrieves conversation data from the database. This is automated using a scheduler. The retrieved data is analyzed using TensorFlow or PyTorch to generate and update an AI model specific to the user. OpenAI's Fine Tuning API is used for training. The generated and updated model is stored in the database.

[1371] Interacting with Clonebots

[1372] Users can invite clonebots to join group chats in communication applications to interact with other users.

[1373] A friend sends a question in a group chat: "What did you do over the weekend?"

[1374] The device forwards this message to the server via the LINE API.

[1375] The server uses OpenAI's GPT-3 API to generate appropriate replies, taking into account past conversation data and emotional states. The replies are then sent to the user's device via the LINE API and displayed in the group chat.

[1376] Examples of specific examples and prompts

[1377] For example, if a user sends a message to a clonebot on the LINE app saying, "Last night's movie was great," the server will recognize the emotion as "enjoyment" and generate a reply saying, "Yeah, it was really great. What should we watch next?"

[1378] Example prompt sentence:

[1379] "Make a clone bot of me."

[1380] "The movie yesterday was interesting."

[1381] "What did you do over the weekend?"

[1382] In this way, the present invention can realize natural and interactive dialogue that reflects the user's emotions and speaking style. By combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[1383] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1384] Step 1:

[1385] The user adds the clone bot to their friends on the LINE app and sets up the initial settings. At this stage, the user scans the QR code and sends a message saying, "Make my clone bot."

[1386] Input: QR code scan and initial message

[1387] Output: Initialization request

[1388] Step 2:

[1389] The server receives an initial setup request through the LINE API. It uses Node.js to parse this request and generate a new user ID. It uses the UUID library to generate a unique ID.

[1390] Input: Initial Setup Request

[1391] Output: User ID

[1392] Step 3:

[1393] The server stores the generated user ID in a MySQL database and creates a profile for the user, using SQLAlchemy for database operations.

[1394] Input: User ID

[1395] Output: User profile stored in the database

[1396] Step 4:

[1397] Users can exchange messages with the clone bot on a daily basis using the LINE app, for example, sending a message like "I was busy this morning."

[1398] Input: Message from user

[1399] Output: None

[1400] Step 5:

[1401] The device forwards the message to the server via the LINE API, and Node.js is used to receive and analyze the message.

[1402] Input: User's message

[1403] Output: Message received by the server

[1404] Step 6:

[1405] The server parses the received messages, extracting message content and metadata (sender information, timestamp, etc.), and analyzes the emotional state using Python's NLTK library.

[1406] Input: The message received by the server

[1407] Output: Parsed message content and sentiment data

[1408] Step 7:

[1409] The server extracts conversation data and emotion data and stores it in a database. Data is added to the database using SQLAlchemy.

[1410] Input: Parsed message content and sentiment data

[1411] Output: Conversation data and emotion data stored in a database

[1412] Step 8:

[1413] The server periodically retrieves conversation data from the database. This process is automated by a scheduler, and the data is analyzed using TensorFlow and PyTorch.

[1414] Input: Conversation data retrieved from the database

[1415] Output: Analysis results

[1416] Step 9:

[1417] The server uses OpenAI's Fine Tuning API to train a user-specific AI model, then optimizes the model based on the analysis results.

[1418] Input: Analysis results

[1419] Output: A trained artificial intelligence model

[1420] Step 10:

[1421] After the server completes the training, it stores the updated artificial intelligence model in a database.

[1422] Input: A trained artificial intelligence model

[1423] Output: Artificial intelligence model stored in a database

[1424] Step 11:

[1425] A user invites a clonebot to join a group chat in a communication application and allows it to interact with other users.

[1426] Input: None

[1427] Output: None

[1428] Step 12:

[1429] A friend sends a question in a group chat: "What did you do over the weekend?"

[1430] Input: Message from a friend

[1431] Output: None

[1432] Step 13:

[1433] The device forwards this message to the server via the LINE API, and receives the message using Node.js.

[1434] Input: Message from a friend

[1435] Output: Message received by the server

[1436] Step 14:

[1437] The server takes into account past conversation data and emotional state and generates appropriate replies using OpenAI's GPT-3 API.

[1438] Input: Received messages, past conversation data, emotion data

[1439] Output: The generated reply

[1440] Step 15:

[1441] The server generates a reply and sends it to the user's device via the LINE API, where it is displayed in the group chat.

[1442] Input: Generated reply

[1443] Output: Reply sent to the user's device

[1444] This step allows users to enjoy a natural, interactive dialogue experience that reflects their own emotions and speaking style.

[1445] (Application example 2)

[1446] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1447] In modern autonomous vehicles, users want a more comfortable and stress-free driving experience. However, conventional autonomous driving systems lack the ability to engage in natural dialogue with users and provide emotionally appropriate responses. This can lead to distrust and frustration among users. Furthermore, the lack of a dialogue model tailored to individual users results in repetitive, uniform responses, leading to lower user satisfaction.

[1448] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1449] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a sentiment analysis means for analyzing the user's sentiment, a means for adjusting the reply content based on the sentiment analysis, and a means for sending the generated replies to the user terminal. This enables recognition of the user's sentiment and a natural dialogue appropriate to that sentiment. Furthermore, training a user-specific dialogue model enables dialogue optimized for each individual user, improving the user experience.

[1450] "Initial setting means" refers to the means by which a user accesses the system and makes the necessary settings.

[1451] "Conversation data" refers to information such as messages and voices exchanged between a user and a system.

[1452] "Collection and storage means" refers to the means for accumulating and managing conversation data obtained from users.

[1453] "Analysis means" refers to a means for analyzing collected conversation data and learning the user's speaking style and context.

[1454] A "training means" is a means for generating a user-specific dialogue model based on the analyzed data.

[1455] The "generation means" is a means for generating conversation replies using the generated user-specific model.

[1456] "Emotion analysis means" is a means for estimating and analyzing emotions from user messages.

[1457] The "reply adjustment means" is a means for appropriately adjusting the reply content generated based on the result of emotion analysis.

[1458] The "transmitting means" is a means for transmitting the generated reply to the user terminal.

[1459] "Virtual driver" refers to a virtual driver bot that interacts with the user inside an autonomous vehicle and provides information about the driving situation and destination.

[1460] This invention relates to the construction of a system for use in autonomous vehicles, specifically for analyzing user emotions and providing natural dialogue accordingly. The system consists of the following main components:

[1461] User Device

[1462] The user terminal is a smartphone or an in-vehicle infotainment system, and is the device through which the user accesses the system and performs initial setup and daily interaction. Initial setup includes installing applications and setting up a user-specific profile. The "initial setup means" allows the user to register the necessary information for the system.

[1463] server

[1464] The server is the main device that stores and manages the collected conversation data, and is responsible for analysis and training. Specifically, it performs the following processes:

[1465] Conversation data collection and storage: Conversation data with the user is acquired and stored in a database. For example, a message such as "The roads are crowded" is stored.

[1466] Analysis method: Analyzes collected conversation data to learn about the user's speaking style and context. This analysis is done using machine learning algorithms and natural language processing techniques.

[1467] Training method: A dialogue model specific to the user is generated based on the analyzed data. This training is performed using OpenAI's Fine tuning API, among other tools.

[1468] Sentiment analysis: The Sentimental library is used to estimate and analyze the sentiment of users' messages. For example, if a user says "I'm a little tired," the sentiment is analyzed as negative.

[1469] Reply adjustment: The generated reply content is adjusted based on the results of sentiment analysis, allowing for the generation of appropriate reply content according to the user's emotions.

[1470] Transmission method: The generated reply is sent to the user's device. For example, a message such as "Are you tired? Shall we take a break at the next service area?" is sent to the user's device.

[1471] Specific examples

[1472] For example, if a user in an autonomous vehicle says, "Could you please speed up a bit?", the system will analyze the message using its "emotion analysis means" and generate an appropriate reply if necessary. The reply generated will be, "We can't speed up, but we are ensuring a safe route," and sent to the user's device. In this way, users can experience natural dialogue that reflects their emotions.

[1473] Examples of prompts include:

[1474] "If a user says they're tired, suggest an appropriate place to rest."

[1475] "If the user's sentiment is positive, recommend them the next tourist attraction they should visit."

[1476] This invention allows users to travel more comfortably and stress-free in self-driving vehicles, and is expected to improve user satisfaction by providing emotion-based dialogue.

[1477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1478] Step 1:

[1479] Initial Setup

[1480] The user installs the application on their own device and performs initial setup. Specifically, they start the application and enter the necessary information. The entered information is sent to the server via the "initial setup means." The server uses this information to generate a user-specific profile and stores it in a database. Input: User information, Output: User profile.

[1481] Step 2:

[1482] Conversation data collection

[1483] The user starts a conversation on the device. The device sends the user's message to the server via the LINE API or the in-car infotainment system. The server stores the received message in a database using the "conversation data collection and storage means." Input: User's message, Output: Stored conversation data.

[1484] Step 3:

[1485] Analysis of conversation data

[1486] The server periodically retrieves conversation data from the database. The "analysis means" analyzes the collected conversation data and learns the user's speaking style and context. In this process, machine learning algorithms (e.g., natural language processing technology) are used. Input: saved conversation data, Output: analysis results.

[1487] Step 4:

[1488] Training the dialogue model

[1489] A "training method" generates a user-specific dialogue model based on the analyzed data. This training is performed using OpenAI's Fine tuning API, etc. After training, the generated model is saved in a database. Input: Analysis results, Output: User-specific dialogue model.

[1490] Step 5:

[1491] Emotion analysis

[1492] The server analyzes new messages received from users using the "sentiment analysis means." This analysis uses the Sentimental library to determine whether the user's emotion is "positive" or "negative." For example, if the message is "tired," it is determined to be negative. Input: User's message, Output: Emotion determination result.

[1493] Step 6:

[1494] Generate and moderate replies

[1495] The "generation means" generates a conversational reply using a dialogue model dedicated to the user. The "reply adjustment means" then adjusts the reply content based on the results of the emotion analysis. For example, if the user says "I'm a little tired," the reply content is adjusted to "Shall we take a break at the next service area?" Input: Emotion determination result and user-specific dialogue model, Output: Adjusted reply.

[1496] Step 7:

[1497] Sending a reply

[1498] The "transmission means" sends the generated and adjusted reply to the user's terminal. The user can check the reply through the terminal. Input: adjusted reply, Output: message displayed on the user's terminal.

[1499] Step 8:

[1500] User experience updates

[1501] The collected new conversation data and the results of sentiment analysis are periodically learned and updated by the "analysis means" and "training means." This ensures that the user-specific dialogue model always reflects the latest conversation patterns and emotional state. Input: New conversation data and sentiment analysis results, Output: Updated dialogue model.

[1502] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1504] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1505] [Fourth embodiment]

[1506] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1507] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1508] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1509] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1510] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1511] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1512] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1513] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1514] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1515] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1516] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1517] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1518] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1519] This invention relates to a dialogue system using a clonebot personalized for each user. This system learns the user's speaking style and way of thinking, generates an AI model dedicated to the user, and realizes natural dialogue that mimics the user's speaking style.

[1520] System Configuration

[1521] This system consists of the following main components:

[1522] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[1523] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[1524] 3. Database: Stores and manages user conversation data and trained models.

[1525] Initial setting method

[1526] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[1527] Server: Receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[1528] Conversation data collection and storage method

[1529] User: Interacts with the clonebot on a daily basis, for example sending a message saying "I was busy this morning."

[1530] Device: Sends a message to the server via the LINE API.

[1531] Server: Analyzes received messages and stores conversation data (content, sender information, timestamp, etc.) in a database.

[1532] Analysis and learning tools

[1533] Server: Periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1534] Server: Trains a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[1535] Server: After training is complete, save the updated model to the database.

[1536] Interacting with Clonebots

[1537] Users: Invite your clonebot to join a group chat and have it participate in conversations with other users.

[1538] Friends: Asking in a group chat, "What did you do over the weekend?"

[1539] Device: Sends the received message to the server via the LINE API.

[1540] Server: Analyzes the received message and generates a reply using a user-specific AI model. Specifically, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[1541] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[1542] Specific examples

[1543] For example, if Tanaka were to use CloneBot, the following conversations would be possible: If Tanaka sends a message to CloneBot on LINE saying, "Last night's movie was great," CloneBot would reply, "Yeah, I enjoyed it too. What shall we see next?" Also, if Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on Tanaka's behalf, saying, "I went camping over the weekend."

[1544] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[1545] The processing flow will be explained below.

[1546] Step 1:

[1547] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[1548] Step 2:

[1549] Device: Sends the user's message to the server via the LINE API.

[1550] Step 3:

[1551] Server: Receives the initialization message and generates a new user identification (user ID).

[1552] Step 4:

[1553] Server: Save the new user's profile (user ID, initial settings information) in the database.

[1554] Step 5:

[1555] Server: Generates the initial setup completion message and sends it to the user.

[1556] Step 6:

[1557] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[1558] Step 7:

[1559] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1560] Step 8:

[1561] Device: Sends messages from users to the server via the LINE API.

[1562] Step 9:

[1563] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[1564] Step 10:

[1565] Server: Save the extracted conversation data in a database.

[1566] Step 11:

[1567] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style and context.

[1568] Step 12:

[1569] Server: Generates or updates a user-specific AI model based on the analysis results. Training is performed using OpenAI's Fine tuning API.

[1570] Step 13:

[1571] Server: After training is complete, the updated AI model is saved in the database.

[1572] Step 14:

[1573] User: Invite your clonebot to join a group chat and have it interact with other users.

[1574] Step 15:

[1575] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1576] Step 16:

[1577] Device: Send this message to the server via the LINE API.

[1578] Step 17:

[1579] Server: Analyzes the received message and generates an appropriate reply. Creates natural responses based on the user's past conversation data and training results.

[1580] Step 18:

[1581] Server: The generated reply message is sent to the user's device via the LINE API.

[1582] Step 19:

[1583] Terminal: Clonebot's reply message appears in group chat.

[1584] For example, if Mr. Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the clonebot will generate a reply saying, "Yes, it was really interesting. What shall we see next?" In this way, natural dialogue that reflects the user's characteristics is realized.

[1585] Example 1

[1586] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1587] Conventional dialogue systems have struggled to achieve natural dialogue that takes into account the individual speaking style and context of each user. Furthermore, no dialogue system to date has used a generative AI model that can accurately learn and mimic a user's speaking style and way of thinking. This has made it difficult for users to enjoy dialogue that reflects their own individuality, and the system's responses can feel unnatural.

[1588] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1589] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific generative AI model based on the learned data, a means for generating conversation replies using the generated user-specific generative AI model, and a means for transmitting the generated replies to the user terminal, thereby enabling the user to enjoy natural conversations that reflect their own speaking style and way of thinking.

[1590] "Initialization means" refers to the process or function by which a user registers with the system.

[1591] "Conversation data" refers to the messages and dialogue exchanged between the user and the clonebot.

[1592] "Means of collection and storage" refers to the functions and processes for collecting conversation data with users and storing it in a database.

[1593] "Means of analysis" refers to the technologies and algorithms used to analyze collected conversational data and understand the user's speaking style and context.

[1594] "Means of learning" refers to the process of training users' speaking and expressing themselves based on analytical data.

[1595] A "generative AI model" refers to an artificial intelligence model that is trained to mimic a user's speech style and context.

[1596] "Training method" refers to the process by which a generative AI model is trained based on user conversation data.

[1597] "Means for generating replies" refers to the ability to use a generative AI model to automatically create replies that mimic the way a user would speak.

[1598] "User Device" refers to the device a user uses to interact with a clonebot, including a smartphone or computer.

[1599] "Means for sending" refers to the function or process for sending the generated reply message to the user terminal.

[1600] "Means of updating" refers to the process of periodically retraining the generative AI model based on collected conversational data and updating the model based on the latest information.

[1601] A "clonebot" is an AI assistant that is trained to mimic the way a user speaks and thinks.

[1602] "Group chat" refers to a chat function that allows multiple users to interact at the same time.

[1603] "Other users' messages" refers to messages sent by users other than the CloneBot user within a group chat.

[1604] This invention relates to a dialogue system using a user-specific clonebot, which learns the user's speaking style and way of thinking, generates a user-specific generative AI model, and realizes natural dialogue that mimics the user's speaking style.

[1605] System Configuration

[1606] This system consists of the following main components:

[1607] 1. User device: A device such as a smartphone or PC on which the user interacts with the clonebot using a communication app.

[1608] 2. Server: Hosts the clonebot generative AI model and performs data processing, analysis, and training.

[1609] 3. Database: Stores and manages user conversation data and trained models.

[1610] Initial setting method

[1611] User: The user adds the clone bot as a friend in the communication app and performs the initial setup. Specifically, the user adds the clone bot by scanning the QR code and sends a message saying, "Make my clone bot."

[1612] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to the database.

[1613] Conversation data collection and storage method

[1614] User: The user interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1615] Device: The user device sends this message to the server through the API.

[1616] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[1617] Analysis and learning tools

[1618] Server: The server periodically retrieves conversation data from the database and analyzes it. Specifically, it uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1619] Server: The server trains a generative AI model specific to the user based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[1620] Server: After training is complete, save the updated model to the database.

[1621] Interacting with Clonebots

[1622] User: Users can invite clonebots to group chats and have them participate in conversations with other users.

[1623] Friends: Friends ask in group chats, "What did you do over the weekend?"

[1624] Device: The user device sends the received message to the server via the API.

[1625] Server: The server analyzes the received message and generates a reply using a generative AI model specific to the user. For example, it creates a natural response such as "I went camping over the weekend. It was fun!" based on past conversation data and the user's speaking style learned through training.

[1626] Server: The generated reply message is sent to the user's device via the API and displayed in the group chat.

[1627] Specific examples

[1628] For example, when a user uses this system, the following dialogue is possible:

[1629] If a user sends a message to CloneBot in a communication app saying, "The movie last night was great," CloneBot will reply, "I enjoyed it too. What shall we watch next?" If a user's friend asks in a group chat, "What did you do over the weekend?" CloneBot can answer on the user's behalf, saying, "I went camping over the weekend."

[1630] In this way, the present invention allows users to enjoy natural and interactive conversations through a clone bot with a rich personality. Furthermore, by using the collection, analysis, training, and generation methods, responses that precisely mimic the user's speaking style can be realized.

[1631] Examples of prompt statements

[1632] For example, "Generate a scenario in which you talk to a clonebot about a movie you saw yesterday."

[1633] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1634] Step 1:

[1635] Initial Setup

[1636] User: The user adds the clone bot as a friend in a communication app and sends a message saying, "Make a clone bot of me."

[1637] Input: Default message from the user

[1638] Output: Initialization request sent to server

[1639] What it does: The user scans the QR code in the app, adds the clonebot as a friend, and then types and sends a message saying, "Make a clonebot of me."

[1640] Step 2:

[1641] User Registration

[1642] Server: The server receives the initial setup request and registers the new user, generating a user ID and saving the profile to a database.

[1643] Input: Initial Setup Request

[1644] Output: User ID and profile are saved in the database.

[1645] What happens: The server analyzes the received message and stores the new user's information in the database. It generates a user ID using an ID generation algorithm and stores the profile information.

[1646] Step 3:

[1647] Conversation data collection

[1648] User: The user interacts with the clonebot on a daily basis, sending messages such as "I was busy this morning."

[1649] Input: Conversational message from user

[1650] Output: The message is sent to the server

[1651] Specific operation: The user interacts with the clonebot and sends messages, which the device forwards to the server via API.

[1652] Step 4:

[1653] Saving conversation data

[1654] Server: The server analyzes the received messages and stores the conversation data (content, sender information, timestamp, etc.) in a database.

[1655] Input: Received message

[1656] Output: Parsed conversation data is saved in a database

[1657] Specific operation: The server analyzes the contents of the message, adds sender information and a timestamp, and stores the message in a database.

[1658] Step 5:

[1659] Analysis of conversation data

[1660] Server: The server periodically retrieves conversation data from the database and uses machine learning algorithms to analyze the user's speaking style, grammar, and idiosyncratic expressions.

[1661] Input: Conversation data retrieved from the database

[1662] Output: Analysis results

[1663] How it works: The server periodically retrieves conversation data stored in the database and analyzes it using machine learning algorithms. The analysis results include the user's unique speaking style and expressions.

[1664] Step 6:

[1665] Training a generative AI model

[1666] Server: The server trains a user-specific generative AI model based on the analysis results. For training, it uses OpenAI's Fine tuning API.

[1667] Input: Analysis results

[1668] Output: A trained generative AI model

[1669] Specific operation: The server uses the acquired analysis results to train the generative AI model. It uses OpenAI's Fine tuning API to optimize the model for the user.

[1670] Step 7:

[1671] Saving the generated AI model

[1672] Server: Stores the trained generative AI model in a database.

[1673] Input: A trained generative AI model

[1674] Output: Generative AI model stored in a database

[1675] What happens: The trained model is saved in a database and ready to be used for future dialogue generation.

[1676] Step 8:

[1677] Interacting with Clonebots

[1678] User: Users can invite clonebots to group chats to participate in conversations with other users.

[1679] Input: Message from other users

[1680] Output: A reply message is sent by the clonebot

[1681] What it does: A friend asks, "What did you do over the weekend?" The device sends this message to a server via an API. The server analyzes the message and uses a trained generative AI model to generate a natural-sounding response, such as "I went camping over the weekend." The server then sends the generated reply message to the user's device, where it appears in the group chat.

[1682] (Application example 1)

[1683] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] Current food delivery services lack personalized recommendations and interactions based on users' individual preferences, which makes the ordering process cumbersome, as users are presented with the same operations and options every time. Furthermore, suggestions do not take into account users' food and beverage preferences or eating history, which makes it time-consuming to find more suitable options.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1686] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a means for learning the user's preferences using the above means and recommending meals and drinks, and a means for transmitting the generated recommendations to the user terminal, thereby enabling the user to receive personalized dialogue and recommendations based on their individual preferences.

[1687] "User-registered initial setting means" is a function that allows users to input initial information into the system and customize the system individually.

[1688] "Means for collecting and storing conversation data with users" refers to the function of recording and storing the content of conversations users have with the system.

[1689] "Means of analyzing collected conversation data and learning the user's speaking style and context" refers to a function that understands the user's speaking style, unique expressions, and context based on recorded conversation data.

[1690] "Means to train user-specific models based on learned data" refers to the ability to create individually personalized AI models based on the user's learned speech patterns and context.

[1691] "Means for generating conversation replies using a generated user-specific model" is a function that uses a user-specific AI model created to generate replies tailored to the user's style.

[1692] "Means of learning the user's preferences using the above means and recommending meals and drinks" refers to a function that analyzes the user's favorite dishes and drinks based on the user's conversation data and past ordering history, and makes recommendations accordingly.

[1693] "Means for sending the generated recommendation content to the user's device" refers to a function that notifies the user's device of the recommendation content created by the AI ​​model.

[1694] The present invention relates to a system that provides a food delivery assistant customized for each user. The system collects and analyzes user conversation data, generates a user-specific AI model, and recommends meals and beverages based on the user's preferences.

[1695] System Configuration

[1696] The system mainly consists of the following components:

[1697] 1. User device: The user interacts with the clonebot on a device such as a smartphone or computer.

[1698] 2. Server: Hosts the CloneBot AI model and performs data processing, analysis, and training.

[1699] 3. Database: Stores and manages user conversation data and trained models.

[1700] Hardware and software used

[1701] Hardware: Smartphones, cloud servers

[1702] Software: LINE API, OpenAI Fine-tuning API

[1703] User device behavior

[1704] On the user's device, the user adds the clonebot as a friend through LINE or a dedicated app and starts the initial setup process by scanning the QR code and sending a message saying, "Help me deliver food."

[1705] Server Operation

[1706] The server receives the initial setup request sent from the device, generates a user ID, and saves the profile in a database. It then collects and saves daily conversation data and periodically analyzes it.

[1707] Analysis and Training

[1708] The server analyzes the user's speaking style and preferences based on the collected conversation data, and uses the OpenAI Fine-tuning API to train a user-specific AI model, which is then used to generate natural-sounding dialogue based on the user's preferences and context.

[1709] Dietary and Food Recommendations

[1710] The server learns the user's preferences based on their conversation data and past ordering history, so that when the user orders food and drink, it can make recommendations that are tailored to the user's preferences.

[1711] Interacting with Clonebots

[1712] If a user wants to order a specific meal, they can ask the clonebot, "What would you like to eat today?" and the clonebot can make a suggestion based on the user's preferences, such as, "I feel like eating Italian today. That pasta with meat sauce is especially delicious." Also, if the user wants to order the same dish again, the order details are saved, so they can be reordered with a simple command, such as, "Order the pasta with meat sauce again."

[1713] Specific examples

[1714] For example, if a user asks the clonebot, "What kind of food do you recommend today?", it will suggest, "You seem to be coming down with a cold today. How about some stomach-friendly Japanese food?" If the user asks, "Which restaurant specifically?", it will respond, "What's the nearest Japanese restaurant? It was delicious the last time I ordered."

[1715] Prompt Sentence Examples

[1716] "What dish do you recommend today?"

[1717] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[1718] In this way, the system of the present invention can provide personalized interactions and recommendations based on the user's individual preferences, significantly improving the food delivery ordering process.

[1719] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1720] Step 1:

[1721] Users add the clone bot as a friend via LINE or a dedicated app and send it a message saying, "Help with food delivery."

[1722] Input: User's initial setup request

[1723] Output: Notification of the start of the initialization process

[1724] How it works: The user scans the QR code, becomes friends with the clonebot in the LINE app, sends an initial setup message, and sends a request to the server.

[1725] Step 2:

[1726] The server receives the initial setup request, generates a user ID and saves the profile in a database.

[1727] Input: Initial Setup Request

[1728] Output: User ID and profile creation and saving

[1729] Behavior: The server analyzes the initial configuration request data, generates a new user ID, and stores the generated user ID and profile in the database.

[1730] Step 3:

[1731] Collect and store daily conversation data with users.

[1732] Input: User's daily conversation data

[1733] Output: Saved conversation data

[1734] Operation: The device sends the user's conversation data to the server via the LINE API. The server stores the received data in a database.

[1735] Step 4:

[1736] The server analyzes the collected conversation data and learns the user's speaking style and preferences.

[1737] Input: Saved conversation data

[1738] Output: Analyzed speaking style and preference data

[1739] How it works: The server periodically retrieves conversation data from the database and uses natural language processing techniques to analyze the user's speaking style, context, and preferences.

[1740] Step 5:

[1741] The server trains a user-specific AI model based on the learned data.

[1742] Input: Parsed data

[1743] Output: A trained, user-specific AI model

[1744] How it works: The server uses the analyzed data to train a user-specific CloneBot model using the OpenAI Fine-tuning API.

[1745] Step 6:

[1746] The server generates conversations and recommendation replies using the generated user-specific model.

[1747] Input: Trained AI model, user request

[1748] Output: Natural conversation and recommendation replies

[1749] How it works: The server receives a user request and uses the trained model to generate a natural-sounding response based on the user's speaking style and preferences.

[1750] Step 7:

[1751] The server sends the generated reply to the user terminal.

[1752] Input: Generated reply

[1753] Output: Reply displayed on the user's terminal

[1754] Behavior: The server sends the generated reply to the user's device via the LINE API, and the reply is displayed on the user's device.

[1755] Specific examples

[1756] Example prompt sentence:

[1757] "What dish do you recommend today?"

[1758] "I feel like eating Italian food today. The pasta with meat sauce there is especially delicious."

[1759] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1760] The present invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining it with an emotion engine.

[1761] System Configuration

[1762] This system consists of the following main components:

[1763] 1. User device: A device such as a smartphone or PC on which the user interacts with the clone bot using the LINE app.

[1764] 2. Server: Hosts the CloneBot AI model and emotion engine, and performs data processing, analysis, and training.

[1765] 3. Database: Stores and manages user conversation data and trained models.

[1766] 4. Emotion Engine: An engine that analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[1767] Initial setting method

[1768] User: Add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, the user scans the QR code to add the clone bot and sends a message saying, "Make my clone bot." This starts the initial setup.

[1769] Server: Receives the initial setup request and generates a new user identification (User ID), which stores the new user's profile in the database.

[1770] Conversation data collection and storage method

[1771] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1772] Device: Sends a message to the server via the LINE API.

[1773] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.). Additionally, it uses an emotion engine to analyze the emotional state of the message.

[1774] Server: Stores the extracted conversation data and emotion data in a database.

[1775] Analysis and learning tools

[1776] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[1777] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[1778] Server: After training is complete, save the updated model to the database.

[1779] Interacting with Clonebots

[1780] User: Invite your clonebot to join a group chat and have it interact with other users.

[1781] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1782] Device: Send this message to the server via the LINE API.

[1783] Server: Analyzes received messages and generates appropriate replies. Creates natural responses based on the user's past conversation data, training results, and emotional state. Using an emotion engine, it recognizes emotions such as "fun" or "difficult" and generates appropriate replies.

[1784] Server: The generated reply message is sent to the user's device via the LINE API and displayed in the group chat.

[1785] Specific examples

[1786] For example, if Tanaka sends a message to CloneBot saying, "The movie yesterday was great," CloneBot will recognize the emotion as "enjoyment" and reply, "Yeah, it was really great. What shall we watch next?" If Tanaka's friend asks in a group chat, "What did you do over the weekend?" CloneBot can generate a response that includes an emotional expression, such as, "I went camping over the weekend. It was so refreshing!"

[1787] In this way, the present invention allows users to enjoy a natural and interactive dialogue experience that reflects their own emotions and speaking style. Furthermore, by combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[1788] The processing flow will be explained below.

[1789] Step 1:

[1790] User: Adds a clone bot in the LINE app and sends the initial message: "Make a clone bot of me."

[1791] Step 2:

[1792] Device: Sends the user's message to the server via the LINE API.

[1793] Step 3:

[1794] Server: Receives the initialization message and generates a new user identification (user ID).

[1795] Step 4:

[1796] Server: Save the new user's profile (user ID, initial settings information) in the database.

[1797] Step 5:

[1798] Server: Generates the initial setup completion message and sends it to the user.

[1799] Step 6:

[1800] Device: A message that initial setup is complete will be displayed in the LINE app on the user's device.

[1801] Step 7:

[1802] User: Interacts with the clonebot on a daily basis, for example by sending a message saying, "I was busy this morning."

[1803] Step 8:

[1804] Device: Sends messages from users to the server via the LINE API.

[1805] Step 9:

[1806] Server: Analyzes received messages and extracts conversation data (content, sender information, timestamp, etc.).

[1807] Step 10:

[1808] Server: Analyzes the user's emotional state from the received message using the emotion engine. For example, recognizes the emotion "stress" from the keyword "busy."

[1809] Step 11:

[1810] Server: Stores the extracted conversation data and emotion data in a database.

[1811] Step 12:

[1812] Server: Periodically retrieves conversation data from the database and uses machine learning algorithms and an emotion engine to analyze the user's speaking style, context, and emotional state.

[1813] Step 13:

[1814] Server: Generates or updates a user-specific AI model based on the analysis results. For training, OpenAI's Fine tuning API is used.

[1815] Step 14:

[1816] Server: After training is complete, the updated AI model is saved in the database.

[1817] Step 15:

[1818] User: Invite your clonebot to join a group chat and have it interact with other users.

[1819] Step 16:

[1820] Friends: Send a question in a group chat: "What did you do over the weekend?"

[1821] Step 17:

[1822] Device: Send this message to the server via the LINE API.

[1823] Step 18:

[1824] Server: Analyzes the received message and generates an appropriate reply. It creates natural responses based on the user's past conversation data, training results, and emotional state. For example, it uses the emotion engine to recognize the emotion "fun" and generates a reply such as, "I went camping this weekend. It was so refreshing!"

[1825] Step 19:

[1826] Server: The generated reply message is sent to the user's device via the LINE API.

[1827] Step 20:

[1828] Terminal: Clonebot's reply message appears in group chat.

[1829] As a concrete example, if Tanaka sends a message to the clonebot saying, "Yesterday's movie was interesting," the emotion engine will recognize the emotion of "enjoyment" from the keyword "interesting." The server will generate a reply that reflects this emotion: "Yes, it was really interesting. What shall we watch next?" and send it to Tanaka's device via the LINE API. In this way, natural dialogue that reflects the user's emotions and speaking style is realized.

[1830] Example 2

[1831] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1832] Conventional dialogue systems have struggled to achieve natural dialogue that reflects the user's emotional state. Furthermore, there are limited means to individually learn a user's speaking style and context and generate a model dedicated to that user. Furthermore, mechanisms for effectively analyzing and storing collected conversation data and regularly updating models are inadequate.

[1833] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1834] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific artificial intelligence model based on the learned data, a means for generating conversation replies using the generated user-specific artificial intelligence model, a means for sending the generated replies to the user terminal, and a means for analyzing the emotional state of the message using an emotion engine. This makes it possible to realize natural, interactive dialogue that reflects the user's emotions and speaking style, and to continuously generate and update a user-specific model.

[1835] "User" refers to an individual or end user of an interactive system.

[1836] The "initial setting means" includes a setting procedure that a user must perform before using the dialogue system.

[1837] "Conversation Data" refers to information, including text messages, that a user sends and receives through an interactive system.

[1838] "Means for collection and storage" refers to the function by which the server receives conversation data, analyzes it, and stores it in a database.

[1839] "Means for analyzing" refers to the methods and technologies that use collected conversational data to analyze a user's speaking style, context, and emotional state.

[1840] "Learning means" refers to the process of accumulating information about users based on the analysis results and using it to improve the quality of future interactions.

[1841] "Artificial intelligence model" refers to a computer program that uses machine learning and natural language processing techniques to mimic a user's speech patterns and context.

[1842] "Training" refers to the process of using collected and learned data to improve and optimize an artificial intelligence model.

[1843] "Means for generating replies" refers to the ability to automatically generate replies to users' messages using a trained artificial intelligence model.

[1844] An "emotion engine" is a software tool that analyzes emotions from users' messages and uses that information to reflect them in the content of the conversation.

[1845] "Communications application" refers to application software that allows a user to send and receive messages.

[1846] This invention relates to a dialogue system using clone bots individualized to each user, and in particular provides a system that realizes natural dialogue that reflects the user's emotional state by combining an emotion engine. The main components of this system include a user terminal, a server, a database, and an emotion engine.

[1847] System Configuration

[1848] 1. User terminal: The user terminal of this system is a device such as a smartphone or PC, which the user uses to interact with the clonebot through the LINE app.

[1849] Example: A user sends a message to a clone bot on the LINE app saying, "The movie yesterday was great."

[1850] 2. Server: The server hosts the CloneBot's artificial intelligence model and emotion engine, and performs data processing, analysis, and training.

[1851] Software used: Node.js, Python, NLTK, TensorFlow, PyTorch, OpenAI API

[1852] Example: A server parses messages received from users and analyzes their emotional state using Python's NLTK.

[1853] 3. Database: The database stores and manages user conversation data and trained models.

[1854] Software used: MySQL, SQLAlchemy

[1855] 4. Emotion Engine: The emotion engine analyzes emotions from user messages and is used to analyze conversation data and generate replies.

[1856] Initial setting method

[1857] Users add the clone bot as a friend within the LINE app and perform the initial setup. Specifically, users add the clone bot by scanning the QR code and send a message saying, "Make my clone bot."

[1858] The server receives the initial setup request and generates a new user identity (user ID), which saves the new user's profile in the database. It uses the UUID library to generate a unique ID and saves it in the MySQL database.

[1859] Conversation data collection and storage method

[1860] Users exchange messages with the clonebot on a daily basis, for example, sending a message saying, "I was busy this morning."

[1861] The device forwards the sent message to the server via the LINE API.

[1862] The server parses the received messages and extracts metadata such as content, sender information, and timestamps. It also analyzes the sentiment of the messages using Python's NLTK. The analyzed data is stored in a database using SQLAlchemy.

[1863] Analysis and learning tools

[1864] The server periodically retrieves conversation data from the database. This is automated using a scheduler. The retrieved data is analyzed using TensorFlow or PyTorch to generate and update an AI model specific to the user. OpenAI's Fine Tuning API is used for training. The generated and updated model is stored in the database.

[1865] Interacting with Clonebots

[1866] Users can invite clonebots to join group chats in communication applications to interact with other users.

[1867] A friend sends a question in a group chat: "What did you do over the weekend?"

[1868] The device forwards this message to the server via the LINE API.

[1869] The server uses OpenAI's GPT-3 API to generate appropriate replies, taking into account past conversation data and emotional states. The replies are then sent to the user's device via the LINE API and displayed in the group chat.

[1870] Examples of specific examples and prompts

[1871] For example, if a user sends a message to a clonebot on the LINE app saying, "Last night's movie was great," the server will recognize the emotion as "enjoyment" and generate a reply saying, "Yeah, it was really great. What should we watch next?"

[1872] Example prompt sentence:

[1873] "Make a clone bot of me."

[1874] "The movie yesterday was interesting."

[1875] "What did you do over the weekend?"

[1876] In this way, the present invention can realize natural and interactive dialogue that reflects the user's emotions and speaking style. By combining it with an emotion engine, it is possible to provide an even more realistic user experience.

[1877] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1878] Step 1:

[1879] The user adds the clone bot to their friends on the LINE app and sets up the initial settings. At this stage, the user scans the QR code and sends a message saying, "Make my clone bot."

[1880] Input: QR code scan and initial message

[1881] Output: Initialization request

[1882] Step 2:

[1883] The server receives an initial setup request through the LINE API. It uses Node.js to parse this request and generate a new user ID. It uses the UUID library to generate a unique ID.

[1884] Input: Initial Setup Request

[1885] Output: User ID

[1886] Step 3:

[1887] The server stores the generated user ID in a MySQL database and creates a profile for the user, using SQLAlchemy for database operations.

[1888] Input: User ID

[1889] Output: User profile stored in the database

[1890] Step 4:

[1891] Users can exchange messages with the clone bot on a daily basis using the LINE app, for example, sending a message like "I was busy this morning."

[1892] Input: Message from user

[1893] Output: None

[1894] Step 5:

[1895] The device forwards the message to the server via the LINE API, and Node.js is used to receive and analyze the message.

[1896] Input: User's message

[1897] Output: Message received by the server

[1898] Step 6:

[1899] The server parses the received messages, extracting message content and metadata (sender information, timestamp, etc.), and analyzes the emotional state using Python's NLTK library.

[1900] Input: The message received by the server

[1901] Output: Parsed message content and sentiment data

[1902] Step 7:

[1903] The server extracts conversation data and emotion data and stores it in a database. Data is added to the database using SQLAlchemy.

[1904] Input: Parsed message content and sentiment data

[1905] Output: Conversation data and emotion data stored in a database

[1906] Step 8:

[1907] The server periodically retrieves conversation data from the database. This process is automated by a scheduler, and the data is analyzed using TensorFlow and PyTorch.

[1908] Input: Conversation data retrieved from the database

[1909] Output: Analysis results

[1910] Step 9:

[1911] The server uses OpenAI's Fine Tuning API to train a user-specific AI model, then optimizes the model based on the analysis results.

[1912] Input: Analysis results

[1913] Output: A trained artificial intelligence model

[1914] Step 10:

[1915] After the server completes the training, it stores the updated artificial intelligence model in a database.

[1916] Input: A trained artificial intelligence model

[1917] Output: Artificial intelligence model stored in a database

[1918] Step 11:

[1919] A user invites a clonebot to join a group chat in a communication application and allows it to interact with other users.

[1920] Input: None

[1921] Output: None

[1922] Step 12:

[1923] A friend sends a question in a group chat: "What did you do over the weekend?"

[1924] Input: Message from a friend

[1925] Output: None

[1926] Step 13:

[1927] The device forwards this message to the server via the LINE API, and receives the message using Node.js.

[1928] Input: Message from a friend

[1929] Output: Message received by the server

[1930] Step 14:

[1931] The server takes into account past conversation data and emotional state and generates appropriate replies using OpenAI's GPT-3 API.

[1932] Input: Received messages, past conversation data, emotion data

[1933] Output: The generated reply

[1934] Step 15:

[1935] The server generates a reply and sends it to the user's device via the LINE API, where it is displayed in the group chat.

[1936] Input: Generated reply

[1937] Output: Reply sent to the user's device

[1938] This step allows users to enjoy a natural, interactive dialogue experience that reflects their own emotions and speaking style.

[1939] (Application example 2)

[1940] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1941] In modern autonomous vehicles, users want a more comfortable and stress-free driving experience. However, conventional autonomous driving systems lack the ability to engage in natural dialogue with users and provide emotionally appropriate responses. This can lead to distrust and frustration among users. Furthermore, the lack of a dialogue model tailored to individual users results in repetitive, uniform responses, leading to lower user satisfaction.

[1942] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1943] In this invention, the server includes an initial setting means for user registration, a means for collecting and saving conversation data with the user, a means for analyzing the collected conversation data and learning the user's speaking style and context, a means for training a user-specific model based on the learned data, a means for generating conversation replies using the generated user-specific model, a sentiment analysis means for analyzing the user's sentiment, a means for adjusting the reply content based on the sentiment analysis, and a means for sending the generated replies to the user terminal. This enables recognition of the user's sentiment and a natural dialogue appropriate to that sentiment. Furthermore, training a user-specific dialogue model enables dialogue optimized for each individual user, improving the user experience.

[1944] "Initial setting means" refers to the means by which a user accesses the system and makes the necessary settings.

[1945] "Conversation data" refers to information such as messages and voices exchanged between a user and a system.

[1946] "Collection and storage means" refers to the means for accumulating and managing conversation data obtained from users.

[1947] "Analysis means" refers to a means for analyzing collected conversation data and learning the user's speaking style and context.

[1948] A "training means" is a means for generating a user-specific dialogue model based on the analyzed data.

[1949] The "generation means" is a means for generating conversation replies using the generated user-specific model.

[1950] "Emotion analysis means" is a means for estimating and analyzing emotions from user messages.

[1951] The "reply adjustment means" is a means for appropriately adjusting the reply content generated based on the result of emotion analysis.

[1952] The "transmitting means" is a means for transmitting the generated reply to the user terminal.

[1953] "Virtual driver" refers to a virtual driver bot that interacts with the user inside an autonomous vehicle and provides information about the driving situation and destination.

[1954] This invention relates to the construction of a system for use in autonomous vehicles, specifically for analyzing user emotions and providing natural dialogue accordingly. The system consists of the following main components:

[1955] User Device

[1956] The user terminal is a smartphone or an in-vehicle infotainment system, and is the device through which the user accesses the system and performs initial setup and daily interaction. Initial setup includes installing applications and setting up a user-specific profile. The "initial setup means" allows the user to register the necessary information for the system.

[1957] server

[1958] The server is the main device that stores and manages the collected conversation data, and is responsible for analysis and training. Specifically, it performs the following processes:

[1959] Conversation data collection and storage: Conversation data with the user is acquired and stored in a database. For example, a message such as "The roads are crowded" is stored.

[1960] Analysis method: Analyzes collected conversation data to learn about the user's speaking style and context. This analysis is done using machine learning algorithms and natural language processing techniques.

[1961] Training method: A dialogue model specific to the user is generated based on the analyzed data. This training is performed using OpenAI's Fine tuning API, among other tools.

[1962] Sentiment analysis: The Sentimental library is used to estimate and analyze the sentiment of users' messages. For example, if a user says "I'm a little tired," the sentiment is analyzed as negative.

[1963] Reply adjustment: The generated reply content is adjusted based on the results of sentiment analysis, allowing for the generation of appropriate reply content according to the user's emotions.

[1964] Transmission method: The generated reply is sent to the user's device. For example, a message such as "Are you tired? Shall we take a break at the next service area?" is sent to the user's device.

[1965] Specific examples

[1966] For example, if a user in an autonomous vehicle says, "Could you please speed up a bit?", the system will analyze the message using its "emotion analysis means" and generate an appropriate reply if necessary. The reply generated will be, "We can't speed up, but we are ensuring a safe route," and sent to the user's device. In this way, users can experience natural dialogue that reflects their emotions.

[1967] Examples of prompts include:

[1968] "If a user says they're tired, suggest an appropriate place to rest."

[1969] "If the user's sentiment is positive, recommend them the next tourist attraction they should visit."

[1970] This invention allows users to travel more comfortably and stress-free in self-driving vehicles, and is expected to improve user satisfaction by providing emotion-based dialogue.

[1971] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1972] Step 1:

[1973] Initial Setup

[1974] The user installs the application on their own device and performs initial setup. Specifically, they start the application and enter the necessary information. The entered information is sent to the server via the "initial setup means." The server uses this information to generate a user-specific profile and stores it in a database. Input: User information, Output: User profile.

[1975] Step 2:

[1976] Conversation data collection

[1977] The user starts a conversation on the device. The device sends the user's message to the server via the LINE API or the in-car infotainment system. The server stores the received message in a database using the "conversation data collection and storage means." Input: User's message, Output: Stored conversation data.

[1978] Step 3:

[1979] Analysis of conversation data

[1980] The server periodically retrieves conversation data from the database. The "analysis means" analyzes the collected conversation data and learns the user's speaking style and context. In this process, machine learning algorithms (e.g., natural language processing technology) are used. Input: saved conversation data, Output: analysis results.

[1981] Step 4:

[1982] Training the dialogue model

[1983] A "training method" generates a user-specific dialogue model based on the analyzed data. This training is performed using OpenAI's Fine tuning API, etc. After training, the generated model is saved in a database. Input: Analysis results, Output: User-specific dialogue model.

[1984] Step 5:

[1985] Emotion analysis

[1986] The server analyzes new messages received from users using the "sentiment analysis means." This analysis uses the Sentimental library to determine whether the user's emotion is "positive" or "negative." For example, if the message is "tired," it is determined to be negative. Input: User's message, Output: Emotion determination result.

[1987] Step 6:

[1988] Generate and moderate replies

[1989] The "generation means" generates a conversational reply using a dialogue model dedicated to the user. The "reply adjustment means" then adjusts the reply content based on the results of the emotion analysis. For example, if the user says "I'm a little tired," the reply content is adjusted to "Shall we take a break at the next service area?" Input: Emotion determination result and user-specific dialogue model, Output: Adjusted reply.

[1990] Step 7:

[1991] Sending a reply

[1992] The "transmission means" sends the generated and adjusted reply to the user's terminal. The user can check the reply through the terminal. Input: adjusted reply, Output: message displayed on the user's terminal.

[1993] Step 8:

[1994] User experience updates

[1995] The collected new conversation data and the results of sentiment analysis are periodically learned and updated by the "analysis means" and "training means." This ensures that the user-specific dialogue model always reflects the latest conversation patterns and emotional state. Input: New conversation data and sentiment analysis results, Output: Updated dialogue model.

[1996] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1997] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1998] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1999] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2000] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2001] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2002] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2003] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2004] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2005] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2006] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2007] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2008] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2009] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2010] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2011] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2012] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2013] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2014] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2015] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2016] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2017] The following is further disclosed regarding the above embodiment.

[2018] (Claim 1)

[2019] An initial setting means that users can register;

[2020] A means of collecting and storing conversation data with users;

[2021] A means of analyzing collected conversation data and learning the user's speaking style and context;

[2022] A means to train a user-specific model based on the learned data; and

[2023] a means for generating conversation replies using the generated user-specific model; and

[2024] means for transmitting the generated reply to the user terminal;

[2025] A system including:

[2026] (Claim 2)

[2027] 10. The system of claim 1, further comprising means for periodically analyzing the collected conversational data and updating the model based on the learned data.

[2028] (Claim 3)

[2029] The system of claim 1, further comprising means for a user to invite a clone bot to a group chat and for the clone bot to automatically generate and send replies to messages from other users that mimic the user's speaking style.

[2030] "Example 1"

[2031] (Claim 1)

[2032] An initial setting means that users can register;

[2033] A means of collecting and storing conversation data with users;

[2034] A means of analyzing collected conversation data and learning the user's speaking style and context;

[2035] A means to train a user-specific generative AI model based on the learned data; and

[2036] a means for generating conversation replies using a generated user-specific generative AI model; and

[2037] means for transmitting the generated reply to the user terminal;

[2038] A system including:

[2039] (Claim 2)

[2040] 10. The system of claim 1, further comprising means for periodically analyzing the collected conversational data and updating the generative AI model based on the learned data.

[2041] (Claim 3)

[2042] 10. The system of claim 1, further comprising means for a user to invite a clonebot to a group chat and for the clonebot to automatically generate and send replies to messages from other users that mimic the user's speaking style.

[2043] "Application Example 1"

[2044] (Claim 1)

[2045] An initial setting means that users can register;

[2046] A means of collecting and storing conversation data with users;

[2047] A means of analyzing collected conversation data and learning the user's speaking style and context;

[2048] A means to train a user-specific model based on the learned data; and

[2049] a means for generating conversation replies using the generated user-specific model; and

[2050] A means for learning user preferences using the above means and making recommendations about food and drink;

[2051] means for transmitting the generated recommendation content to a user terminal;

[2052] A system including:

[2053] (Claim 2)

[2054] 10. The system of claim 1, further comprising means for periodically analyzing the collected conversational data and updating the model based on the learned data.

[2055] (Claim 3)

[2056] 10. The system of claim 1, further comprising means for a user to invite a clonebot to a group chat and for the clonebot to automatically generate and send replies to messages from other users that mimic the user's speaking style.

[2057] "Example 2: Combining Emotion Engines"

[2058] (Claim 1)

[2059] An initial setting means that users can register;

[2060] A means of collecting and storing conversation data with users;

[2061] A means of analyzing collected conversation data and learning the user's speaking style and context;

[2062] A means to train a user-specific artificial intelligence model based on the learned data; and

[2063] a means for generating conversation replies using the generated user-specific artificial intelligence model; and

[2064] means for transmitting the generated reply to the user terminal;

[2065] means for analyzing the emotional state of the message using an emotion engine;

[2066] A system including:

[2067] (Claim 2)

[2068] 10. The system of claim 1, further comprising means for periodically analyzing the collected conversation data and updating the artificial intelligence model based on the learned data.

[2069] (Claim 3)

[2070] 10. The system of claim 1, further comprising means for a user to invite a clonebot to a communication application and for the clonebot to automatically generate and send replies to messages from other users that mimic the user's speaking style.

[2071] "Application example 2 when combining emotion engines"

[2072] (Claim 1)

[2073] An initial setting means that users can register;

[2074] A means of collecting and storing conversation data with users;

[2075] A means of analyzing collected conversation data and learning the user's speaking style and context;

[2076] A means to train a user-specific model based on the learned data; and

[2077] a means for generating conversation replies using the generated user-specific model; and

[2078] An emotion analysis means for analyzing the emotion of a user;

[2079] means for tailoring reply content based on sentiment analysis;

[2080] means for transmitting the generated reply to the user terminal;

[2081] A system including:

[2082] (Claim 2)

[2083] 10. The system of claim 1, further comprising means for periodically analyzing the collected conversational data and updating the model based on the learned data.

[2084] (Claim 3)

[2085] 10. The system of claim 1, further comprising means for allowing a user to invite a virtual driver to a group chat and for automatically generating and sending replies to messages from other users that mimic the user's speaking style. [Explanation of symbols]

[2086] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. An initial setting means that users can register; A means of collecting and storing conversation data with users, A means of analyzing collected conversation data and learning the user's speaking style and context; A means to train a user-specific model based on the learned data; and a means for generating conversation replies using the generated user-specific model; and means for transmitting the generated reply to the user terminal; A system including:

2. 10. The system of claim 1, further comprising means for periodically analyzing collected conversation data and updating the model based on the learned data.

3. The system of claim 1, further comprising means for a user to invite a clone bot to a group chat and for the clone bot to automatically generate and send replies to messages from other users that mimic the user's speaking style.

Images