system

Abstract

Provide a system. An input means for a user to input basic information; a storage means for storing the input basic information; generating means for generating an interactive avatar based on the stored basic information; an interaction means for a user to interact with the generated interaction avatar; an analysis means for storing and analyzing the content of the dialogue; updating means for updating an interactive avatar based on the analysis result of said analyzing means; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, the number of people feeling loneliness and social isolation is increasing. In particular, for elderly people living alone and individuals with limited social interactions, a lack of daily interaction can have a negative impact on their mental health and quality of life. However, conventional dialogue systems have struggled to deeply understand individual users' interests and grow and adapt over time. Therefore, there is a need for technology that provides effective dialogue partners to reduce loneliness. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means: An input means for a user to input basic information is provided, and a storage means for storing this basic information is also provided. Further, a generation means is provided for generating an interactive avatar based on the stored basic information. An interaction means is provided for a user to interact with the generated interactive avatar, and an analysis means is provided for storing and analyzing the content of the interaction. A system is provided that includes an update means for updating the interactive avatar based on the analysis results. The system also includes a means for sequentially saving and analyzing the interaction history, a means for adaptively changing the avatar's interaction content and profile based on the analysis results, and a growth means for building a continuous relationship with the user. This enables the interactive avatar to deeply understand the user's interests and concerns, and to grow and adapt over time, contributing to a reduction in feelings of loneliness.

[0006] "Input means" means a means by which a user inputs personal data or other information into the system.

[0007] "Storage means" refers to a means for storing input personal data and interaction history in a database or other storage.

[0008] The "generation means" is a means for generating an interactive avatar based on the stored personal data.

[0009] The "interaction means" is a means for carrying out a two-way conversation between the generated interactive avatar and the user.

[0010] The "analysis means" is a means for saving the contents of the dialogue and analyzing it using natural language processing technology or the like.

[0011] The "update means" is a means for updating the profile and dialogue content of the dialogue avatar based on the analysis results.

[0012] "Growth means" refers to a means for evolving the conversational avatar over time in accordance with the user's interests and concerns.

[0013] The "dialogue history" is a record of past conversations between a user and a dialogue avatar.

[0014] A "natural language processing engine" is a software engine that analyzes user text input and performs language understanding and response generation.

[0015] A "profile" is a collection of data that includes the personality and preferences of an interactive avatar, as well as the user's interests and concerns. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] MODE FOR CARRYING OUT THE INVENTION

[0038] System Configuration and Functions

[0039] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user. The operation of the system is explained in detail below.

[0040] 1. User Registration Process

[0041] A user first logs in to the system and enters basic information, such as name, age, gender, hobbies, interests, etc. To enter this information, the user uses a web form as an input method.

[0042] The terminal receives the information entered by the user and sends it to the server, which then analyzes the received data in a format such as JSON and stores it in a database.

[0043] 2. Initial avatar generation and conversation start

[0044] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[0045] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0046] 3. Facilitating and managing the dialogue

[0047] The terminal sends the message entered by the user to the server, which then analyzes the received message using a natural language processing engine and generates an appropriate reply. The generated reply is then sent back to the terminal and displayed to the user.

[0048] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[0049] 4. Avatar Growth and Evolution

[0050] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[0051] The avatar grows over time, allowing for more personalized interactions, and this growth mechanism allows the user and avatar to build a closer relationship.

[0052] Specific examples

[0053] Initial registration and interaction

[0054] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0055] 2. The device receives this information and sends it to the server, which stores it in a database.

[0056] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0057] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0058] 5. The user replies, "I like cooking."

[0059] Ongoing dialogue

[0060] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[0061] 2. The terminal displays this message to the user, who then replies.

[0062] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0063] In this way, the system can continually improve its interactions with the user, building a deeper understanding and connection.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[0067] Step 2:

[0068] The terminal receives the input user information and transmits it to the server.

[0069] Step 3:

[0070] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[0071] Step 4:

[0072] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[0073] Step 5:

[0074] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[0075] Step 6:

[0076] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[0077] Step 7:

[0078] The user enters an initial message in the text box and clicks the "Send" button.

[0079] Step 8:

[0080] The terminal receives the user's input message and sends it to the server.

[0081] Step 9:

[0082] The server uses a natural language processing engine to analyze the received message and generate an appropriate reply, such as "I see you enjoy cooking. What did you make recently?"

[0083] Step 10:

[0084] The server generates a reply and sends it to the terminal.

[0085] Step 11:

[0086] The terminal displays the received reply to the user.

[0087] Step 12:

[0088] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[0089] Step 13:

[0090] The server uses analytical means to analyze the stored dialogue history, for example, to determine that the user frequently talks about Italian food.

[0091] Step 14:

[0092] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[0093] Step 15:

[0094] Users interact with their avatars periodically, and the avatars evolve based on the user's interests. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] In modern society, the number of people feeling lonely is increasing. Conventional systems have difficulty providing truly personalized interactions with users and lack the means to build lasting relationships. In this situation, there is a need for a system that can reduce users' feelings of loneliness and provide a comfortable interaction experience.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes: input means for a user to input basic information; storage means for storing the input basic information; generation means for generating a conversational avatar based on the stored basic information; dialogue means for the user to dialogue with the generated conversational avatar; means for sending the user's message to the server and analyzing it with a natural language processing engine to generate an appropriate response; analysis means for storing and analyzing the content of the dialogue; update means for updating the conversational avatar based on the analysis result of the analysis means; and means for storing a history of ongoing dialogue in a database and analyzing it to improve the quality of future dialogues. This allows the user to have a personalized conversation experience and reduces feelings of loneliness.

[0100] "Input means" refers to the means by which system users input basic information (such as name, age, gender, hobbies, interests, etc.), such as using a web form.

[0101] The "storage means" refers to a means for storing the basic information entered by the user and the dialogue history in a database, and refers to a database system such as MySQL (registered trademark) or MongoDB.

[0102] The "generation means" refers to a means for generating an interactive avatar based on the stored basic information, and uses a generative AI model (e.g., GPT-3 (registered trademark)) to create the characteristics of the interactive avatar.

[0103] The "interaction means" is a means for a user to interact with the generated interaction avatar, and includes a text box and an interaction interface displayed on the terminal.

[0104] A "natural language processing engine" is a system that analyzes a user's message and generates an appropriate response based on that analysis, using technologies such as spaCy and BERT.

[0105] The "analysis means" refers to a means for saving and analyzing the content of a dialogue, and a system for analyzing the saved dialogue history.

[0106] The "update means" is a means for updating the interactive avatar based on the analysis results of the analysis means, and for example, modifies the avatar profile and the interactive script using a reinforcement learning model.

[0107] "Growth means" refers to means for updating the profile of the interactive avatar and enabling more personalized interactions in order to build a continuous relationship with the user.

[0108] A "database" is a system for managing stored basic information and interaction history, and for searching and extracting information as needed, including, for example, SQL databases and NoSQL databases.

[0109] A "generative AI model" is a machine learning model used to generate a conversational avatar and create a conversation script based on basic user information, including, for example, GPT-3.

[0110] System Configuration and Functions

[0111] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[0112] User Registration Process

[0113] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, and interests using a web form. The device then formats this information into JSON format and sends it to the server via an HTTP POST request. The server then parses the received data and stores it in a database system such as MySQL or MongoDB.

[0114] Initial avatar generation and conversation start

[0115] The server generates a conversational avatar using a generative AI model (e.g., GPT-3) using Python and TENSORFLOW (registered trademark) based on the stored user basic information. Based on the generated avatar, the server generates an initial conversation script and sends it to the device. The device receives this script and displays it to the user through a conversational interface using HTML and JavaScript (registered trademark). For example, it displays a message such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0116] Facilitating and managing dialogue

[0117] The user enters a message in the displayed text box and presses the send button. For example, the user enters "I like cooking." The device sends this input message to the server. The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The generated reply is sent back to the device and displayed to the user. For example, a message such as "What dish have you cooked recently?" is displayed.

[0118] Avatar Growth and Evolution

[0119] The server stores all dialogue history in a database and analyzes it using analytical tools. This analysis identifies changes in the user's interests and new concerns. Utilizing a natural language processing engine, the server classifies the user's interests using a clustering algorithm (e.g., the K-means algorithm). Based on the results, the avatar's profile and dialogue script are updated. For example, if the user frequently talks about Italian food, the next dialogue might include a question such as, "Have you found any new Italian recipes?"

[0120] Specific examples

[0121] 1. Example of user registration and dialogue initiation

[0122] Users log in to the system and enter basic information such as their name, age, and hobbies.

[0123] The device receives this information and sends it to the server, which stores it in a database.

[0124] The server generates an avatar based on the user information and creates an initial dialogue script.

[0125] The terminal receives the generated dialogue script and displays it to the user. The prompt text displayed is "Hello, I'm your new friend. What are your recent hobbies?"

[0126] The user replies, "I like cooking."

[0127] 2. An example of ongoing dialogue

[0128] The server analyzes the user's dialogue message and generates an appropriate response, for example, "What dish did you cook recently?"

[0129] The terminal displays this message to the user, who then replies.

[0130] The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar's profile will be updated, and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0131] This allows the system to continually improve its interactions with the user, building a deeper understanding and relationship.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1:

[0134] Users log in to the system and enter basic information such as name, age, gender, hobbies, interests, etc. using a web form built with HTML and JavaScript.

[0135] Input: Basic information (name, age, gender, hobbies, interests)

[0136] Output: Basic information data in JSON format

[0137] Step 2:

[0138] The terminal receives the basic information entered by the user, converts it into JSON format, and sends it to the server via an HTTP POST request.

[0139] Input: Basic information entered by the user into the web form

[0140] Output: JSON format data sent to the server

[0141] Step 3:

[0142] The server parses the received basic information data in JSON format and stores it in a database (e.g., MySQL or MongoDB). This storage process is performed transactionally to ensure data integrity.

[0143] Input: Basic information data in JSON format sent from the terminal

[0144] Output: Basic information stored in the database

[0145] Step 4:

[0146] The server generates a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored basic information. The generative AI model is operated using Python and TensorFlow.

[0147] Input: Basic information stored in the database

[0148] Output: Generated conversational avatar

[0149] Step 5:

[0150] The server generates an initial dialogue script based on the generated dialogue avatar, using NLG (Natural Language Generation) technology.

[0151] Input: Generated conversation avatar

[0152] Output: Initial interaction script

[0153] Step 6:

[0154] The terminal receives the initial dialogue script sent from the server and displays it to the user through a dialogue interface using HTML and JavaScript. For example, it displays "Hello, I'm your new friend. What are your recent hobbies?"

[0155] Input: Initial interaction script sent by the server

[0156] Output: The interactive interface and initial message displayed to the user

[0157] Step 7:

[0158] The user enters a message in the text box of the dialogue interface and presses the send button. For example, the user enters "I like cooking."

[0159] Input: The message the user types in the text box

[0160] Output: User message to be sent

[0161] Step 8:

[0162] The terminal transmits the message entered by the user to the server.

[0163] Input: The message entered by the user

[0164] Output: User message sent to the server

[0165] Step 9:

[0166] The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The analysis mainly involves semantic analysis and contextual understanding of the text. For example, the message generated is "What dishes have you cooked recently?"

[0167] Input: User message sent from the terminal

[0168] Output: The generated reply message

[0169] Step 10:

[0170] The server sends the generated reply message to the terminal.

[0171] Input: The generated reply message

[0172] Output: Reply message sent to the terminal

[0173] Step 11:

[0174] The terminal receives the reply message sent from the server and displays it to the user through the dialogue interface.

[0175] Input: Reply message sent from the server

[0176] Output: The reply message that is displayed to the user

[0177] Step 12:

[0178] The server stores all interaction history in a database. The stored interaction history is used to identify user interests and new concerns through analytical means, such as clustering algorithms and natural language processing techniques.

[0179] Input: Dialogue history

[0180] Output: Dialogue history stored in a database

[0181] Step 13:

[0182] The server updates the avatar's profile and dialogue script based on the analysis results. This uses a reinforcement learning model to allow the avatar to grow incrementally, making the dialogue with the user more personalized. For example, if the user frequently talks about Italian food, the next dialogue topic might be, "Have you found any new Italian recipes?"

[0183] Input: Analysis results

[0184] Output: Updated avatar profile and dialogue scripts

[0185] Through each of the above steps, the system continuously improves the interaction with the user, helping to reduce the user's sense of loneliness.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Conventional conversational avatar systems focus on conversational functions to reduce feelings of loneliness, but do not sufficiently consider the purchasing experience and product recommendations that users gain through conversation. Furthermore, the lack of a dynamic product recommendation function based on the user's interests limits the improvement of the purchasing experience. This has led to the challenge of making it difficult to accurately tap into the user's latent purchasing motivation.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes input means for a user to input basic information, storage means for storing the input basic information, generation means for generating an interactive avatar based on the stored basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for storing and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, recommendation means for improving the purchasing experience using the interactive avatar, generation means for recommending products based on the user's input, and display means for displaying the products recommended by the recommendation means. This enables product recommendations based on the user's dialogue history and interests, thereby improving the purchasing experience while reducing feelings of loneliness through dialogue.

[0191] A "user" is an individual who uses the system to interact with interactive avatars and receive product recommendations.

[0192] "Basic information" refers to personal data such as the user's name, age, gender, hobbies, and interests.

[0193] "Input means" refers to the interface or device that allows users to input basic information into the system.

[0194] "Storage means" refers to a technical element that stores the input basic information and dialogue content in a database.

[0195] The "generation means" is a function or system for automatically creating a conversational avatar based on the stored basic information.

[0196] "Interaction means" refers to an interface or system that allows the user to exchange messages with the generated interactive avatar.

[0197] "Analysis means" refers to the technology or algorithm used to analyze the history and content of interactions and update or improve the interaction avatar.

[0198] The "update means" is a function for correcting and updating the profile and dialogue content of the dialogue avatar based on the results of the analysis means.

[0199] "Recommendation means" is a function that uses a conversational avatar to suggest the most suitable products to the user in order to improve the purchasing experience.

[0200] The "display means" refers to a display or interface for visually presenting the generated dialogue script and recommended products to the user.

[0201] "Purchasing experience" refers to the experience a user has during the process of selecting a product, purchasing it, or obtaining information about it.

[0202] MODE FOR CARRYING OUT THE INVENTION

[0203] System Configuration and Functions

[0204] This invention is a system that provides interactive avatars using generative AI to improve the shopping experience while reducing feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[0205] User Registration Process

[0206] A user first logs in to the system and enters basic information, including name, age, gender, hobbies, and interests. The user uses a web form as an input method. The device receives the information entered by the user and sends it to the server. The server parses the received data into a format such as JSON and stores it in a database.

[0207] Initial avatar generation and conversation start

[0208] The server generates a conversational avatar based on the stored basic information. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the terminal. The terminal receives this script and displays a conversation screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0209] Facilitating and managing dialogue

[0210] The terminal sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., OpenAI (registered trademark) API) and generates an appropriate reply. The generated reply is sent back to the terminal and displayed to the user. The entire history of this dialogue is accumulated in a database by the storage means. This stored history is analyzed by the analysis means to improve the quality of future dialogues.

[0211] Product recommendation feature implementation

[0212] The product recommendation function is provided to improve the purchasing experience through interaction between the interactive avatar and the user. The server uses a generating means to recommend optimal products to the user based on the user's basic information and interaction history. The recommended products are visually presented to the user through the display means of the terminal.

[0213] Avatar Growth and Evolution

[0214] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated using an update means. The avatar grows over time, enabling more personalized dialogue.

[0215] Example

[0216] Initial registration and interaction

[0217] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0218] 2. The device receives this information and sends it to the server, which stores it in a database.

[0219] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0220] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0221] 5. The user replies, "I like cooking."

[0222] Ongoing dialogue and product recommendations

[0223] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[0224] 2. The terminal displays this message to the user, who then replies.

[0225] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0226] 4. Furthermore, to improve the shopping experience, the avatar will recommend products such as, "Today's special Italian cooking ingredients are here."

[0227] Prompt Sentence Examples

[0228] "User name: Taro Yamamoto

[0229] Age: 30

[0230] Gender: Male

[0231] Hobbies: Fashion, gadgets

[0232] Interests: Latest trends, innovative technologies

[0233] Please generate a suitable initial interaction script for this user."

[0234] In this way, the present invention is optimized to reduce the user's sense of loneliness and improve the purchasing experience through interaction.

[0235] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0236] Step 1:

[0237] A user logs into the system and fills in basic information such as name, age, gender, hobbies, and interests in a web form.

[0238] Input: User's basic information (name, age, gender, hobbies, interests, etc.)

[0239] Output: Basic information sent to the terminal in JSON format

[0240] Step 2:

[0241] The device sends the basic information entered by the user to the server, which then parses the information in JSON format and stores it in a database.

[0242] Input: User basic information (JSON format)

[0243] Output: Basic information stored in the database

[0244] Step 3:

[0245] The server generates an interactive avatar based on the stored basic information.

[0246] Input: Basic information (from database)

[0247] Output: Generated conversational avatar

[0248] Step 4:

[0249] The server creates an initial interaction script for the avatar and sends it to the terminal, which receives the script and displays it to the user.

[0250] Input: Basic information of the generated conversational avatar

[0251] Output: The initial interaction script is displayed to the user

[0252] Step 5:

[0253] The user inputs and sends a message to the avatar.

[0254] Input: User interaction message

[0255] Output: Message sent to the terminal

[0256] Step 6:

[0257] The terminal sends the user's message to the server, which uses a natural language processing engine to analyze the message and generate an appropriate reply.

[0258] Input: User interaction message

[0259] Output: The generated avatar returned by the server

[0260] Step 7:

[0261] The server generates a reply and sends it to the terminal, which displays the reply to the user.

[0262] Input: Reply message from server

[0263] Output: Avatar reply shown to the user

[0264] Step 8:

[0265] The server stores the content of the dialogue in a database and analyzes the dialogue history.

[0266] Input: Interaction history between user and avatar

[0267] Output: Parsed data

[0268] Step 9:

[0269] The server updates the interactive avatar based on the analysis results and generates new scripts and profiles.

[0270] Input: Analyzed interaction history data

[0271] Output: Updated interactive avatar scripts and profiles

[0272] Step 10:

[0273] The server generates a script for recommending products based on the user's basic information and interaction history, and transmits the recommended products to the terminal, which then displays the recommended products to the user.

[0274] Input: User basic information, interaction history

[0275] Output: Recommended product information

[0276] Through this series of steps, users can reduce their sense of loneliness through interaction with a conversational avatar, further improving their purchasing experience.

[0277] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0278] MODE FOR CARRYING OUT THE INVENTION

[0279] System Configuration and Functions

[0280] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user. The operation of the system is explained in detail below.

[0281] 1. User registration and basic information entry

[0282] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, interests, etc. To do so, the user uses a web form as an input method.

[0283] The terminal receives the information entered by the user and sends it to the server, which analyzes the received data and stores it in a database. The storage means used here is the one used.

[0284] 2. Initial avatar generation and conversation start

[0285] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[0286] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0287] 3. Dialogue progression and emotional engine utilization

[0288] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue."

[0289] The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as, "You seem tired. How do you relax?" The generated reply is sent back to the device and displayed to the user.

[0290] 4. Saving and analyzing dialogue history

[0291] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[0292] The server analyzes the stored dialogue history using an analysis means. By using a natural language processing engine and an emotion engine in combination, it identifies changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[0293] 5. Avatar Growth and Relationship Development

[0294] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "fatigue" or "stress," the avatar will add more conversation scripts about relaxation methods and stress management.

[0295] Users regularly interact with their avatars, which evolve based on the user's interests and emotions. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens. This allows users to receive emotional support and reduce feelings of loneliness.

[0296] Specific examples

[0297] Initial registration and interaction

[0298] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0299] 2. The device receives this information and sends it to the server, which stores it in a database.

[0300] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0301] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0302] 5. The user replies, "I like cooking."

[0303] Continuous dialogue and emotion recognition

[0304] 1. The server analyzes the user's dialogue message and generates the following appropriate response: For example, it generates a reply such as, "I see you enjoy cooking. What dish have you cooked recently?"

[0305] 2. The terminal displays this message to the user, who then replies.

[0306] 3. The server stores the dialogue history and continues to analyze it. If the user types, "I'm very tired today," the emotion engine recognizes "fatigue" and generates a response accordingly. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[0307] 4. The server updates the avatar's profile and dialogue scripts, taking into account the user's dialogue history and emotional state.

[0308] In this way, the system can continuously improve its dialogue with the user and, by leveraging the emotion engine, provide appropriate support depending on the user's emotional state.

[0309] The processing flow will be explained below.

[0310] Step 1:

[0311] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[0312] Step 2:

[0313] The terminal receives the input user information and transmits it to the server.

[0314] Step 3:

[0315] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[0316] Step 4:

[0317] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[0318] Step 5:

[0319] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[0320] Step 6:

[0321] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[0322] Step 7:

[0323] The user enters an initial message in the text box and clicks the "Send" button.

[0324] Step 8:

[0325] The terminal receives the user's input message and sends it to the server.

[0326] Step 9:

[0327] The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, if the message says "I'm very tired today," the emotion is recognized as "fatigue."

[0328] Step 10:

[0329] The server generates an appropriate response based on the analysis results of the emotion engine, for example, "You seem tired. How do you relax?"

[0330] Step 11:

[0331] The server generates a reply and sends it to the terminal.

[0332] Step 12:

[0333] The terminal displays the received reply to the user.

[0334] Step 13:

[0335] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[0336] Step 14:

[0337] The server uses analytical means to analyze the stored dialogue history and emotional state, and uses a natural language processing engine and an emotion engine in combination to identify changes in the user's interests and emotions.

[0338] Step 15:

[0339] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[0340] Step 16:

[0341] Users interact with the avatar periodically, and the avatar evolves based on the user's interests and emotions. The server continuously analyzes and updates the interaction data, deepening the user's relationship with the avatar.

[0342] Example 2

[0343] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] In modern society, many people feel lonely, and effective countermeasures are needed. Furthermore, there is a need for a system that can accurately grasp a user's emotional state and alleviate loneliness through dialogue. Conventional systems can only provide one-way dialogue based on basic user information, making it difficult to dynamically respond to changes in the user's emotional state or interests.

[0345] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for the user to input basic information, a data storage means for saving the input basic information, a generation means for generating an interactive avatar based on the saved basic information, an interaction means for the user to interact with the generated interactive avatar, a natural language processing means for analyzing the user's message, an emotion analysis means for recognizing the user's emotion, and an update means for updating the interactive avatar based on the analysis result. This enables dynamic interaction according to the user's emotional state and interests, thereby reducing feelings of loneliness.

[0346] "User" refers to the end user of the system who provides basic information and interactive input.

[0347] "Input means" refers to the interface through which users input basic information, such as a web form or an application screen.

[0348] "Data storage means" refers to a mechanism for retaining the basic information entered, including databases and cloud storage.

[0349] "Generation means" refers to the part of the system that has the function of generating an interactive avatar based on stored basic information, and may use a generative AI model.

[0350] "Interactive means" refers to a mechanism for communication between the generated interactive avatar and the user, such as an interactive screen or chat interface.

[0351] "Natural language processing means" refers to technology for analyzing and understanding a user's message, including a natural language processing engine.

[0352] "Emotion analysis means" refers to technology for recognizing the emotional state of a user from their message, and this corresponds to the emotion engine.

[0353] The "update means" refers to a system part that has the function of dynamically changing the profile and dialogue script of the dialogue avatar based on the analysis results.

[0354] "Dialogue history" refers to a record of communication between a user and a dialogue avatar, including saved messages and responses.

[0355] "Analysis means" refers to technology for analyzing stored interaction history and identifying changes in a user's interests and emotions.

[0356] "Growth means" refers to a mechanism for evolving the profile and script of an interactive avatar through continuous dialogue with the user.

[0357] MODE FOR CARRYING OUT THE INVENTION

[0358] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user.

[0359] 1. User registration and basic information entry

[0360] Users log in to the system and enter basic information such as name, age, gender, hobbies, and interests using a web form. The device receives the entered information and sends it to the server. The server analyzes the received data and stores it in a database. The database used here is, for example, MySQL.

[0361] 2. Initial avatar generation and conversation start

[0362] The server generates a conversational avatar based on the stored basic information. A generative AI model (e.g., GPT-3) is used for this generation. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the device. The device receives this script and displays a conversation screen to the user. The user begins a conversation with the avatar by entering a message in the text box and sending it.

[0363] 3. Dialogue progression and emotional engine utilization

[0364] The terminal sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., NLTK). Furthermore, it recognizes the user's emotional state using an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer). For example, a message such as "I'm very tired today" is recognized as "fatigue." The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as "You're tired, aren't you? How do you relax?" The generated reply is sent from the server to the terminal, and the terminal displays it to the user.

[0365] 4. Saving and analyzing dialogue history

[0366] The server stores the dialogue history in a database. The stored dialogue history is used as an analytical tool to improve the quality of future dialogues. The server uses a natural language processing engine and an emotion engine to analyze the stored dialogue history and identify changes in the user's interests and emotions. Based on the analysis results, the dialogue avatar's profile and dialogue script are dynamically updated.

[0367] 5. Avatar Growth and Relationship Development

[0368] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "tired" or "stressed," the avatar will add more conversation scripts about relaxation methods and stress management. As the user regularly interacts with the avatar and the avatar evolves based on the user's interests and emotions, the user can receive emotional support and reduce feelings of loneliness.

[0369] Specific examples

[0370] Initial registration and interaction

[0371] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0372] 2. The device sends this information to the server, which stores it in a database.

[0373] 3. The server generates an avatar based on the saved user information and creates an initial interaction script.

[0374] 4. The terminal receives the dialogue script sent from the server and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0375] 5. The user replies, "I love cooking."

[0376] Continuous dialogue and emotion recognition

[0377] 1. The server analyzes the user's dialogue message and generates an appropriate response, such as "I see you enjoy cooking. What did you cook recently?"

[0378] 2. The terminal displays this message to the user, who then replies.

[0379] 3. The server stores and continuously analyzes the conversation history. For example, if a user types, "I'm very tired today," the emotion engine recognizes this and generates a response tailored to the user's needs. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[0380] 4. The server updates the avatar's profile and dialogue script based on the analysis results.

[0381] Prompt Sentence Examples

[0382] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[0383] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0384] The flow of this system's program processing

[0385] Step 1: Enter basic user information

[0386] Input: A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests into a web form.

[0387] Processing: The device receives the entered information and sends it to the server using an HTTP POST request.

[0388] Output: The server parses the received data and stores it in a database.

[0389] Specific operation: The server executes an SQL INSERT statement using a storage means, for example, in a MySQL database, to save the data.

[0390] Step 2: Generate the initial avatar

[0391] Input: The server retrieves basic information stored in a database.

[0392] Processing: The server generates an interactive avatar based on the stored basic information. It uses a generative AI model (e.g., GPT-3) to create an avatar profile.

[0393] Output: Generates an initial dialogue script for the generated avatar.

[0394] Specific operation: The user's basic information is input into the generative AI model as a prompt sentence, and an appropriate dialogue script is output.

[0395] Step 3: Send the initial interaction script

[0396] Input: The generated avatar and initial dialogue script.

[0397] Processing: The server sends the generated dialogue script to the terminal.

[0398] Output: The terminal receives the script and displays an interactive screen to the user.

[0399] Specific operation: The terminal uses HTML and JavaScript to render the interactive screen and display the script.

[0400] Step 4: Start interacting with the user

[0401] Input: The user types a message in the text box and sends it.

[0402] Processing: The terminal receives the user's input message and sends it to the server.

[0403] Output: The server passes the received message to a natural language processing engine (e.g., NLTK) for analysis.

[0404] Specific operation: The server passes the message to a natural language processing engine, performs tokenization and morphological analysis, and obtains the analysis results.

[0405] Step 5: Recognizing your emotional state

[0406] Input: Parsed message data.

[0407] Processing: The server uses an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotional state.

[0408] Output: Emotional state recognition results.

[0409] Specific operation: The server passes the analysis results to the emotion engine and outputs the emotional state (e.g., sadness, joy, anger, etc.).

[0410] Step 6: Generate an appropriate reply

[0411] Input: Emotional state recognition results.

[0412] Processing: The server again uses the generative AI model to generate an appropriate reply.

[0413] Output: The generated reply message.

[0414] Specific operation: The user's emotional state is input into the generative AI model as a prompt sentence, and an appropriate reply sentence is output.

[0415] Step 7: Send and view the reply message

[0416] Input: The generated reply message.

[0417] Processing: The server generates a reply and sends it to the terminal.

[0418] Output: The terminal displays the reply message to the user.

[0419] Specific operation: The terminal adds the new message to the interactive screen so that the user can view it.

[0420] Step 8: Saving conversation history

[0421] Input: User and avatar interaction history.

[0422] Processing: The server stores the dialogue history in a database.

[0423] Output: The saved interaction history.

[0424] Specific operation: The server saves the conversation history to the database using the SQL INSERT statement.

[0425] Step 9: Analyzing the dialogue history and updating the avatar

[0426] Input: Saved interaction history.

[0427] Processing: The server uses a natural language processing engine and an emotion engine to analyze the dialogue history and identify changes in the user's interests and emotions.

[0428] Output: Profile and dialogue script updates based on the analysis results.

[0429] Specific operation: Based on the analysis results, the avatar's profile and dialogue script are dynamically changed and reflected in the next dialogue with the user.

[0430] Prompt Sentence Examples

[0431] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[0432] (Application example 2)

[0433] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0434] In modern society, user interaction, especially in virtual environments, is an important factor in improving engagement and customer satisfaction. However, conventional dialogue systems have difficulty properly recognizing and responding to users' emotional states. Furthermore, they lack the ability to dynamically adjust dialogue content, which hinders the provision of personalized services. This can lead to users feeling isolated and reduces the quality of their experience in virtual environments. To solve this problem, a system with emotion recognition capabilities and the ability to dynamically adjust dialogue content is needed.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes input means for the user to input basic information, storage means for saving the input basic information, generation means for generating an interactive avatar based on the saved basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for saving and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, and means for dialogue with the user in a virtual store, recognizing the user's emotional state using an emotion engine, and dynamically changing the content of the response. This makes it possible to provide appropriate dialogue and personalized services tailored to the user's emotional state in the virtual environment.

[0436] "User" means an individual human being or end user who uses the system.

[0437] "Basic information" refers to information necessary for generating a conversational avatar, such as the user's name, age, gender, hobbies, and interests.

[0438] "Input means" refers to the interface or device that allows users to input basic information into the system.

[0439] "Storage means" refers to a mechanism or device for storing the input basic information and dialogue history in a database or the like.

[0440] "Generator" refers to the process or function that creates an interactive avatar based on the stored basic information.

[0441] "Interaction means" refers to the functions and mechanisms that enable two-way communication between the generated interactive avatar and the user.

[0442] "Analysis means" refers to methods and technologies for analyzing the content of a conversation and understanding the user's emotional state and interests.

[0443] The "update means" refers to a mechanism or method for adaptively changing the profile and dialogue content of the dialogue avatar based on the analysis results.

[0444] An "emotion engine" is an algorithm or technology that recognizes a user's emotional state and provides appropriate feedback.

[0445] A "virtual store" is a virtual store environment created on the Internet where users can browse and purchase products.

[0446] "Personalized services" refer to services that are tailored to the user's individual interests and emotional state.

[0447] This invention is a system that provides a conversational avatar using generative AI to reduce users' feelings of loneliness. In particular, by combining an emotion engine, it senses the user's emotional state and dynamically changes the content of the conversation. This invention aims to provide personalized services according to the user's emotions, especially in virtual stores.

[0448] System Configuration

[0449] The system is constructed by three components: a server, a terminal, and a user. The roles of each component are as follows:

[0450] User registration and basic information entry

[0451] Users must first log in to the system and enter basic information such as their name, age, gender, hobbies, and interests. This is done using input means on the user's device (smartphone, smart glasses, head-mounted display, etc.).

[0452] Initial avatar generation and conversation start

[0453] The server generates a conversational avatar based on the stored basic information. The generated avatar is optimized taking into account the user's basic information. The server sends an initial conversation script to the terminal, and the user can start a conversation with the avatar through a conversation screen on the terminal.

[0454] Dialogue progression and emotional engine utilization

[0455] When a user enters a dialogue message, the device sends it to the server. The server analyzes the message using a natural language processing engine (e.g., the Hugging Face Transformer model) and then uses an emotion engine to recognize the user's emotional state. For example, a message like "I'm very tired today" is recognized as "fatigue." Based on this emotional state, the server generates a reply using OpenAI's GPT-3 model and provides an appropriate response to the user.

[0456] Saving and analyzing conversation history

[0457] The entire dialogue history is saved. The saved history is analyzed and used to improve the quality of future dialogues. As an analytical tool, the server analyzes the dialogue history and uses a natural language processing engine and emotion engine to identify changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively modified.

[0458] Application in virtual stores

[0459] When interacting with users in the virtual store, the avatar recognizes their emotional state and provides personalized services. For example, if a user says, "I've been so busy and stressed this week," the emotion engine will recognize this as "stress" and provide a response such as, "I recommend this aroma diffuser. It contains a highly relaxing scent."

[0460] In this way, appropriate dialogue and personalized services tailored to the user's emotional state can be provided even in a virtual environment.

[0461] Prompt Sentence Examples

[0462] Below is an example of a prompt sentence when a user says, "I've been very busy at work lately and it's stressful."

[0463] User is feeling stressed. User says: Work has been so busy lately that it's been stressful. Response:

[0464] This prompt sentence is fed into OpenAI's GPT-3 to generate an avatar response.

[0465] As described above, by combining an emotion engine and a generative AI, the present invention can personalize interactions with users in a virtual store and improve the user experience.

[0466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0467] Step 1:

[0468] The user inputs basic information using the terminal. The terminal uses an input means to collect basic information such as the user's name, age, gender, hobbies, and interests, and sends this information to the server. The input is done through an input form such as a text field.

[0469] Input: User's basic information (name, age, gender, hobbies, interests)

[0470] Output: A data packet containing basic information

[0471] Step 2:

[0472] The server stores the received basic information in a database using a storage means, and the stored information is used for future interactions and avatar generation.

[0473] Input: A data packet containing basic information

[0474] Output: Basic information stored in the database

[0475] Step 3:

[0476] The server generates an interactive avatar based on the stored basic information using a generating means, and the generated avatar reflects the user's basic information and is ready for personalized interaction.

[0477] Input: Basic information stored in the database

[0478] Output: Profile of the conversational avatar

[0479] Step 4:

[0480] The server creates an initial dialogue script for the generated dialogue avatar and sends it to the terminal, which uses the received script to display a dialogue screen and prompts the user to start a dialogue.

[0481] Input: Interactive avatar profile

[0482] Output: Initial interaction script

[0483] Step 5:

[0484] Users can start exchanging messages with the avatar through an interactive screen on their device. When users enter questions or comments, the messages are sent to the server via their device.

[0485] Input: User input message

[0486] Output: Message data from the terminal to the server

[0487] Step 6:

[0488] The server uses a natural language processing engine to analyze the user's message and an emotion engine to recognize the user's emotional state. For example, the server recognizes the message "I'm very tired today" as "fatigue."

[0489] Input: User input message

[0490] Output: Emotional state label (e.g., fatigue)

[0491] Step 7:

[0492] The server generates an appropriate response using OpenAI's GPT-3 based on the emotional state label. It constructs a prompt sentence and feeds it into the generative AI to create a reply (e.g., "User is feeling tired. User says: I'm very tired today.").

[0493] Input: Emotional state label, user input message

[0494] Output: The generated response message

[0495] Step 8:

[0496] The server generates a response message and sends it to the terminal, which displays it to the user and continues the dialogue.

[0497] Input: The generated response message

[0498] Output: Response message displayed on the terminal

[0499] Step 9:

[0500] The server stores the dialogue history in a database using a storage means and analyzes the history using an analysis means, thereby clarifying changes in the user's interests and emotions and obtaining information to improve the content of future dialogues.

[0501] Input: Dialogue history

[0502] Output: Analysis results stored in a database

[0503] Step 10:

[0504] The server uses an updater to change the profile and dialogue script of the dialogue avatar based on the analysis results, thereby improving the quality of responses to the user and providing personalized services.

[0505] Input: Analysis results

[0506] Output: Updated interactive avatar profile and script

[0507] Through the above steps, the invention can provide personalized services by interacting with the user in the virtual store according to their emotional state.

[0508] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0509] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0510] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0511] [Second embodiment]

[0512] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0513] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0514] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0515] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0516] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0517] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0518] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0519] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0520] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0521] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0522] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0523] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0524] MODE FOR CARRYING OUT THE INVENTION

[0525] System Configuration and Functions

[0526] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user. The operation of the system is explained in detail below.

[0527] 1. User Registration Process

[0528] A user first logs in to the system and enters basic information, such as name, age, gender, hobbies, interests, etc. To enter this information, the user uses a web form as an input method.

[0529] The terminal receives the information entered by the user and sends it to the server, which then analyzes the received data in a format such as JSON and stores it in a database.

[0530] 2. Initial avatar generation and conversation start

[0531] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[0532] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0533] 3. Facilitating and managing the dialogue

[0534] The terminal sends the message entered by the user to the server, which then analyzes the received message using a natural language processing engine and generates an appropriate reply. The generated reply is then sent back to the terminal and displayed to the user.

[0535] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[0536] 4. Avatar Growth and Evolution

[0537] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[0538] The avatar grows over time, allowing for more personalized interactions, and this growth mechanism allows the user and avatar to build a closer relationship.

[0539] Specific examples

[0540] Initial registration and interaction

[0541] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0542] 2. The device receives this information and sends it to the server, which stores it in a database.

[0543] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0544] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0545] 5. The user replies, "I like cooking."

[0546] Ongoing dialogue

[0547] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[0548] 2. The terminal displays this message to the user, who then replies.

[0549] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0550] In this way, the system can continually improve its interactions with the user, building a deeper understanding and connection.

[0551] The processing flow will be explained below.

[0552] Step 1:

[0553] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[0554] Step 2:

[0555] The terminal receives the input user information and transmits it to the server.

[0556] Step 3:

[0557] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[0558] Step 4:

[0559] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[0560] Step 5:

[0561] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[0562] Step 6:

[0563] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[0564] Step 7:

[0565] The user enters an initial message in the text box and clicks the "Send" button.

[0566] Step 8:

[0567] The terminal receives the user's input message and sends it to the server.

[0568] Step 9:

[0569] The server uses a natural language processing engine to analyze the received message and generate an appropriate reply, such as "I see you enjoy cooking. What did you make recently?"

[0570] Step 10:

[0571] The server generates a reply and sends it to the terminal.

[0572] Step 11:

[0573] The terminal displays the received reply to the user.

[0574] Step 12:

[0575] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[0576] Step 13:

[0577] The server uses analytical means to analyze the stored dialogue history, for example, to determine that the user frequently talks about Italian food.

[0578] Step 14:

[0579] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[0580] Step 15:

[0581] Users interact with their avatars periodically, and the avatars evolve based on the user's interests. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens.

[0582] Example 1

[0583] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0584] In modern society, the number of people feeling lonely is increasing. Conventional systems have difficulty providing truly personalized interactions with users and lack the means to build lasting relationships. In this situation, there is a need for a system that can reduce users' feelings of loneliness and provide a comfortable interaction experience.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0586] In this invention, the server includes: input means for a user to input basic information; storage means for storing the input basic information; generation means for generating a conversational avatar based on the stored basic information; dialogue means for the user to dialogue with the generated conversational avatar; means for sending the user's message to the server and analyzing it with a natural language processing engine to generate an appropriate response; analysis means for storing and analyzing the content of the dialogue; update means for updating the conversational avatar based on the analysis result of the analysis means; and means for storing a history of ongoing dialogue in a database and analyzing it to improve the quality of future dialogues. This allows the user to have a personalized conversation experience and reduces feelings of loneliness.

[0587] "Input means" refers to the means by which system users input basic information (such as name, age, gender, hobbies, interests, etc.), such as using a web form.

[0588] "Storage means" refers to a means for storing basic information entered by users and interaction history in a database, such as a database system such as MySQL or MongoDB.

[0589] "Generation means" means a means for generating a conversational avatar based on the stored basic information, and creates the characteristics of the conversational avatar using a generative AI model (e.g., GPT-3).

[0590] The "interaction means" is a means for a user to interact with the generated interaction avatar, and includes a text box and an interaction interface displayed on the terminal.

[0591] A "natural language processing engine" is a system that analyzes a user's message and generates an appropriate response based on that analysis, using technologies such as spaCy and BERT.

[0592] The "analysis means" refers to a means for saving and analyzing the content of a dialogue, and a system for analyzing the saved dialogue history.

[0593] The "update means" is a means for updating the interactive avatar based on the analysis results of the analysis means, and for example, modifies the avatar profile and the interactive script using a reinforcement learning model.

[0594] "Growth means" refers to means for updating the profile of the interactive avatar and enabling more personalized interactions in order to build a continuous relationship with the user.

[0595] A "database" is a system for managing stored basic information and interaction history, and for searching and extracting information as needed, including, for example, SQL databases and NoSQL databases.

[0596] A "generative AI model" is a machine learning model used to generate a conversational avatar and create a conversation script based on basic user information, including, for example, GPT-3.

[0597] System Configuration and Functions

[0598] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[0599] User Registration Process

[0600] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, and interests using a web form. The device then formats this information into JSON format and sends it to the server via an HTTP POST request. The server then parses the received data and stores it in a database system such as MySQL or MongoDB.

[0601] Initial avatar generation and conversation start

[0602] The server uses Python and TensorFlow to generate a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored user's basic information. Based on the generated avatar, the server generates an initial conversation script and sends it to the device. The device receives this script and displays it to the user through a conversational interface using HTML and JavaScript. For example, it displays a message such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0603] Facilitating and managing dialogue

[0604] The user enters a message in the displayed text box and presses the send button. For example, the user enters "I like cooking." The device sends this input message to the server. The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The generated reply is sent back to the device and displayed to the user. For example, a message such as "What dish have you cooked recently?" is displayed.

[0605] Avatar Growth and Evolution

[0606] The server stores all dialogue history in a database and analyzes it using analytical tools. This analysis identifies changes in the user's interests and new concerns. Utilizing a natural language processing engine, the server classifies the user's interests using a clustering algorithm (e.g., the K-means algorithm). Based on the results, the avatar's profile and dialogue script are updated. For example, if the user frequently talks about Italian food, the next dialogue might include a question such as, "Have you found any new Italian recipes?"

[0607] Specific examples

[0608] 1. Example of user registration and dialogue initiation

[0609] Users log in to the system and enter basic information such as their name, age, and hobbies.

[0610] The device receives this information and sends it to the server, which stores it in a database.

[0611] The server generates an avatar based on the user information and creates an initial dialogue script.

[0612] The terminal receives the generated dialogue script and displays it to the user. The prompt text displayed is "Hello, I'm your new friend. What are your recent hobbies?"

[0613] The user replies, "I like cooking."

[0614] 2. An example of ongoing dialogue

[0615] The server analyzes the user's dialogue message and generates an appropriate response, for example, "What dish did you cook recently?"

[0616] The terminal displays this message to the user, who then replies.

[0617] The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar's profile will be updated, and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0618] This allows the system to continually improve its interactions with the user, building a deeper understanding and relationship.

[0619] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0620] Step 1:

[0621] Users log in to the system and enter basic information such as name, age, gender, hobbies, interests, etc. using a web form built with HTML and JavaScript.

[0622] Input: Basic information (name, age, gender, hobbies, interests)

[0623] Output: Basic information data in JSON format

[0624] Step 2:

[0625] The terminal receives the basic information entered by the user, converts it into JSON format, and sends it to the server via an HTTP POST request.

[0626] Input: Basic information entered by the user into the web form

[0627] Output: JSON format data sent to the server

[0628] Step 3:

[0629] The server parses the received basic information data in JSON format and stores it in a database (e.g., MySQL or MongoDB). This storage process is performed transactionally to ensure data integrity.

[0630] Input: Basic information data in JSON format sent from the terminal

[0631] Output: Basic information stored in the database

[0632] Step 4:

[0633] The server generates a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored basic information. The generative AI model is operated using Python and TensorFlow.

[0634] Input: Basic information stored in the database

[0635] Output: Generated conversational avatar

[0636] Step 5:

[0637] The server generates an initial dialogue script based on the generated dialogue avatar, using NLG (Natural Language Generation) technology.

[0638] Input: Generated conversation avatar

[0639] Output: Initial interaction script

[0640] Step 6:

[0641] The terminal receives the initial dialogue script sent from the server and displays it to the user through a dialogue interface using HTML and JavaScript. For example, it displays "Hello, I'm your new friend. What are your recent hobbies?"

[0642] Input: Initial interaction script sent by the server

[0643] Output: The interactive interface and initial message displayed to the user

[0644] Step 7:

[0645] The user enters a message in the text box of the dialogue interface and presses the send button. For example, the user enters "I like cooking."

[0646] Input: The message the user types in the text box

[0647] Output: User message to be sent

[0648] Step 8:

[0649] The terminal transmits the message entered by the user to the server.

[0650] Input: The message entered by the user

[0651] Output: User message sent to the server

[0652] Step 9:

[0653] The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The analysis mainly involves semantic analysis and contextual understanding of the text. For example, the message generated is "What dishes have you cooked recently?"

[0654] Input: User message sent from the terminal

[0655] Output: The generated reply message

[0656] Step 10:

[0657] The server sends the generated reply message to the terminal.

[0658] Input: The generated reply message

[0659] Output: Reply message sent to the terminal

[0660] Step 11:

[0661] The terminal receives the reply message sent from the server and displays it to the user through the dialogue interface.

[0662] Input: Reply message sent from the server

[0663] Output: The reply message that is displayed to the user

[0664] Step 12:

[0665] The server stores all interaction history in a database. The stored interaction history is used to identify user interests and new concerns through analytical means, such as clustering algorithms and natural language processing techniques.

[0666] Input: Dialogue history

[0667] Output: Dialogue history stored in a database

[0668] Step 13:

[0669] The server updates the avatar's profile and dialogue script based on the analysis results. This uses a reinforcement learning model to allow the avatar to grow incrementally, making the dialogue with the user more personalized. For example, if the user frequently talks about Italian food, the next dialogue topic might be, "Have you found any new Italian recipes?"

[0670] Input: Analysis results

[0671] Output: Updated avatar profile and dialogue scripts

[0672] Through each of the above steps, the system continuously improves the interaction with the user, helping to reduce the user's sense of loneliness.

[0673] (Application example 1)

[0674] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] Conventional conversational avatar systems focus on conversational functions to reduce feelings of loneliness, but do not sufficiently consider the purchasing experience and product recommendations that users gain through conversation. Furthermore, the lack of a dynamic product recommendation function based on the user's interests limits the improvement of the purchasing experience. This has led to the challenge of making it difficult to accurately tap into the user's latent purchasing motivation.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0677] In this invention, the server includes input means for a user to input basic information, storage means for storing the input basic information, generation means for generating an interactive avatar based on the stored basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for storing and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, recommendation means for improving the purchasing experience using the interactive avatar, generation means for recommending products based on the user's input, and display means for displaying the products recommended by the recommendation means. This enables product recommendations based on the user's dialogue history and interests, thereby improving the purchasing experience while reducing feelings of loneliness through dialogue.

[0678] A "user" is an individual who uses the system to interact with interactive avatars and receive product recommendations.

[0679] "Basic information" refers to personal data such as the user's name, age, gender, hobbies, and interests.

[0680] "Input means" refers to the interface or device that allows users to input basic information into the system.

[0681] "Storage means" refers to a technical element that stores the input basic information and dialogue content in a database.

[0682] The "generation means" is a function or system for automatically creating a conversational avatar based on the stored basic information.

[0683] "Interaction means" refers to an interface or system that allows the user to exchange messages with the generated interactive avatar.

[0684] "Analysis means" refers to the technology or algorithm used to analyze the history and content of interactions and update or improve the interaction avatar.

[0685] The "update means" is a function for correcting and updating the profile and dialogue content of the dialogue avatar based on the results of the analysis means.

[0686] "Recommendation means" is a function that uses a conversational avatar to suggest the most suitable products to the user in order to improve the purchasing experience.

[0687] The "display means" refers to a display or interface for visually presenting the generated dialogue script and recommended products to the user.

[0688] "Purchasing experience" refers to the experience a user has during the process of selecting a product, purchasing it, or obtaining information about it.

[0689] MODE FOR CARRYING OUT THE INVENTION

[0690] System Configuration and Functions

[0691] This invention is a system that provides interactive avatars using generative AI to improve the shopping experience while reducing feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[0692] User Registration Process

[0693] A user first logs in to the system and enters basic information, including name, age, gender, hobbies, and interests. The user uses a web form as an input method. The device receives the information entered by the user and sends it to the server. The server parses the received data into a format such as JSON and stores it in a database.

[0694] Initial avatar generation and conversation start

[0695] The server generates a conversational avatar based on the stored basic information. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the terminal. The terminal receives this script and displays a conversation screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0696] Facilitating and managing dialogue

[0697] The terminal sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., OpenAI API) and generates an appropriate reply. The generated reply is sent back to the terminal and displayed to the user. The entire history of this dialogue is accumulated in a database by the storage means. This stored history is analyzed by the analysis means to improve the quality of future dialogues.

[0698] Product recommendation feature implementation

[0699] The product recommendation function is provided to improve the purchasing experience through interaction between the interactive avatar and the user. The server uses a generating means to recommend optimal products to the user based on the user's basic information and interaction history. The recommended products are visually presented to the user through the display means of the terminal.

[0700] Avatar Growth and Evolution

[0701] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated using an update means. The avatar grows over time, enabling more personalized dialogue.

[0702] Example

[0703] Initial registration and interaction

[0704] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0705] 2. The device receives this information and sends it to the server, which stores it in a database.

[0706] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0707] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0708] 5. The user replies, "I like cooking."

[0709] Ongoing dialogue and product recommendations

[0710] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[0711] 2. The terminal displays this message to the user, who then replies.

[0712] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[0713] 4. Furthermore, to improve the shopping experience, the avatar will recommend products such as, "Today's special Italian cooking ingredients are here."

[0714] Prompt Sentence Examples

[0715] "User name: Taro Yamamoto

[0716] Age: 30

[0717] Gender: Male

[0718] Hobbies: Fashion, gadgets

[0719] Interests: Latest trends, innovative technologies

[0720] Please generate a suitable initial interaction script for this user."

[0721] In this way, the present invention is optimized to reduce the user's sense of loneliness and improve the purchasing experience through interaction.

[0722] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0723] Step 1:

[0724] A user logs into the system and fills in basic information such as name, age, gender, hobbies, and interests in a web form.

[0725] Input: User's basic information (name, age, gender, hobbies, interests, etc.)

[0726] Output: Basic information sent to the terminal in JSON format

[0727] Step 2:

[0728] The device sends the basic information entered by the user to the server, which then parses the information in JSON format and stores it in a database.

[0729] Input: User basic information (JSON format)

[0730] Output: Basic information stored in the database

[0731] Step 3:

[0732] The server generates an interactive avatar based on the stored basic information.

[0733] Input: Basic information (from database)

[0734] Output: Generated conversational avatar

[0735] Step 4:

[0736] The server creates an initial interaction script for the avatar and sends it to the terminal, which receives the script and displays it to the user.

[0737] Input: Basic information of the generated conversational avatar

[0738] Output: The initial interaction script is displayed to the user

[0739] Step 5:

[0740] The user inputs and sends a message to the avatar.

[0741] Input: User interaction message

[0742] Output: Message sent to the terminal

[0743] Step 6:

[0744] The terminal sends the user's message to the server, which uses a natural language processing engine to analyze the message and generate an appropriate reply.

[0745] Input: User interaction message

[0746] Output: The generated avatar returned by the server

[0747] Step 7:

[0748] The server generates a reply and sends it to the terminal, which displays the reply to the user.

[0749] Input: Reply message from server

[0750] Output: Avatar reply shown to the user

[0751] Step 8:

[0752] The server stores the content of the dialogue in a database and analyzes the dialogue history.

[0753] Input: Interaction history between user and avatar

[0754] Output: Parsed data

[0755] Step 9:

[0756] The server updates the interactive avatar based on the analysis results and generates new scripts and profiles.

[0757] Input: Analyzed interaction history data

[0758] Output: Updated interactive avatar scripts and profiles

[0759] Step 10:

[0760] The server generates a script for recommending products based on the user's basic information and interaction history, and transmits the recommended products to the terminal, which then displays the recommended products to the user.

[0761] Input: User basic information, interaction history

[0762] Output: Recommended product information

[0763] Through this series of steps, users can reduce their sense of loneliness through interaction with a conversational avatar, further improving their purchasing experience.

[0764] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0765] MODE FOR CARRYING OUT THE INVENTION

[0766] System Configuration and Functions

[0767] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user. The operation of the system is explained in detail below.

[0768] 1. User registration and basic information entry

[0769] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, interests, etc. To do so, the user uses a web form as an input method.

[0770] The terminal receives the information entered by the user and sends it to the server, which analyzes the received data and stores it in a database. The storage means used here is the one used.

[0771] 2. Initial avatar generation and conversation start

[0772] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[0773] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[0774] 3. Dialogue progression and emotional engine utilization

[0775] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue."

[0776] The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as, "You seem tired. How do you relax?" The generated reply is sent back to the device and displayed to the user.

[0777] 4. Saving and analyzing dialogue history

[0778] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[0779] The server analyzes the stored dialogue history using an analysis means. By using a natural language processing engine and an emotion engine in combination, it identifies changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[0780] 5. Avatar Growth and Relationship Development

[0781] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "fatigue" or "stress," the avatar will add more conversation scripts about relaxation methods and stress management.

[0782] Users regularly interact with their avatars, which evolve based on the user's interests and emotions. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens. This allows users to receive emotional support and reduce feelings of loneliness.

[0783] Specific examples

[0784] Initial registration and interaction

[0785] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0786] 2. The device receives this information and sends it to the server, which stores it in a database.

[0787] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[0788] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0789] 5. The user replies, "I like cooking."

[0790] Continuous dialogue and emotion recognition

[0791] 1. The server analyzes the user's dialogue message and generates the following appropriate response: For example, it generates a reply such as, "I see you enjoy cooking. What dish have you cooked recently?"

[0792] 2. The terminal displays this message to the user, who then replies.

[0793] 3. The server stores the dialogue history and continues to analyze it. If the user types, "I'm very tired today," the emotion engine recognizes "fatigue" and generates a response accordingly. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[0794] 4. The server updates the avatar's profile and dialogue scripts, taking into account the user's dialogue history and emotional state.

[0795] In this way, the system can continuously improve its dialogue with the user and, by leveraging the emotion engine, provide appropriate support depending on the user's emotional state.

[0796] The processing flow will be explained below.

[0797] Step 1:

[0798] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[0799] Step 2:

[0800] The terminal receives the input user information and transmits it to the server.

[0801] Step 3:

[0802] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[0803] Step 4:

[0804] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[0805] Step 5:

[0806] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[0807] Step 6:

[0808] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[0809] Step 7:

[0810] The user enters an initial message in the text box and clicks the "Send" button.

[0811] Step 8:

[0812] The terminal receives the user's input message and sends it to the server.

[0813] Step 9:

[0814] The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, if the message says "I'm very tired today," the emotion is recognized as "fatigue."

[0815] Step 10:

[0816] The server generates an appropriate response based on the analysis results of the emotion engine, for example, "You seem tired. How do you relax?"

[0817] Step 11:

[0818] The server generates a reply and sends it to the terminal.

[0819] Step 12:

[0820] The terminal displays the received reply to the user.

[0821] Step 13:

[0822] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[0823] Step 14:

[0824] The server uses analytical means to analyze the stored dialogue history and emotional state, and uses a natural language processing engine and an emotion engine in combination to identify changes in the user's interests and emotions.

[0825] Step 15:

[0826] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[0827] Step 16:

[0828] Users interact with the avatar periodically, and the avatar evolves based on the user's interests and emotions. The server continuously analyzes and updates the interaction data, deepening the user's relationship with the avatar.

[0829] Example 2

[0830] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0831] In modern society, many people feel lonely, and effective countermeasures are needed. Furthermore, there is a need for a system that can accurately grasp a user's emotional state and alleviate loneliness through dialogue. Conventional systems can only provide one-way dialogue based on basic user information, making it difficult to dynamically respond to changes in the user's emotional state or interests.

[0832] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for the user to input basic information, a data storage means for saving the input basic information, a generation means for generating an interactive avatar based on the saved basic information, an interaction means for the user to interact with the generated interactive avatar, a natural language processing means for analyzing the user's message, an emotion analysis means for recognizing the user's emotion, and an update means for updating the interactive avatar based on the analysis result. This enables dynamic interaction according to the user's emotional state and interests, thereby reducing feelings of loneliness.

[0833] "User" refers to the end user of the system who provides basic information and interactive input.

[0834] "Input means" refers to the interface through which users input basic information, such as a web form or an application screen.

[0835] "Data storage means" refers to a mechanism for retaining the basic information entered, including databases and cloud storage.

[0836] "Generation means" refers to the part of the system that has the function of generating an interactive avatar based on stored basic information, and may use a generative AI model.

[0837] "Interactive means" refers to a mechanism for communication between the generated interactive avatar and the user, such as an interactive screen or chat interface.

[0838] "Natural language processing means" refers to technology for analyzing and understanding a user's message, including a natural language processing engine.

[0839] "Emotion analysis means" refers to technology for recognizing the emotional state of a user from their message, and this corresponds to the emotion engine.

[0840] The "update means" refers to a system part that has the function of dynamically changing the profile and dialogue script of the dialogue avatar based on the analysis results.

[0841] "Dialogue history" refers to a record of communication between a user and a dialogue avatar, including saved messages and responses.

[0842] "Analysis means" refers to technology for analyzing stored interaction history and identifying changes in a user's interests and emotions.

[0843] "Growth means" refers to a mechanism for evolving the profile and script of an interactive avatar through continuous dialogue with the user.

[0844] MODE FOR CARRYING OUT THE INVENTION

[0845] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user.

[0846] 1. User registration and basic information entry

[0847] Users log in to the system and enter basic information such as name, age, gender, hobbies, and interests using a web form. The device receives the entered information and sends it to the server. The server analyzes the received data and stores it in a database. The database used here is, for example, MySQL.

[0848] 2. Initial avatar generation and conversation start

[0849] The server generates a conversational avatar based on the stored basic information. A generative AI model (e.g., GPT-3) is used for this generation. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the device. The device receives this script and displays a conversation screen to the user. The user begins a conversation with the avatar by entering a message in the text box and sending it.

[0850] 3. Dialogue progression and emotional engine utilization

[0851] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., NLTK). It then uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue." The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as "You seem tired. How do you relax?" The generated reply is sent from the server to the device, which then displays it to the user.

[0852] 4. Saving and analyzing dialogue history

[0853] The server stores the dialogue history in a database. The stored dialogue history is used as an analytical tool to improve the quality of future dialogues. The server uses a natural language processing engine and an emotion engine to analyze the stored dialogue history and identify changes in the user's interests and emotions. Based on the analysis results, the dialogue avatar's profile and dialogue script are dynamically updated.

[0854] 5. Avatar Growth and Relationship Development

[0855] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "tired" or "stressed," the avatar will add more conversation scripts about relaxation methods and stress management. As the user regularly interacts with the avatar and the avatar evolves based on the user's interests and emotions, the user can receive emotional support and reduce feelings of loneliness.

[0856] Specific examples

[0857] Initial registration and interaction

[0858] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[0859] 2. The device sends this information to the server, which stores it in a database.

[0860] 3. The server generates an avatar based on the saved user information and creates an initial interaction script.

[0861] 4. The terminal receives the dialogue script sent from the server and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[0862] 5. The user replies, "I love cooking."

[0863] Continuous dialogue and emotion recognition

[0864] 1. The server analyzes the user's dialogue message and generates an appropriate response, such as "I see you enjoy cooking. What did you cook recently?"

[0865] 2. The terminal displays this message to the user, who then replies.

[0866] 3. The server stores and continuously analyzes the conversation history. For example, if a user types, "I'm very tired today," the emotion engine recognizes this and generates a response tailored to the user's needs. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[0867] 4. The server updates the avatar's profile and dialogue script based on the analysis results.

[0868] Prompt Sentence Examples

[0869] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[0870] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0871] The flow of this system's program processing

[0872] Step 1: Enter basic user information

[0873] Input: A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests into a web form.

[0874] Processing: The device receives the entered information and sends it to the server using an HTTP POST request.

[0875] Output: The server parses the received data and stores it in a database.

[0876] Specific operation: The server executes an SQL INSERT statement using a storage means, for example, in a MySQL database, to save the data.

[0877] Step 2: Generate the initial avatar

[0878] Input: The server retrieves basic information stored in a database.

[0879] Processing: The server generates an interactive avatar based on the stored basic information. It uses a generative AI model (e.g., GPT-3) to create an avatar profile.

[0880] Output: Generates an initial dialogue script for the generated avatar.

[0881] Specific operation: The user's basic information is input into the generative AI model as a prompt sentence, and an appropriate dialogue script is output.

[0882] Step 3: Send the initial interaction script

[0883] Input: The generated avatar and initial dialogue script.

[0884] Processing: The server sends the generated dialogue script to the terminal.

[0885] Output: The terminal receives the script and displays an interactive screen to the user.

[0886] Specific operation: The terminal uses HTML and JavaScript to render the interactive screen and display the script.

[0887] Step 4: Start interacting with the user

[0888] Input: The user types a message in the text box and sends it.

[0889] Processing: The terminal receives the user's input message and sends it to the server.

[0890] Output: The server passes the received message to a natural language processing engine (e.g., NLTK) for analysis.

[0891] Specific operation: The server passes the message to a natural language processing engine, performs tokenization and morphological analysis, and obtains the analysis results.

[0892] Step 5: Recognizing your emotional state

[0893] Input: Parsed message data.

[0894] Processing: The server uses an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotional state.

[0895] Output: Emotional state recognition results.

[0896] Specific operation: The server passes the analysis results to the emotion engine and outputs the emotional state (e.g., sadness, joy, anger, etc.).

[0897] Step 6: Generate an appropriate reply

[0898] Input: Emotional state recognition results.

[0899] Processing: The server again uses the generative AI model to generate an appropriate reply.

[0900] Output: The generated reply message.

[0901] Specific operation: The user's emotional state is input into the generative AI model as a prompt sentence, and an appropriate reply sentence is output.

[0902] Step 7: Send and view the reply message

[0903] Input: The generated reply message.

[0904] Processing: The server generates a reply and sends it to the terminal.

[0905] Output: The terminal displays the reply message to the user.

[0906] Specific operation: The terminal adds the new message to the interactive screen so that the user can view it.

[0907] Step 8: Saving conversation history

[0908] Input: User and avatar interaction history.

[0909] Processing: The server stores the dialogue history in a database.

[0910] Output: The saved interaction history.

[0911] Specific operation: The server saves the conversation history to the database using the SQL INSERT statement.

[0912] Step 9: Analyzing the dialogue history and updating the avatar

[0913] Input: Saved interaction history.

[0914] Processing: The server uses a natural language processing engine and an emotion engine to analyze the dialogue history and identify changes in the user's interests and emotions.

[0915] Output: Profile and dialogue script updates based on the analysis results.

[0916] Specific operation: Based on the analysis results, the avatar's profile and dialogue script are dynamically changed and reflected in the next dialogue with the user.

[0917] Prompt Sentence Examples

[0918] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[0919] (Application example 2)

[0920] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] In modern society, user interaction, especially in virtual environments, is an important factor in improving engagement and customer satisfaction. However, conventional dialogue systems have difficulty properly recognizing and responding to users' emotional states. Furthermore, they lack the ability to dynamically adjust dialogue content, which hinders the provision of personalized services. This can lead to users feeling isolated and reduces the quality of their experience in virtual environments. To solve this problem, a system with emotion recognition capabilities and the ability to dynamically adjust dialogue content is needed.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes input means for the user to input basic information, storage means for saving the input basic information, generation means for generating an interactive avatar based on the saved basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for saving and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, and means for dialogue with the user in a virtual store, recognizing the user's emotional state using an emotion engine, and dynamically changing the content of the response. This makes it possible to provide appropriate dialogue and personalized services tailored to the user's emotional state in the virtual environment.

[0923] "User" means an individual human being or end user who uses the system.

[0924] "Basic information" refers to information necessary for generating a conversational avatar, such as the user's name, age, gender, hobbies, and interests.

[0925] "Input means" refers to the interface or device that allows users to input basic information into the system.

[0926] "Storage means" refers to a mechanism or device for storing the input basic information and dialogue history in a database or the like.

[0927] "Generator" refers to the process or function that creates an interactive avatar based on the stored basic information.

[0928] "Interaction means" refers to the functions and mechanisms that enable two-way communication between the generated interactive avatar and the user.

[0929] "Analysis means" refers to methods and technologies for analyzing the content of a conversation and understanding the user's emotional state and interests.

[0930] The "update means" refers to a mechanism or method for adaptively changing the profile and dialogue content of the dialogue avatar based on the analysis results.

[0931] An "emotion engine" is an algorithm or technology that recognizes a user's emotional state and provides appropriate feedback.

[0932] A "virtual store" is a virtual store environment created on the Internet where users can browse and purchase products.

[0933] "Personalized services" refer to services that are tailored to the user's individual interests and emotional state.

[0934] This invention is a system that provides a conversational avatar using generative AI to reduce users' feelings of loneliness. In particular, by combining an emotion engine, it senses the user's emotional state and dynamically changes the content of the conversation. This invention aims to provide personalized services according to the user's emotions, especially in virtual stores.

[0935] System Configuration

[0936] The system is constructed by three components: a server, a terminal, and a user. The roles of each component are as follows:

[0937] User registration and basic information entry

[0938] Users must first log in to the system and enter basic information such as their name, age, gender, hobbies, and interests. This is done using input means on the user's device (smartphone, smart glasses, head-mounted display, etc.).

[0939] Initial avatar generation and conversation start

[0940] The server generates a conversational avatar based on the stored basic information. The generated avatar is optimized taking into account the user's basic information. The server sends an initial conversation script to the terminal, and the user can start a conversation with the avatar through a conversation screen on the terminal.

[0941] Dialogue progression and emotional engine utilization

[0942] When a user enters a dialogue message, the device sends it to the server. The server analyzes the message using a natural language processing engine (e.g., the Hugging Face Transformer model) and then uses an emotion engine to recognize the user's emotional state. For example, a message like "I'm very tired today" is recognized as "fatigue." Based on this emotional state, the server generates a reply using OpenAI's GPT-3 model and provides an appropriate response to the user.

[0943] Saving and analyzing conversation history

[0944] The entire dialogue history is saved. The saved history is analyzed and used to improve the quality of future dialogues. As an analytical tool, the server analyzes the dialogue history and uses a natural language processing engine and emotion engine to identify changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively modified.

[0945] Application in virtual stores

[0946] When interacting with users in the virtual store, the avatar recognizes their emotional state and provides personalized services. For example, if a user says, "I've been so busy and stressed this week," the emotion engine will recognize this as "stress" and provide a response such as, "I recommend this aroma diffuser. It contains a highly relaxing scent."

[0947] In this way, appropriate dialogue and personalized services tailored to the user's emotional state can be provided even in a virtual environment.

[0948] Prompt Sentence Examples

[0949] Below is an example of a prompt sentence when a user says, "I've been very busy at work lately and it's stressful."

[0950] User is feeling stressed. User says: Work has been so busy lately that it's been stressful. Response:

[0951] This prompt sentence is fed into OpenAI's GPT-3 to generate an avatar response.

[0952] As described above, by combining an emotion engine and a generative AI, the present invention can personalize interactions with users in a virtual store and improve the user experience.

[0953] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0954] Step 1:

[0955] The user inputs basic information using the terminal. The terminal uses an input means to collect basic information such as the user's name, age, gender, hobbies, and interests, and sends this information to the server. The input is done through an input form such as a text field.

[0956] Input: User's basic information (name, age, gender, hobbies, interests)

[0957] Output: A data packet containing basic information

[0958] Step 2:

[0959] The server stores the received basic information in a database using a storage means, and the stored information is used for future interactions and avatar generation.

[0960] Input: A data packet containing basic information

[0961] Output: Basic information stored in the database

[0962] Step 3:

[0963] The server generates an interactive avatar based on the stored basic information using a generating means, and the generated avatar reflects the user's basic information and is ready for personalized interaction.

[0964] Input: Basic information stored in the database

[0965] Output: Profile of the conversational avatar

[0966] Step 4:

[0967] The server creates an initial dialogue script for the generated dialogue avatar and sends it to the terminal, which uses the received script to display a dialogue screen and prompts the user to start a dialogue.

[0968] Input: Interactive avatar profile

[0969] Output: Initial interaction script

[0970] Step 5:

[0971] Users can start exchanging messages with the avatar through an interactive screen on their device. When users enter questions or comments, the messages are sent to the server via their device.

[0972] Input: User input message

[0973] Output: Message data from the terminal to the server

[0974] Step 6:

[0975] The server uses a natural language processing engine to analyze the user's message and an emotion engine to recognize the user's emotional state. For example, the server recognizes the message "I'm very tired today" as "fatigue."

[0976] Input: User input message

[0977] Output: Emotional state label (e.g., fatigue)

[0978] Step 7:

[0979] The server generates an appropriate response using OpenAI's GPT-3 based on the emotional state label. It constructs a prompt sentence and feeds it into the generative AI to create a reply (e.g., "User is feeling tired. User says: I'm very tired today.").

[0980] Input: Emotional state label, user input message

[0981] Output: The generated response message

[0982] Step 8:

[0983] The server generates a response message and sends it to the terminal, which displays it to the user and continues the dialogue.

[0984] Input: The generated response message

[0985] Output: Response message displayed on the terminal

[0986] Step 9:

[0987] The server stores the dialogue history in a database using a storage means and analyzes the history using an analysis means, thereby clarifying changes in the user's interests and emotions and obtaining information to improve the content of future dialogues.

[0988] Input: Dialogue history

[0989] Output: Analysis results stored in a database

[0990] Step 10:

[0991] The server uses an updater to change the profile and dialogue script of the dialogue avatar based on the analysis results, thereby improving the quality of responses to the user and providing personalized services.

[0992] Input: Analysis results

[0993] Output: Updated interactive avatar profile and script

[0994] Through the above steps, the invention can provide personalized services by interacting with the user in the virtual store according to their emotional state.

[0995] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0996] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0997] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0998] [Third embodiment]

[0999] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1000] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1001] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1002] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1003] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1004] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1005] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1006] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1007] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1008] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1009] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1010] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1011] MODE FOR CARRYING OUT THE INVENTION

[1012] System Configuration and Functions

[1013] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user. The operation of the system is explained in detail below.

[1014] 1. User Registration Process

[1015] A user first logs in to the system and enters basic information, such as name, age, gender, hobbies, interests, etc. To enter this information, the user uses a web form as an input method.

[1016] The terminal receives the information entered by the user and sends it to the server, which then analyzes the received data in a format such as JSON and stores it in a database.

[1017] 2. Initial avatar generation and conversation start

[1018] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[1019] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1020] 3. Facilitating and managing the dialogue

[1021] The terminal sends the message entered by the user to the server, which then analyzes the received message using a natural language processing engine and generates an appropriate reply. The generated reply is then sent back to the terminal and displayed to the user.

[1022] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[1023] 4. Avatar Growth and Evolution

[1024] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[1025] The avatar grows over time, allowing for more personalized interactions, and this growth mechanism allows the user and avatar to build a closer relationship.

[1026] Specific examples

[1027] Initial registration and interaction

[1028] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1029] 2. The device receives this information and sends it to the server, which stores it in a database.

[1030] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1031] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1032] 5. The user replies, "I like cooking."

[1033] Ongoing dialogue

[1034] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[1035] 2. The terminal displays this message to the user, who then replies.

[1036] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1037] In this way, the system can continually improve its interactions with the user, building a deeper understanding and connection.

[1038] The processing flow will be explained below.

[1039] Step 1:

[1040] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[1041] Step 2:

[1042] The terminal receives the input user information and transmits it to the server.

[1043] Step 3:

[1044] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[1045] Step 4:

[1046] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[1047] Step 5:

[1048] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[1049] Step 6:

[1050] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[1051] Step 7:

[1052] The user enters an initial message in the text box and clicks the "Send" button.

[1053] Step 8:

[1054] The terminal receives the user's input message and sends it to the server.

[1055] Step 9:

[1056] The server uses a natural language processing engine to analyze the received message and generate an appropriate reply, such as "I see you enjoy cooking. What did you make recently?"

[1057] Step 10:

[1058] The server generates a reply and sends it to the terminal.

[1059] Step 11:

[1060] The terminal displays the received reply to the user.

[1061] Step 12:

[1062] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[1063] Step 13:

[1064] The server uses analytical means to analyze the stored dialogue history, for example, to determine that the user frequently talks about Italian food.

[1065] Step 14:

[1066] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[1067] Step 15:

[1068] Users interact with their avatars periodically, and the avatars evolve based on the user's interests. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens.

[1069] Example 1

[1070] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1071] In modern society, the number of people feeling lonely is increasing. Conventional systems have difficulty providing truly personalized interactions with users and lack the means to build lasting relationships. In this situation, there is a need for a system that can reduce users' feelings of loneliness and provide a comfortable interaction experience.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1073] In this invention, the server includes: input means for a user to input basic information; storage means for storing the input basic information; generation means for generating a conversational avatar based on the stored basic information; dialogue means for the user to dialogue with the generated conversational avatar; means for sending the user's message to the server and analyzing it with a natural language processing engine to generate an appropriate response; analysis means for storing and analyzing the content of the dialogue; update means for updating the conversational avatar based on the analysis result of the analysis means; and means for storing a history of ongoing dialogue in a database and analyzing it to improve the quality of future dialogues. This allows the user to have a personalized conversation experience and reduces feelings of loneliness.

[1074] "Input means" refers to the means by which system users input basic information (such as name, age, gender, hobbies, interests, etc.), such as using a web form.

[1075] "Storage means" refers to a means for storing basic information entered by users and interaction history in a database, such as a database system such as MySQL or MongoDB.

[1076] "Generation means" means a means for generating a conversational avatar based on the stored basic information, and creates the characteristics of the conversational avatar using a generative AI model (e.g., GPT-3).

[1077] The "interaction means" is a means for a user to interact with the generated interaction avatar, and includes a text box and an interaction interface displayed on the terminal.

[1078] A "natural language processing engine" is a system that analyzes a user's message and generates an appropriate response based on that analysis, using technologies such as spaCy and BERT.

[1079] The "analysis means" refers to a means for saving and analyzing the content of a dialogue, and a system for analyzing the saved dialogue history.

[1080] The "update means" is a means for updating the interactive avatar based on the analysis results of the analysis means, and for example, modifies the avatar profile and the interactive script using a reinforcement learning model.

[1081] "Growth means" refers to means for updating the profile of the interactive avatar and enabling more personalized interactions in order to build a continuous relationship with the user.

[1082] A "database" is a system for managing stored basic information and interaction history, and for searching and extracting information as needed, including, for example, SQL databases and NoSQL databases.

[1083] A "generative AI model" is a machine learning model used to generate a conversational avatar and create a conversation script based on basic user information, including, for example, GPT-3.

[1084] System Configuration and Functions

[1085] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[1086] User Registration Process

[1087] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, and interests using a web form. The device then formats this information into JSON format and sends it to the server via an HTTP POST request. The server then parses the received data and stores it in a database system such as MySQL or MongoDB.

[1088] Initial avatar generation and conversation start

[1089] The server uses Python and TensorFlow to generate a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored user's basic information. Based on the generated avatar, the server generates an initial conversation script and sends it to the device. The device receives this script and displays it to the user through a conversational interface using HTML and JavaScript. For example, it displays a message such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1090] Facilitating and managing dialogue

[1091] The user enters a message in the displayed text box and presses the send button. For example, the user enters "I like cooking." The device sends this input message to the server. The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The generated reply is sent back to the device and displayed to the user. For example, a message such as "What dish have you cooked recently?" is displayed.

[1092] Avatar Growth and Evolution

[1093] The server stores all dialogue history in a database and analyzes it using analytical tools. This analysis identifies changes in the user's interests and new concerns. Utilizing a natural language processing engine, the server classifies the user's interests using a clustering algorithm (e.g., the K-means algorithm). Based on the results, the avatar's profile and dialogue script are updated. For example, if the user frequently talks about Italian food, the next dialogue might include a question such as, "Have you found any new Italian recipes?"

[1094] Specific examples

[1095] 1. Example of user registration and dialogue initiation

[1096] Users log in to the system and enter basic information such as their name, age, and hobbies.

[1097] The device receives this information and sends it to the server, which stores it in a database.

[1098] The server generates an avatar based on the user information and creates an initial dialogue script.

[1099] The terminal receives the generated dialogue script and displays it to the user. The prompt text displayed is "Hello, I'm your new friend. What are your recent hobbies?"

[1100] The user replies, "I like cooking."

[1101] 2. An example of ongoing dialogue

[1102] The server analyzes the user's dialogue message and generates an appropriate response, for example, "What dish did you cook recently?"

[1103] The terminal displays this message to the user, who then replies.

[1104] The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar's profile will be updated, and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1105] This allows the system to continually improve its interactions with the user, building a deeper understanding and relationship.

[1106] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1107] Step 1:

[1108] Users log in to the system and enter basic information such as name, age, gender, hobbies, interests, etc. using a web form built with HTML and JavaScript.

[1109] Input: Basic information (name, age, gender, hobbies, interests)

[1110] Output: Basic information data in JSON format

[1111] Step 2:

[1112] The terminal receives the basic information entered by the user, converts it into JSON format, and sends it to the server via an HTTP POST request.

[1113] Input: Basic information entered by the user into the web form

[1114] Output: JSON format data sent to the server

[1115] Step 3:

[1116] The server parses the received basic information data in JSON format and stores it in a database (e.g., MySQL or MongoDB). This storage process is performed transactionally to ensure data integrity.

[1117] Input: Basic information data in JSON format sent from the terminal

[1118] Output: Basic information stored in the database

[1119] Step 4:

[1120] The server generates a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored basic information. The generative AI model is operated using Python and TensorFlow.

[1121] Input: Basic information stored in the database

[1122] Output: Generated conversational avatar

[1123] Step 5:

[1124] The server generates an initial dialogue script based on the generated dialogue avatar, using NLG (Natural Language Generation) technology.

[1125] Input: Generated conversation avatar

[1126] Output: Initial interaction script

[1127] Step 6:

[1128] The terminal receives the initial dialogue script sent from the server and displays it to the user through a dialogue interface using HTML and JavaScript. For example, it displays "Hello, I'm your new friend. What are your recent hobbies?"

[1129] Input: Initial interaction script sent by the server

[1130] Output: The interactive interface and initial message displayed to the user

[1131] Step 7:

[1132] The user enters a message in the text box of the dialogue interface and presses the send button. For example, the user enters "I like cooking."

[1133] Input: The message the user types in the text box

[1134] Output: User message to be sent

[1135] Step 8:

[1136] The terminal transmits the message entered by the user to the server.

[1137] Input: The message entered by the user

[1138] Output: User message sent to the server

[1139] Step 9:

[1140] The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The analysis mainly involves semantic analysis and contextual understanding of the text. For example, the message generated is "What dishes have you cooked recently?"

[1141] Input: User message sent from the terminal

[1142] Output: The generated reply message

[1143] Step 10:

[1144] The server sends the generated reply message to the terminal.

[1145] Input: The generated reply message

[1146] Output: Reply message sent to the terminal

[1147] Step 11:

[1148] The terminal receives the reply message sent from the server and displays it to the user through the dialogue interface.

[1149] Input: Reply message sent from the server

[1150] Output: The reply message that is displayed to the user

[1151] Step 12:

[1152] The server stores all interaction history in a database. The stored interaction history is used to identify user interests and new concerns through analytical means, such as clustering algorithms and natural language processing techniques.

[1153] Input: Dialogue history

[1154] Output: Dialogue history stored in a database

[1155] Step 13:

[1156] The server updates the avatar's profile and dialogue script based on the analysis results. This uses a reinforcement learning model to allow the avatar to grow incrementally, making the dialogue with the user more personalized. For example, if the user frequently talks about Italian food, the next dialogue topic might be, "Have you found any new Italian recipes?"

[1157] Input: Analysis results

[1158] Output: Updated avatar profile and dialogue scripts

[1159] Through each of the above steps, the system continuously improves the interaction with the user, helping to reduce the user's sense of loneliness.

[1160] (Application example 1)

[1161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1162] Conventional conversational avatar systems focus on conversational functions to reduce feelings of loneliness, but do not sufficiently consider the purchasing experience and product recommendations that users gain through conversation. Furthermore, the lack of a dynamic product recommendation function based on the user's interests limits the improvement of the purchasing experience. This has led to the challenge of making it difficult to accurately tap into the user's latent purchasing motivation.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1164] In this invention, the server includes input means for a user to input basic information, storage means for storing the input basic information, generation means for generating an interactive avatar based on the stored basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for storing and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, recommendation means for improving the purchasing experience using the interactive avatar, generation means for recommending products based on the user's input, and display means for displaying the products recommended by the recommendation means. This enables product recommendations based on the user's dialogue history and interests, thereby improving the purchasing experience while reducing feelings of loneliness through dialogue.

[1165] A "user" is an individual who uses the system to interact with interactive avatars and receive product recommendations.

[1166] "Basic information" refers to personal data such as the user's name, age, gender, hobbies, and interests.

[1167] "Input means" refers to the interface or device that allows users to input basic information into the system.

[1168] "Storage means" refers to a technical element that stores the input basic information and dialogue content in a database.

[1169] The "generation means" is a function or system for automatically creating a conversational avatar based on the stored basic information.

[1170] "Interaction means" refers to an interface or system that allows the user to exchange messages with the generated interactive avatar.

[1171] "Analysis means" refers to the technology or algorithm used to analyze the history and content of interactions and update or improve the interaction avatar.

[1172] The "update means" is a function for correcting and updating the profile and dialogue content of the dialogue avatar based on the results of the analysis means.

[1173] "Recommendation means" is a function that uses a conversational avatar to suggest the most suitable products to the user in order to improve the purchasing experience.

[1174] The "display means" refers to a display or interface for visually presenting the generated dialogue script and recommended products to the user.

[1175] "Purchasing experience" refers to the experience a user has during the process of selecting a product, purchasing it, or obtaining information about it.

[1176] MODE FOR CARRYING OUT THE INVENTION

[1177] System Configuration and Functions

[1178] This invention is a system that provides interactive avatars using generative AI to improve the shopping experience while reducing feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[1179] User Registration Process

[1180] A user first logs in to the system and enters basic information, including name, age, gender, hobbies, and interests. The user uses a web form as an input method. The device receives the information entered by the user and sends it to the server. The server parses the received data into a format such as JSON and stores it in a database.

[1181] Initial avatar generation and conversation start

[1182] The server generates a conversational avatar based on the stored basic information. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the terminal. The terminal receives this script and displays a conversation screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1183] Facilitating and managing dialogue

[1184] The terminal sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., OpenAI API) and generates an appropriate reply. The generated reply is sent back to the terminal and displayed to the user. The entire history of this dialogue is accumulated in a database by the storage means. This stored history is analyzed by the analysis means to improve the quality of future dialogues.

[1185] Product recommendation feature implementation

[1186] The product recommendation function is provided to improve the purchasing experience through interaction between the interactive avatar and the user. The server uses a generating means to recommend optimal products to the user based on the user's basic information and interaction history. The recommended products are visually presented to the user through the display means of the terminal.

[1187] Avatar Growth and Evolution

[1188] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated using an update means. The avatar grows over time, enabling more personalized dialogue.

[1189] Example

[1190] Initial registration and interaction

[1191] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1192] 2. The device receives this information and sends it to the server, which stores it in a database.

[1193] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1194] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1195] 5. The user replies, "I like cooking."

[1196] Ongoing dialogue and product recommendations

[1197] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[1198] 2. The terminal displays this message to the user, who then replies.

[1199] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1200] 4. Furthermore, to improve the shopping experience, the avatar will recommend products such as, "Today's special Italian cooking ingredients are here."

[1201] Prompt Sentence Examples

[1202] "User name: Taro Yamamoto

[1203] Age: 30

[1204] Gender: Male

[1205] Hobbies: Fashion, gadgets

[1206] Interests: Latest trends, innovative technologies

[1207] Please generate a suitable initial interaction script for this user."

[1208] In this way, the present invention is optimized to reduce the user's sense of loneliness and improve the purchasing experience through interaction.

[1209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1210] Step 1:

[1211] A user logs into the system and fills in basic information such as name, age, gender, hobbies, and interests in a web form.

[1212] Input: User's basic information (name, age, gender, hobbies, interests, etc.)

[1213] Output: Basic information sent to the terminal in JSON format

[1214] Step 2:

[1215] The device sends the basic information entered by the user to the server, which then parses the information in JSON format and stores it in a database.

[1216] Input: User basic information (JSON format)

[1217] Output: Basic information stored in the database

[1218] Step 3:

[1219] The server generates an interactive avatar based on the stored basic information.

[1220] Input: Basic information (from database)

[1221] Output: Generated conversational avatar

[1222] Step 4:

[1223] The server creates an initial interaction script for the avatar and sends it to the terminal, which receives the script and displays it to the user.

[1224] Input: Basic information of the generated conversational avatar

[1225] Output: The initial interaction script is displayed to the user

[1226] Step 5:

[1227] The user inputs and sends a message to the avatar.

[1228] Input: User interaction message

[1229] Output: Message sent to the terminal

[1230] Step 6:

[1231] The terminal sends the user's message to the server, which uses a natural language processing engine to analyze the message and generate an appropriate reply.

[1232] Input: User interaction message

[1233] Output: The generated avatar returned by the server

[1234] Step 7:

[1235] The server generates a reply and sends it to the terminal, which displays the reply to the user.

[1236] Input: Reply message from server

[1237] Output: Avatar reply shown to the user

[1238] Step 8:

[1239] The server stores the content of the dialogue in a database and analyzes the dialogue history.

[1240] Input: Interaction history between user and avatar

[1241] Output: Parsed data

[1242] Step 9:

[1243] The server updates the interactive avatar based on the analysis results and generates new scripts and profiles.

[1244] Input: Analyzed interaction history data

[1245] Output: Updated interactive avatar scripts and profiles

[1246] Step 10:

[1247] The server generates a script for recommending products based on the user's basic information and interaction history, and transmits the recommended products to the terminal, which then displays the recommended products to the user.

[1248] Input: User basic information, interaction history

[1249] Output: Recommended product information

[1250] Through this series of steps, users can reduce their sense of loneliness through interaction with a conversational avatar, further improving their purchasing experience.

[1251] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1252] MODE FOR CARRYING OUT THE INVENTION

[1253] System Configuration and Functions

[1254] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user. The operation of the system is explained in detail below.

[1255] 1. User registration and basic information entry

[1256] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, interests, etc. To do so, the user uses a web form as an input method.

[1257] The terminal receives the information entered by the user and sends it to the server, which analyzes the received data and stores it in a database. The storage means used here is the one used.

[1258] 2. Initial avatar generation and conversation start

[1259] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[1260] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1261] 3. Dialogue progression and emotional engine utilization

[1262] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue."

[1263] The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as, "You seem tired. How do you relax?" The generated reply is sent back to the device and displayed to the user.

[1264] 4. Saving and analyzing dialogue history

[1265] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[1266] The server analyzes the stored dialogue history using an analysis means. By using a natural language processing engine and an emotion engine in combination, it identifies changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[1267] 5. Avatar Growth and Relationship Development

[1268] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "fatigue" or "stress," the avatar will add more conversation scripts about relaxation methods and stress management.

[1269] Users regularly interact with their avatars, which evolve based on the user's interests and emotions. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens. This allows users to receive emotional support and reduce feelings of loneliness.

[1270] Specific examples

[1271] Initial registration and interaction

[1272] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1273] 2. The device receives this information and sends it to the server, which stores it in a database.

[1274] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1275] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1276] 5. The user replies, "I like cooking."

[1277] Continuous dialogue and emotion recognition

[1278] 1. The server analyzes the user's dialogue message and generates the following appropriate response: For example, it generates a reply such as, "I see you enjoy cooking. What dish have you cooked recently?"

[1279] 2. The terminal displays this message to the user, who then replies.

[1280] 3. The server stores the dialogue history and continues to analyze it. If the user types, "I'm very tired today," the emotion engine recognizes "fatigue" and generates a response accordingly. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[1281] 4. The server updates the avatar's profile and dialogue scripts, taking into account the user's dialogue history and emotional state.

[1282] In this way, the system can continuously improve its dialogue with the user and, by leveraging the emotion engine, provide appropriate support depending on the user's emotional state.

[1283] The processing flow will be explained below.

[1284] Step 1:

[1285] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[1286] Step 2:

[1287] The terminal receives the input user information and transmits it to the server.

[1288] Step 3:

[1289] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[1290] Step 4:

[1291] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[1292] Step 5:

[1293] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[1294] Step 6:

[1295] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[1296] Step 7:

[1297] The user enters an initial message in the text box and clicks the "Send" button.

[1298] Step 8:

[1299] The terminal receives the user's input message and sends it to the server.

[1300] Step 9:

[1301] The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, if the message says "I'm very tired today," the emotion is recognized as "fatigue."

[1302] Step 10:

[1303] The server generates an appropriate response based on the analysis results of the emotion engine, for example, "You seem tired. How do you relax?"

[1304] Step 11:

[1305] The server generates a reply and sends it to the terminal.

[1306] Step 12:

[1307] The terminal displays the received reply to the user.

[1308] Step 13:

[1309] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[1310] Step 14:

[1311] The server uses analytical means to analyze the stored dialogue history and emotional state, and uses a natural language processing engine and an emotion engine in combination to identify changes in the user's interests and emotions.

[1312] Step 15:

[1313] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[1314] Step 16:

[1315] Users interact with the avatar periodically, and the avatar evolves based on the user's interests and emotions. The server continuously analyzes and updates the interaction data, deepening the user's relationship with the avatar.

[1316] Example 2

[1317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1318] In modern society, many people feel lonely, and effective countermeasures are needed. Furthermore, there is a need for a system that can accurately grasp a user's emotional state and alleviate loneliness through dialogue. Conventional systems can only provide one-way dialogue based on basic user information, making it difficult to dynamically respond to changes in the user's emotional state or interests.

[1319] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for the user to input basic information, a data storage means for saving the input basic information, a generation means for generating an interactive avatar based on the saved basic information, an interaction means for the user to interact with the generated interactive avatar, a natural language processing means for analyzing the user's message, an emotion analysis means for recognizing the user's emotion, and an update means for updating the interactive avatar based on the analysis result. This enables dynamic interaction according to the user's emotional state and interests, thereby reducing feelings of loneliness.

[1320] "User" refers to the end user of the system who provides basic information and interactive input.

[1321] "Input means" refers to the interface through which users input basic information, such as a web form or an application screen.

[1322] "Data storage means" refers to a mechanism for retaining the basic information entered, including databases and cloud storage.

[1323] "Generation means" refers to the part of the system that has the function of generating an interactive avatar based on stored basic information, and may use a generative AI model.

[1324] "Interactive means" refers to a mechanism for communication between the generated interactive avatar and the user, such as an interactive screen or chat interface.

[1325] "Natural language processing means" refers to technology for analyzing and understanding a user's message, including a natural language processing engine.

[1326] "Emotion analysis means" refers to technology for recognizing the emotional state of a user from their message, and this corresponds to the emotion engine.

[1327] The "update means" refers to a system part that has the function of dynamically changing the profile and dialogue script of the dialogue avatar based on the analysis results.

[1328] "Dialogue history" refers to a record of communication between a user and a dialogue avatar, including saved messages and responses.

[1329] "Analysis means" refers to technology for analyzing stored interaction history and identifying changes in a user's interests and emotions.

[1330] "Growth means" refers to a mechanism for evolving the profile and script of an interactive avatar through continuous dialogue with the user.

[1331] MODE FOR CARRYING OUT THE INVENTION

[1332] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user.

[1333] 1. User registration and basic information entry

[1334] Users log in to the system and enter basic information such as name, age, gender, hobbies, and interests using a web form. The device receives the entered information and sends it to the server. The server analyzes the received data and stores it in a database. The database used here is, for example, MySQL.

[1335] 2. Initial avatar generation and conversation start

[1336] The server generates a conversational avatar based on the stored basic information. A generative AI model (e.g., GPT-3) is used for this generation. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the device. The device receives this script and displays a conversation screen to the user. The user begins a conversation with the avatar by entering a message in the text box and sending it.

[1337] 3. Dialogue progression and emotional engine utilization

[1338] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., NLTK). It then uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue." The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as "You seem tired. How do you relax?" The generated reply is sent from the server to the device, which then displays it to the user.

[1339] 4. Saving and analyzing dialogue history

[1340] The server stores the dialogue history in a database. The stored dialogue history is used as an analytical tool to improve the quality of future dialogues. The server uses a natural language processing engine and an emotion engine to analyze the stored dialogue history and identify changes in the user's interests and emotions. Based on the analysis results, the dialogue avatar's profile and dialogue script are dynamically updated.

[1341] 5. Avatar Growth and Relationship Development

[1342] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "tired" or "stressed," the avatar will add more conversation scripts about relaxation methods and stress management. As the user regularly interacts with the avatar and the avatar evolves based on the user's interests and emotions, the user can receive emotional support and reduce feelings of loneliness.

[1343] Specific examples

[1344] Initial registration and interaction

[1345] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1346] 2. The device sends this information to the server, which stores it in a database.

[1347] 3. The server generates an avatar based on the saved user information and creates an initial interaction script.

[1348] 4. The terminal receives the dialogue script sent from the server and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1349] 5. The user replies, "I love cooking."

[1350] Continuous dialogue and emotion recognition

[1351] 1. The server analyzes the user's dialogue message and generates an appropriate response, such as "I see you enjoy cooking. What did you cook recently?"

[1352] 2. The terminal displays this message to the user, who then replies.

[1353] 3. The server stores and continuously analyzes the conversation history. For example, if a user types, "I'm very tired today," the emotion engine recognizes this and generates a response tailored to the user's needs. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[1354] 4. The server updates the avatar's profile and dialogue script based on the analysis results.

[1355] Prompt Sentence Examples

[1356] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[1357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1358] The flow of this system's program processing

[1359] Step 1: Enter basic user information

[1360] Input: A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests into a web form.

[1361] Processing: The device receives the entered information and sends it to the server using an HTTP POST request.

[1362] Output: The server parses the received data and stores it in a database.

[1363] Specific operation: The server executes an SQL INSERT statement using a storage means, for example, in a MySQL database, to save the data.

[1364] Step 2: Generate the initial avatar

[1365] Input: The server retrieves basic information stored in a database.

[1366] Processing: The server generates an interactive avatar based on the stored basic information. It uses a generative AI model (e.g., GPT-3) to create an avatar profile.

[1367] Output: Generates an initial dialogue script for the generated avatar.

[1368] Specific operation: The user's basic information is input into the generative AI model as a prompt sentence, and an appropriate dialogue script is output.

[1369] Step 3: Send the initial interaction script

[1370] Input: The generated avatar and initial dialogue script.

[1371] Processing: The server sends the generated dialogue script to the terminal.

[1372] Output: The terminal receives the script and displays an interactive screen to the user.

[1373] Specific operation: The terminal uses HTML and JavaScript to render the interactive screen and display the script.

[1374] Step 4: Start interacting with the user

[1375] Input: The user types a message in the text box and sends it.

[1376] Processing: The terminal receives the user's input message and sends it to the server.

[1377] Output: The server passes the received message to a natural language processing engine (e.g., NLTK) for analysis.

[1378] Specific operation: The server passes the message to a natural language processing engine, performs tokenization and morphological analysis, and obtains the analysis results.

[1379] Step 5: Recognizing your emotional state

[1380] Input: Parsed message data.

[1381] Processing: The server uses an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotional state.

[1382] Output: Emotional state recognition results.

[1383] Specific operation: The server passes the analysis results to the emotion engine and outputs the emotional state (e.g., sadness, joy, anger, etc.).

[1384] Step 6: Generate an appropriate reply

[1385] Input: Emotional state recognition results.

[1386] Processing: The server again uses the generative AI model to generate an appropriate reply.

[1387] Output: The generated reply message.

[1388] Specific operation: The user's emotional state is input into the generative AI model as a prompt sentence, and an appropriate reply sentence is output.

[1389] Step 7: Send and view the reply message

[1390] Input: The generated reply message.

[1391] Processing: The server generates a reply and sends it to the terminal.

[1392] Output: The terminal displays the reply message to the user.

[1393] Specific operation: The terminal adds the new message to the interactive screen so that the user can view it.

[1394] Step 8: Saving conversation history

[1395] Input: User and avatar interaction history.

[1396] Processing: The server stores the dialogue history in a database.

[1397] Output: The saved interaction history.

[1398] Specific operation: The server saves the conversation history to the database using the SQL INSERT statement.

[1399] Step 9: Analyzing the dialogue history and updating the avatar

[1400] Input: Saved interaction history.

[1401] Processing: The server uses a natural language processing engine and an emotion engine to analyze the dialogue history and identify changes in the user's interests and emotions.

[1402] Output: Profile and dialogue script updates based on the analysis results.

[1403] Specific operation: Based on the analysis results, the avatar's profile and dialogue script are dynamically changed and reflected in the next dialogue with the user.

[1404] Prompt Sentence Examples

[1405] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[1406] (Application example 2)

[1407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1408] In modern society, user interaction, especially in virtual environments, is an important factor in improving engagement and customer satisfaction. However, conventional dialogue systems have difficulty properly recognizing and responding to users' emotional states. Furthermore, they lack the ability to dynamically adjust dialogue content, which hinders the provision of personalized services. This can lead to users feeling isolated and reduces the quality of their experience in virtual environments. To solve this problem, a system with emotion recognition capabilities and the ability to dynamically adjust dialogue content is needed.

[1409] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes input means for the user to input basic information, storage means for saving the input basic information, generation means for generating an interactive avatar based on the saved basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for saving and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, and means for dialogue with the user in a virtual store, recognizing the user's emotional state using an emotion engine, and dynamically changing the content of the response. This makes it possible to provide appropriate dialogue and personalized services tailored to the user's emotional state in the virtual environment.

[1410] "User" means an individual human being or end user who uses the system.

[1411] "Basic information" refers to information necessary for generating a conversational avatar, such as the user's name, age, gender, hobbies, and interests.

[1412] "Input means" refers to the interface or device that allows users to input basic information into the system.

[1413] "Storage means" refers to a mechanism or device for storing the input basic information and dialogue history in a database or the like.

[1414] "Generator" refers to the process or function that creates an interactive avatar based on the stored basic information.

[1415] "Interaction means" refers to the functions and mechanisms that enable two-way communication between the generated interactive avatar and the user.

[1416] "Analysis means" refers to methods and technologies for analyzing the content of a conversation and understanding the user's emotional state and interests.

[1417] The "update means" refers to a mechanism or method for adaptively changing the profile and dialogue content of the dialogue avatar based on the analysis results.

[1418] An "emotion engine" is an algorithm or technology that recognizes a user's emotional state and provides appropriate feedback.

[1419] A "virtual store" is a virtual store environment created on the Internet where users can browse and purchase products.

[1420] "Personalized services" refer to services that are tailored to the user's individual interests and emotional state.

[1421] This invention is a system that provides a conversational avatar using generative AI to reduce users' feelings of loneliness. In particular, by combining an emotion engine, it senses the user's emotional state and dynamically changes the content of the conversation. This invention aims to provide personalized services according to the user's emotions, especially in virtual stores.

[1422] System Configuration

[1423] The system is constructed by three components: a server, a terminal, and a user. The roles of each component are as follows:

[1424] User registration and basic information entry

[1425] Users must first log in to the system and enter basic information such as their name, age, gender, hobbies, and interests. This is done using input means on the user's device (smartphone, smart glasses, head-mounted display, etc.).

[1426] Initial avatar generation and conversation start

[1427] The server generates a conversational avatar based on the stored basic information. The generated avatar is optimized taking into account the user's basic information. The server sends an initial conversation script to the terminal, and the user can start a conversation with the avatar through a conversation screen on the terminal.

[1428] Dialogue progression and emotional engine utilization

[1429] When a user enters a dialogue message, the device sends it to the server. The server analyzes the message using a natural language processing engine (e.g., the Hugging Face Transformer model) and then uses an emotion engine to recognize the user's emotional state. For example, a message like "I'm very tired today" is recognized as "fatigue." Based on this emotional state, the server generates a reply using OpenAI's GPT-3 model and provides an appropriate response to the user.

[1430] Saving and analyzing conversation history

[1431] The entire dialogue history is saved. The saved history is analyzed and used to improve the quality of future dialogues. As an analytical tool, the server analyzes the dialogue history and uses a natural language processing engine and emotion engine to identify changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively modified.

[1432] Application in virtual stores

[1433] When interacting with users in the virtual store, the avatar recognizes their emotional state and provides personalized services. For example, if a user says, "I've been so busy and stressed this week," the emotion engine will recognize this as "stress" and provide a response such as, "I recommend this aroma diffuser. It contains a highly relaxing scent."

[1434] In this way, appropriate dialogue and personalized services tailored to the user's emotional state can be provided even in a virtual environment.

[1435] Prompt Sentence Examples

[1436] Below is an example of a prompt sentence when a user says, "I've been very busy at work lately and it's stressful."

[1437] User is feeling stressed. User says: Work has been so busy lately that it's been stressful. Response:

[1438] This prompt sentence is fed into OpenAI's GPT-3 to generate an avatar response.

[1439] As described above, by combining an emotion engine and a generative AI, the present invention can personalize interactions with users in a virtual store and improve the user experience.

[1440] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1441] Step 1:

[1442] The user inputs basic information using the terminal. The terminal uses an input means to collect basic information such as the user's name, age, gender, hobbies, and interests, and sends this information to the server. The input is done through an input form such as a text field.

[1443] Input: User's basic information (name, age, gender, hobbies, interests)

[1444] Output: A data packet containing basic information

[1445] Step 2:

[1446] The server stores the received basic information in a database using a storage means, and the stored information is used for future interactions and avatar generation.

[1447] Input: A data packet containing basic information

[1448] Output: Basic information stored in the database

[1449] Step 3:

[1450] The server generates an interactive avatar based on the stored basic information using a generating means, and the generated avatar reflects the user's basic information and is ready for personalized interaction.

[1451] Input: Basic information stored in the database

[1452] Output: Profile of the conversational avatar

[1453] Step 4:

[1454] The server creates an initial dialogue script for the generated dialogue avatar and sends it to the terminal, which uses the received script to display a dialogue screen and prompts the user to start a dialogue.

[1455] Input: Interactive avatar profile

[1456] Output: Initial interaction script

[1457] Step 5:

[1458] Users can start exchanging messages with the avatar through an interactive screen on their device. When users enter questions or comments, the messages are sent to the server via their device.

[1459] Input: User input message

[1460] Output: Message data from the terminal to the server

[1461] Step 6:

[1462] The server uses a natural language processing engine to analyze the user's message and an emotion engine to recognize the user's emotional state. For example, the server recognizes the message "I'm very tired today" as "fatigue."

[1463] Input: User input message

[1464] Output: Emotional state label (e.g., fatigue)

[1465] Step 7:

[1466] The server generates an appropriate response using OpenAI's GPT-3 based on the emotional state label. It constructs a prompt sentence and feeds it into the generative AI to create a reply (e.g., "User is feeling tired. User says: I'm very tired today.").

[1467] Input: Emotional state label, user input message

[1468] Output: The generated response message

[1469] Step 8:

[1470] The server generates a response message and sends it to the terminal, which displays it to the user and continues the dialogue.

[1471] Input: The generated response message

[1472] Output: Response message displayed on the terminal

[1473] Step 9:

[1474] The server stores the dialogue history in a database using a storage means and analyzes the history using an analysis means, thereby clarifying changes in the user's interests and emotions and obtaining information to improve the content of future dialogues.

[1475] Input: Dialogue history

[1476] Output: Analysis results stored in a database

[1477] Step 10:

[1478] The server uses an updater to change the profile and dialogue script of the dialogue avatar based on the analysis results, thereby improving the quality of responses to the user and providing personalized services.

[1479] Input: Analysis results

[1480] Output: Updated interactive avatar profile and script

[1481] Through the above steps, the invention can provide personalized services by interacting with the user in the virtual store according to their emotional state.

[1482] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1483] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1484] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1485] [Fourth embodiment]

[1486] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1487] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1488] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1489] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1490] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1491] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1492] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1493] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1494] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1497] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1498] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1499] MODE FOR CARRYING OUT THE INVENTION

[1500] System Configuration and Functions

[1501] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user. The operation of the system is explained in detail below.

[1502] 1. User Registration Process

[1503] A user first logs in to the system and enters basic information, such as name, age, gender, hobbies, interests, etc. To enter this information, the user uses a web form as an input method.

[1504] The terminal receives the information entered by the user and sends it to the server, which then analyzes the received data in a format such as JSON and stores it in a database.

[1505] 2. Initial avatar generation and conversation start

[1506] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[1507] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1508] 3. Facilitating and managing the dialogue

[1509] The terminal sends the message entered by the user to the server, which then analyzes the received message using a natural language processing engine and generates an appropriate reply. The generated reply is then sent back to the terminal and displayed to the user.

[1510] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[1511] 4. Avatar Growth and Evolution

[1512] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[1513] The avatar grows over time, allowing for more personalized interactions, and this growth mechanism allows the user and avatar to build a closer relationship.

[1514] Specific examples

[1515] Initial registration and interaction

[1516] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1517] 2. The device receives this information and sends it to the server, which stores it in a database.

[1518] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1519] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1520] 5. The user replies, "I like cooking."

[1521] Ongoing dialogue

[1522] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[1523] 2. The terminal displays this message to the user, who then replies.

[1524] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1525] In this way, the system can continually improve its interactions with the user, building a deeper understanding and connection.

[1526] The processing flow will be explained below.

[1527] Step 1:

[1528] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[1529] Step 2:

[1530] The terminal receives the input user information and transmits it to the server.

[1531] Step 3:

[1532] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[1533] Step 4:

[1534] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[1535] Step 5:

[1536] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[1537] Step 6:

[1538] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[1539] Step 7:

[1540] The user enters an initial message in the text box and clicks the "Send" button.

[1541] Step 8:

[1542] The terminal receives the user's input message and sends it to the server.

[1543] Step 9:

[1544] The server uses a natural language processing engine to analyze the received message and generate an appropriate reply, such as "I see you enjoy cooking. What did you make recently?"

[1545] Step 10:

[1546] The server generates a reply and sends it to the terminal.

[1547] Step 11:

[1548] The terminal displays the received reply to the user.

[1549] Step 12:

[1550] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[1551] Step 13:

[1552] The server uses analytical means to analyze the stored dialogue history, for example, to determine that the user frequently talks about Italian food.

[1553] Step 14:

[1554] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[1555] Step 15:

[1556] Users interact with their avatars periodically, and the avatars evolve based on the user's interests. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens.

[1557] Example 1

[1558] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1559] In modern society, the number of people feeling lonely is increasing. Conventional systems have difficulty providing truly personalized interactions with users and lack the means to build lasting relationships. In this situation, there is a need for a system that can reduce users' feelings of loneliness and provide a comfortable interaction experience.

[1560] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1561] In this invention, the server includes: input means for a user to input basic information; storage means for storing the input basic information; generation means for generating a conversational avatar based on the stored basic information; dialogue means for the user to dialogue with the generated conversational avatar; means for sending the user's message to the server and analyzing it with a natural language processing engine to generate an appropriate response; analysis means for storing and analyzing the content of the dialogue; update means for updating the conversational avatar based on the analysis result of the analysis means; and means for storing a history of ongoing dialogue in a database and analyzing it to improve the quality of future dialogues. This allows the user to have a personalized conversation experience and reduces feelings of loneliness.

[1562] "Input means" refers to the means by which system users input basic information (such as name, age, gender, hobbies, interests, etc.), such as using a web form.

[1563] "Storage means" refers to a means for storing basic information entered by users and interaction history in a database, such as a database system such as MySQL or MongoDB.

[1564] "Generation means" means a means for generating a conversational avatar based on the stored basic information, and creates the characteristics of the conversational avatar using a generative AI model (e.g., GPT-3).

[1565] The "interaction means" is a means for a user to interact with the generated interaction avatar, and includes a text box and an interaction interface displayed on the terminal.

[1566] A "natural language processing engine" is a system that analyzes a user's message and generates an appropriate response based on that analysis, using technologies such as spaCy and BERT.

[1567] The "analysis means" refers to a means for saving and analyzing the content of a dialogue, and a system for analyzing the saved dialogue history.

[1568] The "update means" is a means for updating the interactive avatar based on the analysis results of the analysis means, and for example, modifies the avatar profile and the interactive script using a reinforcement learning model.

[1569] "Growth means" refers to means for updating the profile of the interactive avatar and enabling more personalized interactions in order to build a continuous relationship with the user.

[1570] A "database" is a system for managing stored basic information and interaction history, and for searching and extracting information as needed, including, for example, SQL databases and NoSQL databases.

[1571] A "generative AI model" is a machine learning model used to generate a conversational avatar and create a conversation script based on basic user information, including, for example, GPT-3.

[1572] System Configuration and Functions

[1573] This invention is a system that provides interactive avatars using generative AI to reduce feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[1574] User Registration Process

[1575] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, and interests using a web form. The device then formats this information into JSON format and sends it to the server via an HTTP POST request. The server then parses the received data and stores it in a database system such as MySQL or MongoDB.

[1576] Initial avatar generation and conversation start

[1577] The server uses Python and TensorFlow to generate a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored user's basic information. Based on the generated avatar, the server generates an initial conversation script and sends it to the device. The device receives this script and displays it to the user through a conversational interface using HTML and JavaScript. For example, it displays a message such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1578] Facilitating and managing dialogue

[1579] The user enters a message in the displayed text box and presses the send button. For example, the user enters "I like cooking." The device sends this input message to the server. The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The generated reply is sent back to the device and displayed to the user. For example, a message such as "What dish have you cooked recently?" is displayed.

[1580] Avatar Growth and Evolution

[1581] The server stores all dialogue history in a database and analyzes it using analytical tools. This analysis identifies changes in the user's interests and new concerns. Utilizing a natural language processing engine, the server classifies the user's interests using a clustering algorithm (e.g., the K-means algorithm). Based on the results, the avatar's profile and dialogue script are updated. For example, if the user frequently talks about Italian food, the next dialogue might include a question such as, "Have you found any new Italian recipes?"

[1582] Specific examples

[1583] 1. Example of user registration and dialogue initiation

[1584] Users log in to the system and enter basic information such as their name, age, and hobbies.

[1585] The device receives this information and sends it to the server, which stores it in a database.

[1586] The server generates an avatar based on the user information and creates an initial dialogue script.

[1587] The terminal receives the generated dialogue script and displays it to the user. The prompt text displayed is "Hello, I'm your new friend. What are your recent hobbies?"

[1588] The user replies, "I like cooking."

[1589] 2. An example of ongoing dialogue

[1590] The server analyzes the user's dialogue message and generates an appropriate response, for example, "What dish did you cook recently?"

[1591] The terminal displays this message to the user, who then replies.

[1592] The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar's profile will be updated, and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1593] This allows the system to continually improve its interactions with the user, building a deeper understanding and relationship.

[1594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1595] Step 1:

[1596] Users log in to the system and enter basic information such as name, age, gender, hobbies, interests, etc. using a web form built with HTML and JavaScript.

[1597] Input: Basic information (name, age, gender, hobbies, interests)

[1598] Output: Basic information data in JSON format

[1599] Step 2:

[1600] The terminal receives the basic information entered by the user, converts it into JSON format, and sends it to the server via an HTTP POST request.

[1601] Input: Basic information entered by the user into the web form

[1602] Output: JSON format data sent to the server

[1603] Step 3:

[1604] The server parses the received basic information data in JSON format and stores it in a database (e.g., MySQL or MongoDB). This storage process is performed transactionally to ensure data integrity.

[1605] Input: Basic information data in JSON format sent from the terminal

[1606] Output: Basic information stored in the database

[1607] Step 4:

[1608] The server generates a conversational avatar using a generative AI model (e.g., GPT-3) based on the stored basic information. The generative AI model is operated using Python and TensorFlow.

[1609] Input: Basic information stored in the database

[1610] Output: Generated conversational avatar

[1611] Step 5:

[1612] The server generates an initial dialogue script based on the generated dialogue avatar, using NLG (Natural Language Generation) technology.

[1613] Input: Generated conversation avatar

[1614] Output: Initial interaction script

[1615] Step 6:

[1616] The terminal receives the initial dialogue script sent from the server and displays it to the user through a dialogue interface using HTML and JavaScript. For example, it displays "Hello, I'm your new friend. What are your recent hobbies?"

[1617] Input: Initial interaction script sent by the server

[1618] Output: The interactive interface and initial message displayed to the user

[1619] Step 7:

[1620] The user enters a message in the text box of the dialogue interface and presses the send button. For example, the user enters "I like cooking."

[1621] Input: The message the user types in the text box

[1622] Output: User message to be sent

[1623] Step 8:

[1624] The terminal transmits the message entered by the user to the server.

[1625] Input: The message entered by the user

[1626] Output: User message sent to the server

[1627] Step 9:

[1628] The server analyzes the received message using a natural language processing engine (e.g., spaCy or BERT) and generates an appropriate reply. The analysis mainly involves semantic analysis and contextual understanding of the text. For example, the message generated is "What dishes have you cooked recently?"

[1629] Input: User message sent from the terminal

[1630] Output: The generated reply message

[1631] Step 10:

[1632] The server sends the generated reply message to the terminal.

[1633] Input: The generated reply message

[1634] Output: Reply message sent to the terminal

[1635] Step 11:

[1636] The terminal receives the reply message sent from the server and displays it to the user through the dialogue interface.

[1637] Input: Reply message sent from the server

[1638] Output: The reply message that is displayed to the user

[1639] Step 12:

[1640] The server stores all interaction history in a database. The stored interaction history is used to identify user interests and new concerns through analytical means, such as clustering algorithms and natural language processing techniques.

[1641] Input: Dialogue history

[1642] Output: Dialogue history stored in a database

[1643] Step 13:

[1644] The server updates the avatar's profile and dialogue script based on the analysis results. This uses a reinforcement learning model to allow the avatar to grow incrementally, making the dialogue with the user more personalized. For example, if the user frequently talks about Italian food, the next dialogue topic might be, "Have you found any new Italian recipes?"

[1645] Input: Analysis results

[1646] Output: Updated avatar profile and dialogue scripts

[1647] Through each of the above steps, the system continuously improves the interaction with the user, helping to reduce the user's sense of loneliness.

[1648] (Application example 1)

[1649] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1650] Conventional conversational avatar systems focus on conversational functions to reduce feelings of loneliness, but do not sufficiently consider the purchasing experience and product recommendations that users gain through conversation. Furthermore, the lack of a dynamic product recommendation function based on the user's interests limits the improvement of the purchasing experience. This has led to the challenge of making it difficult to accurately tap into the user's latent purchasing motivation.

[1651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1652] In this invention, the server includes input means for a user to input basic information, storage means for storing the input basic information, generation means for generating an interactive avatar based on the stored basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for storing and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, recommendation means for improving the purchasing experience using the interactive avatar, generation means for recommending products based on the user's input, and display means for displaying the products recommended by the recommendation means. This enables product recommendations based on the user's dialogue history and interests, thereby improving the purchasing experience while reducing feelings of loneliness through dialogue.

[1653] A "user" is an individual who uses the system to interact with interactive avatars and receive product recommendations.

[1654] "Basic information" refers to personal data such as the user's name, age, gender, hobbies, and interests.

[1655] "Input means" refers to the interface or device that allows users to input basic information into the system.

[1656] "Storage means" refers to a technical element that stores the input basic information and dialogue content in a database.

[1657] The "generation means" is a function or system for automatically creating a conversational avatar based on the stored basic information.

[1658] "Interaction means" refers to an interface or system that allows the user to exchange messages with the generated interactive avatar.

[1659] "Analysis means" refers to the technology or algorithm used to analyze the history and content of interactions and update or improve the interaction avatar.

[1660] The "update means" is a function for correcting and updating the profile and dialogue content of the dialogue avatar based on the results of the analysis means.

[1661] "Recommendation means" is a function that uses a conversational avatar to suggest the most suitable products to the user in order to improve the purchasing experience.

[1662] The "display means" refers to a display or interface for visually presenting the generated dialogue script and recommended products to the user.

[1663] "Purchasing experience" refers to the experience a user has during the process of selecting a product, purchasing it, or obtaining information about it.

[1664] MODE FOR CARRYING OUT THE INVENTION

[1665] System Configuration and Functions

[1666] This invention is a system that provides interactive avatars using generative AI to improve the shopping experience while reducing feelings of loneliness. The system is built by three parties: a server, a device, and a user.

[1667] User Registration Process

[1668] A user first logs in to the system and enters basic information, including name, age, gender, hobbies, and interests. The user uses a web form as an input method. The device receives the information entered by the user and sends it to the server. The server parses the received data into a format such as JSON and stores it in a database.

[1669] Initial avatar generation and conversation start

[1670] The server generates a conversational avatar based on the stored basic information. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the terminal. The terminal receives this script and displays a conversation screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1671] Facilitating and managing dialogue

[1672] The terminal sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., OpenAI API) and generates an appropriate reply. The generated reply is sent back to the terminal and displayed to the user. The entire history of this dialogue is accumulated in a database by the storage means. This stored history is analyzed by the analysis means to improve the quality of future dialogues.

[1673] Product recommendation feature implementation

[1674] The product recommendation function is provided to improve the purchasing experience through interaction between the interactive avatar and the user. The server uses a generating means to recommend optimal products to the user based on the user's basic information and interaction history. The recommended products are visually presented to the user through the display means of the terminal.

[1675] Avatar Growth and Evolution

[1676] The server analyzes the stored dialogue history using an analysis means. By utilizing a natural language processing engine, changes in the user's interests and new concerns are identified. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated using an update means. The avatar grows over time, enabling more personalized dialogue.

[1677] Example

[1678] Initial registration and interaction

[1679] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1680] 2. The device receives this information and sends it to the server, which stores it in a database.

[1681] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1682] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1683] 5. The user replies, "I like cooking."

[1684] Ongoing dialogue and product recommendations

[1685] 1. The server analyzes the user's dialogue message and generates the next appropriate response, for example, a question such as "What dish did you cook recently?"

[1686] 2. The terminal displays this message to the user, who then replies.

[1687] 3. The server stores and analyzes the conversation history. If the user frequently talks about Italian food, the avatar profile will be updated and the next conversation will include topics such as "Have you found any new Italian recipes?"

[1688] 4. Furthermore, to improve the shopping experience, the avatar will recommend products such as, "Today's special Italian cooking ingredients are here."

[1689] Prompt Sentence Examples

[1690] "User name: Taro Yamamoto

[1691] Age: 30

[1692] Gender: Male

[1693] Hobbies: Fashion, gadgets

[1694] Interests: Latest trends, innovative technologies

[1695] Please generate a suitable initial interaction script for this user."

[1696] In this way, the present invention is optimized to reduce the user's sense of loneliness and improve the purchasing experience through interaction.

[1697] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1698] Step 1:

[1699] A user logs into the system and fills in basic information such as name, age, gender, hobbies, and interests in a web form.

[1700] Input: User's basic information (name, age, gender, hobbies, interests, etc.)

[1701] Output: Basic information sent to the terminal in JSON format

[1702] Step 2:

[1703] The device sends the basic information entered by the user to the server, which then parses the information in JSON format and stores it in a database.

[1704] Input: User basic information (JSON format)

[1705] Output: Basic information stored in the database

[1706] Step 3:

[1707] The server generates an interactive avatar based on the stored basic information.

[1708] Input: Basic information (from database)

[1709] Output: Generated conversational avatar

[1710] Step 4:

[1711] The server creates an initial interaction script for the avatar and sends it to the terminal, which receives the script and displays it to the user.

[1712] Input: Basic information of the generated conversational avatar

[1713] Output: The initial interaction script is displayed to the user

[1714] Step 5:

[1715] The user inputs and sends a message to the avatar.

[1716] Input: User interaction message

[1717] Output: Message sent to the terminal

[1718] Step 6:

[1719] The terminal sends the user's message to the server, which uses a natural language processing engine to analyze the message and generate an appropriate reply.

[1720] Input: User interaction message

[1721] Output: The generated avatar returned by the server

[1722] Step 7:

[1723] The server generates a reply and sends it to the terminal, which displays the reply to the user.

[1724] Input: Reply message from server

[1725] Output: Avatar reply shown to the user

[1726] Step 8:

[1727] The server stores the content of the dialogue in a database and analyzes the dialogue history.

[1728] Input: Interaction history between user and avatar

[1729] Output: Parsed data

[1730] Step 9:

[1731] The server updates the interactive avatar based on the analysis results and generates new scripts and profiles.

[1732] Input: Analyzed interaction history data

[1733] Output: Updated interactive avatar scripts and profiles

[1734] Step 10:

[1735] The server generates a script for recommending products based on the user's basic information and interaction history, and transmits the recommended products to the terminal, which then displays the recommended products to the user.

[1736] Input: User basic information, interaction history

[1737] Output: Recommended product information

[1738] Through this series of steps, users can reduce their sense of loneliness through interaction with a conversational avatar, further improving their purchasing experience.

[1739] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1740] MODE FOR CARRYING OUT THE INVENTION

[1741] System Configuration and Functions

[1742] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user. The operation of the system is explained in detail below.

[1743] 1. User registration and basic information entry

[1744] A user first logs in to the system and enters basic information such as name, age, gender, hobbies, interests, etc. To do so, the user uses a web form as an input method.

[1745] The terminal receives the information entered by the user and sends it to the server, which analyzes the received data and stores it in a database. The storage means used here is the one used.

[1746] 2. Initial avatar generation and conversation start

[1747] The server generates a conversational avatar based on the stored basic information. This is called the "generation means." The generated avatar is constructed in a way that allows optimal conversation, taking into account the user's basic information.

[1748] The server creates an initial dialogue script for the generated avatar and sends it to the terminal. The terminal receives the script and displays an interactive screen for the user. To start a conversation with the avatar, the user enters a message in a text box and sends it.

[1749] 3. Dialogue progression and emotional engine utilization

[1750] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue."

[1751] The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as, "You seem tired. How do you relax?" The generated reply is sent back to the device and displayed to the user.

[1752] 4. Saving and analyzing dialogue history

[1753] The entire history of this dialogue is stored in a database by the storage means, and this stored history is analyzed by the analysis means in order to improve the quality of future dialogues.

[1754] The server analyzes the stored dialogue history using an analysis means. By using a natural language processing engine and an emotion engine in combination, it identifies changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively updated by an update means.

[1755] 5. Avatar Growth and Relationship Development

[1756] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "fatigue" or "stress," the avatar will add more conversation scripts about relaxation methods and stress management.

[1757] Users regularly interact with their avatars, which evolve based on the user's interests and emotions. As interactions continue, and the server continues to analyze and update, the relationship with the avatar deepens. This allows users to receive emotional support and reduce feelings of loneliness.

[1758] Specific examples

[1759] Initial registration and interaction

[1760] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1761] 2. The device receives this information and sends it to the server, which stores it in a database.

[1762] 3. The server generates an avatar based on the user information and creates an initial dialogue script.

[1763] 4. The terminal receives the generated dialogue script and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1764] 5. The user replies, "I like cooking."

[1765] Continuous dialogue and emotion recognition

[1766] 1. The server analyzes the user's dialogue message and generates the following appropriate response: For example, it generates a reply such as, "I see you enjoy cooking. What dish have you cooked recently?"

[1767] 2. The terminal displays this message to the user, who then replies.

[1768] 3. The server stores the dialogue history and continues to analyze it. If the user types, "I'm very tired today," the emotion engine recognizes "fatigue" and generates a response accordingly. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[1769] 4. The server updates the avatar's profile and dialogue scripts, taking into account the user's dialogue history and emotional state.

[1770] In this way, the system can continuously improve its dialogue with the user and, by leveraging the emotion engine, provide appropriate support depending on the user's emotional state.

[1771] The processing flow will be explained below.

[1772] Step 1:

[1773] A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests.

[1774] Step 2:

[1775] The terminal receives the input user information and transmits it to the server.

[1776] Step 3:

[1777] The server analyzes the received user information and saves it in the database using the SQL INSERT statement.

[1778] Step 4:

[1779] The server generates a conversational avatar based on the stored user information, using an AI algorithm as the generation method.

[1780] Step 5:

[1781] The server creates an initial dialogue script for the created dialogue avatar and transmits it to the terminal.

[1782] Step 6:

[1783] The terminal displays the received initial dialogue script to the user, for example, "Hello, I'm your new friend. What are your recent hobbies?"

[1784] Step 7:

[1785] The user enters an initial message in the text box and clicks the "Send" button.

[1786] Step 8:

[1787] The terminal receives the user's input message and sends it to the server.

[1788] Step 9:

[1789] The server analyzes the received message using a natural language processing engine and then uses an emotion engine to recognize the user's emotional state. For example, if the message says "I'm very tired today," the emotion is recognized as "fatigue."

[1790] Step 10:

[1791] The server generates an appropriate response based on the analysis results of the emotion engine, for example, "You seem tired. How do you relax?"

[1792] Step 11:

[1793] The server generates a reply and sends it to the terminal.

[1794] Step 12:

[1795] The terminal displays the received reply to the user.

[1796] Step 13:

[1797] The server stores the dialogue history with the user in a database, using a storage means for successively storing new dialogue history.

[1798] Step 14:

[1799] The server uses analytical means to analyze the stored dialogue history and emotional state, and uses a natural language processing engine and an emotion engine in combination to identify changes in the user's interests and emotions.

[1800] Step 15:

[1801] The server updates the conversation avatar's profile and conversation script based on the analysis results. For example, the server uses the update method to prepare a question for the next conversation, such as "Have you found a new Italian recipe?"

[1802] Step 16:

[1803] Users interact with the avatar periodically, and the avatar evolves based on the user's interests and emotions. The server continuously analyzes and updates the interaction data, deepening the user's relationship with the avatar.

[1804] Example 2

[1805] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1806] In modern society, many people feel lonely, and effective countermeasures are needed. Furthermore, there is a need for a system that can accurately grasp a user's emotional state and alleviate loneliness through dialogue. Conventional systems can only provide one-way dialogue based on basic user information, making it difficult to dynamically respond to changes in the user's emotional state or interests.

[1807] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for the user to input basic information, a data storage means for saving the input basic information, a generation means for generating an interactive avatar based on the saved basic information, an interaction means for the user to interact with the generated interactive avatar, a natural language processing means for analyzing the user's message, an emotion analysis means for recognizing the user's emotion, and an update means for updating the interactive avatar based on the analysis result. This enables dynamic interaction according to the user's emotional state and interests, thereby reducing feelings of loneliness.

[1808] "User" refers to the end user of the system who provides basic information and interactive input.

[1809] "Input means" refers to the interface through which users input basic information, such as a web form or an application screen.

[1810] "Data storage means" refers to a mechanism for retaining the basic information entered, including databases and cloud storage.

[1811] "Generation means" refers to the part of the system that has the function of generating an interactive avatar based on stored basic information, and may use a generative AI model.

[1812] "Interactive means" refers to a mechanism for communication between the generated interactive avatar and the user, such as an interactive screen or chat interface.

[1813] "Natural language processing means" refers to technology for analyzing and understanding a user's message, including a natural language processing engine.

[1814] "Emotion analysis means" refers to technology for recognizing the emotional state of a user from their message, and this corresponds to the emotion engine.

[1815] The "update means" refers to a system part that has the function of dynamically changing the profile and dialogue script of the dialogue avatar based on the analysis results.

[1816] "Dialogue history" refers to a record of communication between a user and a dialogue avatar, including saved messages and responses.

[1817] "Analysis means" refers to technology for analyzing stored interaction history and identifying changes in a user's interests and emotions.

[1818] "Growth means" refers to a mechanism for evolving the profile and script of an interactive avatar through continuous dialogue with the user.

[1819] MODE FOR CARRYING OUT THE INVENTION

[1820] This invention is a system that provides a conversational avatar using generative AI to reduce feelings of loneliness, and by combining it with an emotion engine, it recognizes the user's emotional state and dynamically changes the content of the conversation. The system is built by three parties: a server, a terminal, and a user.

[1821] 1. User registration and basic information entry

[1822] Users log in to the system and enter basic information such as name, age, gender, hobbies, and interests using a web form. The device receives the entered information and sends it to the server. The server analyzes the received data and stores it in a database. The database used here is, for example, MySQL.

[1823] 2. Initial avatar generation and conversation start

[1824] The server generates a conversational avatar based on the stored basic information. A generative AI model (e.g., GPT-3) is used for this generation. The generated avatar is constructed to enable optimal conversation, taking into account the user's basic information. The server creates an initial conversation script for the generated avatar and sends it to the device. The device receives this script and displays a conversation screen to the user. The user begins a conversation with the avatar by entering a message in the text box and sending it.

[1825] 3. Dialogue progression and emotional engine utilization

[1826] The device sends the message entered by the user to the server. The server analyzes the received message using a natural language processing engine (e.g., NLTK). It then uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotional state. For example, a message such as "I'm very tired today" is recognized as "fatigue." The server generates an appropriate reply based on the analysis results of the emotion engine. For example, it generates a reply such as "You seem tired. How do you relax?" The generated reply is sent from the server to the device, which then displays it to the user.

[1827] 4. Saving and analyzing dialogue history

[1828] The server stores the dialogue history in a database. The stored dialogue history is used as an analytical tool to improve the quality of future dialogues. The server uses a natural language processing engine and an emotion engine to analyze the stored dialogue history and identify changes in the user's interests and emotions. Based on the analysis results, the dialogue avatar's profile and dialogue script are dynamically updated.

[1829] 5. Avatar Growth and Relationship Development

[1830] The server updates the conversational avatar's profile and conversation scripts based on the analysis results. For example, if the user frequently expresses emotions such as "tired" or "stressed," the avatar will add more conversation scripts about relaxation methods and stress management. As the user regularly interacts with the avatar and the avatar evolves based on the user's interests and emotions, the user can receive emotional support and reduce feelings of loneliness.

[1831] Specific examples

[1832] Initial registration and interaction

[1833] 1. The user logs in to the system and enters basic information such as name, age, and hobbies.

[1834] 2. The device sends this information to the server, which stores it in a database.

[1835] 3. The server generates an avatar based on the saved user information and creates an initial interaction script.

[1836] 4. The terminal receives the dialogue script sent from the server and displays it to the user. For example, the dialogue begins with a question such as, "Hello, I'm your new friend. What are your recent hobbies?"

[1837] 5. The user replies, "I love cooking."

[1838] Continuous dialogue and emotion recognition

[1839] 1. The server analyzes the user's dialogue message and generates an appropriate response, such as "I see you enjoy cooking. What did you cook recently?"

[1840] 2. The terminal displays this message to the user, who then replies.

[1841] 3. The server stores and continuously analyzes the conversation history. For example, if a user types, "I'm very tired today," the emotion engine recognizes this and generates a response tailored to the user's needs. For example, it generates a message like, "You're tired, aren't you? How do you relax?"

[1842] 4. The server updates the avatar's profile and dialogue script based on the analysis results.

[1843] Prompt Sentence Examples

[1844] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[1845] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1846] The flow of this system's program processing

[1847] Step 1: Enter basic user information

[1848] Input: A user logs into the system and enters basic information such as name, age, gender, hobbies, and interests into a web form.

[1849] Processing: The device receives the entered information and sends it to the server using an HTTP POST request.

[1850] Output: The server parses the received data and stores it in a database.

[1851] Specific operation: The server executes an SQL INSERT statement using a storage means, for example, in a MySQL database, to save the data.

[1852] Step 2: Generate the initial avatar

[1853] Input: The server retrieves basic information stored in a database.

[1854] Processing: The server generates an interactive avatar based on the stored basic information. It uses a generative AI model (e.g., GPT-3) to create an avatar profile.

[1855] Output: Generates an initial dialogue script for the generated avatar.

[1856] Specific operation: The user's basic information is input into the generative AI model as a prompt sentence, and an appropriate dialogue script is output.

[1857] Step 3: Send the initial interaction script

[1858] Input: The generated avatar and initial dialogue script.

[1859] Processing: The server sends the generated dialogue script to the terminal.

[1860] Output: The terminal receives the script and displays an interactive screen to the user.

[1861] Specific operation: The terminal uses HTML and JavaScript to render the interactive screen and display the script.

[1862] Step 4: Start interacting with the user

[1863] Input: The user types a message in the text box and sends it.

[1864] Processing: The terminal receives the user's input message and sends it to the server.

[1865] Output: The server passes the received message to a natural language processing engine (e.g., NLTK) for analysis.

[1866] Specific operation: The server passes the message to a natural language processing engine, performs tokenization and morphological analysis, and obtains the analysis results.

[1867] Step 5: Recognizing your emotional state

[1868] Input: Parsed message data.

[1869] Processing: The server uses an emotion engine (e.g. IBM Watson Tone Analyzer) to recognize the user's emotional state.

[1870] Output: Emotional state recognition results.

[1871] Specific operation: The server passes the analysis results to the emotion engine and outputs the emotional state (e.g., sadness, joy, anger, etc.).

[1872] Step 6: Generate an appropriate reply

[1873] Input: Emotional state recognition results.

[1874] Processing: The server again uses the generative AI model to generate an appropriate reply.

[1875] Output: The generated reply message.

[1876] Specific operation: The user's emotional state is input into the generative AI model as a prompt sentence, and an appropriate reply sentence is output.

[1877] Step 7: Send and view the reply message

[1878] Input: The generated reply message.

[1879] Processing: The server generates a reply and sends it to the terminal.

[1880] Output: The terminal displays the reply message to the user.

[1881] Specific operation: The terminal adds the new message to the interactive screen so that the user can view it.

[1882] Step 8: Saving conversation history

[1883] Input: User and avatar interaction history.

[1884] Processing: The server stores the dialogue history in a database.

[1885] Output: The saved interaction history.

[1886] Specific operation: The server saves the conversation history to the database using the SQL INSERT statement.

[1887] Step 9: Analyzing the dialogue history and updating the avatar

[1888] Input: Saved interaction history.

[1889] Processing: The server uses a natural language processing engine and an emotion engine to analyze the dialogue history and identify changes in the user's interests and emotions.

[1890] Output: Profile and dialogue script updates based on the analysis results.

[1891] Specific operation: Based on the analysis results, the avatar's profile and dialogue script are dynamically changed and reflected in the next dialogue with the user.

[1892] Prompt Sentence Examples

[1893] “When a user says, ‘I feel so tired today,’ the emotion engine should recognize ‘tiredness’ and generate a reply that matches that emotion.”

[1894] (Application example 2)

[1895] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1896] In modern society, user interaction, especially in virtual environments, is an important factor in improving engagement and customer satisfaction. However, conventional dialogue systems have difficulty properly recognizing and responding to users' emotional states. Furthermore, they lack the ability to dynamically adjust dialogue content, which hinders the provision of personalized services. This can lead to users feeling isolated and reduces the quality of their experience in virtual environments. To solve this problem, a system with emotion recognition capabilities and the ability to dynamically adjust dialogue content is needed.

[1897] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes input means for the user to input basic information, storage means for saving the input basic information, generation means for generating an interactive avatar based on the saved basic information, dialogue means for the user to dialogue with the generated interactive avatar, analysis means for saving and analyzing the content of the dialogue, update means for updating the interactive avatar based on the analysis result of the analysis means, and means for dialogue with the user in a virtual store, recognizing the user's emotional state using an emotion engine, and dynamically changing the content of the response. This makes it possible to provide appropriate dialogue and personalized services tailored to the user's emotional state in the virtual environment.

[1898] "User" means an individual human being or end user who uses the system.

[1899] "Basic information" refers to information necessary for generating a conversational avatar, such as the user's name, age, gender, hobbies, and interests.

[1900] "Input means" refers to the interface or device that allows users to input basic information into the system.

[1901] "Storage means" refers to a mechanism or device for storing the input basic information and dialogue history in a database or the like.

[1902] "Generator" refers to the process or function that creates an interactive avatar based on the stored basic information.

[1903] "Interaction means" refers to the functions and mechanisms that enable two-way communication between the generated interactive avatar and the user.

[1904] "Analysis means" refers to methods and technologies for analyzing the content of a conversation and understanding the user's emotional state and interests.

[1905] The "update means" refers to a mechanism or method for adaptively changing the profile and dialogue content of the dialogue avatar based on the analysis results.

[1906] An "emotion engine" is an algorithm or technology that recognizes a user's emotional state and provides appropriate feedback.

[1907] A "virtual store" is a virtual store environment created on the Internet where users can browse and purchase products.

[1908] "Personalized services" refer to services that are tailored to the user's individual interests and emotional state.

[1909] This invention is a system that provides a conversational avatar using generative AI to reduce users' feelings of loneliness. In particular, by combining an emotion engine, it senses the user's emotional state and dynamically changes the content of the conversation. This invention aims to provide personalized services according to the user's emotions, especially in virtual stores.

[1910] System Configuration

[1911] The system is constructed by three components: a server, a terminal, and a user. The roles of each component are as follows:

[1912] User registration and basic information entry

[1913] Users must first log in to the system and enter basic information such as their name, age, gender, hobbies, and interests. This is done using input means on the user's device (smartphone, smart glasses, head-mounted display, etc.).

[1914] Initial avatar generation and conversation start

[1915] The server generates a conversational avatar based on the stored basic information. The generated avatar is optimized taking into account the user's basic information. The server sends an initial conversation script to the terminal, and the user can start a conversation with the avatar through a conversation screen on the terminal.

[1916] Dialogue progression and emotional engine utilization

[1917] When a user enters a dialogue message, the device sends it to the server. The server analyzes the message using a natural language processing engine (e.g., the Hugging Face Transformer model) and then uses an emotion engine to recognize the user's emotional state. For example, a message like "I'm very tired today" is recognized as "fatigue." Based on this emotional state, the server generates a reply using OpenAI's GPT-3 model and provides an appropriate response to the user.

[1918] Saving and analyzing conversation history

[1919] The entire dialogue history is saved. The saved history is analyzed and used to improve the quality of future dialogues. As an analytical tool, the server analyzes the dialogue history and uses a natural language processing engine and emotion engine to identify changes in the user's interests and emotions. Based on this, the dialogue avatar's profile and dialogue script are adaptively modified.

[1920] Application in virtual stores

[1921] When interacting with users in the virtual store, the avatar recognizes their emotional state and provides personalized services. For example, if a user says, "I've been so busy and stressed this week," the emotion engine will recognize this as "stress" and provide a response such as, "I recommend this aroma diffuser. It contains a highly relaxing scent."

[1922] In this way, appropriate dialogue and personalized services tailored to the user's emotional state can be provided even in a virtual environment.

[1923] Prompt Sentence Examples

[1924] Below is an example of a prompt sentence when a user says, "I've been very busy at work lately and it's stressful."

[1925] User is feeling stressed. User says: Work has been so busy lately that it's been stressful. Response:

[1926] This prompt sentence is fed into OpenAI's GPT-3 to generate an avatar response.

[1927] As described above, by combining an emotion engine and a generative AI, the present invention can personalize interactions with users in a virtual store and improve the user experience.

[1928] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1929] Step 1:

[1930] The user inputs basic information using the terminal. The terminal uses an input means to collect basic information such as the user's name, age, gender, hobbies, and interests, and sends this information to the server. The input is done through an input form such as a text field.

[1931] Input: User's basic information (name, age, gender, hobbies, interests)

[1932] Output: A data packet containing basic information

[1933] Step 2:

[1934] The server stores the received basic information in a database using a storage means, and the stored information is used for future interactions and avatar generation.

[1935] Input: A data packet containing basic information

[1936] Output: Basic information stored in the database

[1937] Step 3:

[1938] The server generates an interactive avatar based on the stored basic information using a generating means, and the generated avatar reflects the user's basic information and is ready for personalized interaction.

[1939] Input: Basic information stored in the database

[1940] Output: Profile of the conversational avatar

[1941] Step 4:

[1942] The server creates an initial dialogue script for the generated dialogue avatar and sends it to the terminal, which uses the received script to display a dialogue screen and prompts the user to start a dialogue.

[1943] Input: Interactive avatar profile

[1944] Output: Initial interaction script

[1945] Step 5:

[1946] Users can start exchanging messages with the avatar through an interactive screen on their device. When users enter questions or comments, the messages are sent to the server via their device.

[1947] Input: User input message

[1948] Output: Message data from the terminal to the server

[1949] Step 6:

[1950] The server uses a natural language processing engine to analyze the user's message and an emotion engine to recognize the user's emotional state. For example, the server recognizes the message "I'm very tired today" as "fatigue."

[1951] Input: User input message

[1952] Output: Emotional state label (e.g., fatigue)

[1953] Step 7:

[1954] The server generates an appropriate response using OpenAI's GPT-3 based on the emotional state label. It constructs a prompt sentence and feeds it into the generative AI to create a reply (e.g., "User is feeling tired. User says: I'm very tired today.").

[1955] Input: Emotional state label, user input message

[1956] Output: The generated response message

[1957] Step 8:

[1958] The server generates a response message and sends it to the terminal, which displays it to the user and continues the dialogue.

[1959] Input: The generated response message

[1960] Output: Response message displayed on the terminal

[1961] Step 9:

[1962] The server stores the dialogue history in a database using a storage means and analyzes the history using an analysis means, thereby clarifying changes in the user's interests and emotions and obtaining information to improve the content of future dialogues.

[1963] Input: Dialogue history

[1964] Output: Analysis results stored in a database

[1965] Step 10:

[1966] The server uses an updater to change the profile and dialogue script of the dialogue avatar based on the analysis results, thereby improving the quality of responses to the user and providing personalized services.

[1967] Input: Analysis results

[1968] Output: Updated interactive avatar profile and script

[1969] Through the above steps, the invention can provide personalized services by interacting with the user in the virtual store according to their emotional state.

[1970] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1972] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1973] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1974] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1975] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1976] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1977] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1978] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1979] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1980] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1981] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1982] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1983] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1984] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1985] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1986] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1987] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1988] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1989] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1990] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1991] The following is further disclosed regarding the above embodiment.

[1992] (Claim 1)

[1993] an input means for a user to input basic information;

[1994] a storage means for storing the input basic information;

[1995] generating means for generating an interactive avatar based on the stored basic information;

[1996] an interaction means for a user to interact with the generated interaction avatar;

[1997] an analysis means for storing and analyzing the content of the dialogue;

[1998] updating means for updating an interactive avatar based on the analysis result of said analyzing means;

[1999] A system including:

[2000] (Claim 2)

[2001] A means for sequentially saving and analyzing a user's dialogue history;

[2002] means for adaptively changing the content of conversation and the profile of the avatar based on the analysis results;

[2003] and growth measures to build ongoing relationships with users.

[2004] 10. The system of claim 1.

[2005] (Claim 3)

[2006] a means for allowing the dialogue avatar generated by said generation means to analyze the user's message using a natural language processing engine and generate an appropriate response;

[2007] a means for dynamically changing the dialogue content according to the user's interests and concerns based on the analysis results stored by the storage means;

[2008] 10. The system of claim 1, comprising:

[2009] "Example 1"

[2010] (Claim 1)

[2011] an input means for a user to input basic information;

[2012] a storage means for storing the input basic information;

[2013] generating means for generating an interactive avatar based on the stored basic information;

[2014] an interaction means for a user to interact with the generated interaction avatar;

[2015] a means for transmitting a user's message to a server, analyzing the message using a natural language processing engine, and generating an appropriate response;

[2016] an analysis means for storing and analyzing the content of the dialogue;

[2017] updating means for updating an interactive avatar based on the analysis result of said analyzing means;

[2018] a means for storing a history of ongoing interactions in a database and analyzing it to improve the quality of future interactions;

[2019] A system including:

[2020] (Claim 2)

[2021] A means for sequentially saving and analyzing a user's dialogue history;

[2022] means for adaptively changing the content of conversation and the profile of the avatar based on the analysis results;

[2023] and growth measures to build ongoing relationships with users.

[2024] 10. The system of claim 1.

[2025] (Claim 3)

[2026] a means for allowing the dialogue avatar generated by said generation means to analyze the user's message using a natural language processing engine and generate an appropriate response;

[2027] a means for dynamically changing the dialogue content according to the user's interests and concerns based on the analysis results stored by the storage means;

[2028] A means of growth for updating the avatar's profile using the interaction history and forming a close relationship with the user;

[2029] 10. The system of claim 1, comprising:

[2030] "Application Example 1"

[2031] (Claim 1)

[2032] an input means for a user to input basic information;

[2033] a storage means for storing the input basic information;

[2034] generating means for generating an interactive avatar based on the stored basic information;

[2035] an interaction means for a user to interact with the generated interaction avatar;

[2036] an analysis means for storing and analyzing the content of the dialogue;

[2037] updating means for updating an interactive avatar based on the analysis result of said analyzing means;

[2038] A recommendation method for improving the purchasing experience using a conversational avatar;

[2039] generating means for recommending products based on user input;

[2040] a display means for displaying the products recommended by the recommendation means;

[2041] A system including:

[2042] (Claim 2)

[2043] A means for sequentially saving and analyzing a user's dialogue history;

[2044] means for adaptively changing the content of conversation and the profile of the avatar based on the analysis results;

[2045] and growth measures to build ongoing relationships with users.

[2046] 10. The system of claim 1.

[2047] (Claim 3)

[2048] a means for allowing the dialogue avatar generated by said generation means to analyze the user's message using a natural language processing engine and generate an appropriate response;

[2049] a means for dynamically changing the dialogue content according to the user's interests and concerns based on the analysis results stored by the storage means;

[2050] The recommendation means dynamically recommends products based on the user's purchase history and interests;

[2051] 10. The system of claim 1, comprising:

[2052] "Example 2: Combining Emotion Engines"

[2053] (Claim 1)

[2054] an input means for a user to input basic information;

[2055] a data storage means for storing the input basic information;

[2056] generating means for generating an interactive avatar based on the stored basic information;

[2057] an interaction means for a user to interact with the generated interaction avatar;

[2058] natural language processing means for analyzing the user's message;

[2059] emotion analysis means for recognizing the emotion of a user;

[2060] updating means for updating the interactive avatar based on the analysis results;

[2061] A system including:

[2062] (Claim 2)

[2063] A means for sequentially saving and analyzing a user's dialogue history;

[2064] means for adaptively changing the profile and dialogue content of the dialogue avatar based on the analysis results;

[2065] and growth measures to build ongoing relationships with users.

[2066] 10. The system of claim 1.

[2067] (Claim 3)

[2068] a means for the conversational avatar generated by said generating means to analyze the user's message using natural language processing means and generate an appropriate response;

[2069] means for dynamically changing the dialogue content in accordance with the interests and emotions of the user based on the analysis results stored by the storage means;

[2070] Including,

[2071] 10. The system of claim 1.

[2072] "Application example 2 when combining emotion engines"

[2073] (Claim 1)

[2074] an input means for a user to input basic information;

[2075] a storage means for storing the input basic information;

[2076] generating means for generating an interactive avatar based on the stored basic information;

[2077] an interaction means for a user to interact with the generated interaction avatar;

[2078] an analysis means for storing and analyzing the content of the dialogue;

[2079] updating means for updating an interactive avatar based on the analysis result of said analyzing means;

[2080] a means for interacting with a user in a virtual store, recognizing the user's emotional state using an emotion engine, and dynamically changing the content of the response;

[2081] A system including:

[2082] (Claim 2)

[2083] A means for sequentially saving and analyzing a user's dialogue history;

[2084] means for adaptively changing the content of conversation and the profile of the avatar based on the analysis results;

[2085] and growth measures to build ongoing relationships with users.

[2086] 10. The system of claim 1.

[2087] (Claim 3)

[2088] a means for allowing the dialogue avatar generated by said generation means to analyze the user's message using a natural language processing engine and generate an appropriate response;

[2089] a means for dynamically changing the dialogue content according to the user's interests and concerns based on the analysis results stored by the storage means;

[2090] A means of providing personalized services in a virtual store according to the user's emotional state, and of making product suggestions and providing support;

[2091] 10. The system of claim 1, comprising: [Explanation of symbols] 【...

Claims

1. an input means for a user to input basic information; a storage means for storing the input basic information; generating means for generating an interactive avatar based on the stored basic information; an interaction means for a user to interact with the generated interaction avatar; an analysis means for storing and analyzing the content of the dialogue; updating means for updating an interactive avatar based on the analysis result of said analyzing means; A system including:

2. A means for sequentially saving and analyzing a user's dialogue history; means for adaptively changing the content of conversation and the profile of the avatar based on the analysis results; and growth measures to build ongoing relationships with users. The system of claim 1 .

3. a means for allowing the dialogue avatar generated by said generation means to analyze the user's message using a natural language processing engine and generate an appropriate response; a means for dynamically changing the dialogue content according to the user's interests and concerns based on the analysis results stored by the storage means; The system of claim 1 , comprising:

Images