System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input profile information and interest information; means for collecting and storing the user information; means for placing a generated avatar in a chat room and connecting with the user; means for searching for other users similar to the user in the background; means for notifying the user if the other users are found; and means for switching the chat partner from the avatar to the other users if the user approves.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional matching applications have had problems such as awkward communication between users meeting for the first time or initial interactions not progressing smoothly. In such cases, users often lose interest quickly and discontinue use of the application. Furthermore, many users are reluctant to engage in initial communication due to concerns about safety and privacy. For this reason, there has been a demand for a method that allows users to begin communication with confidence and, at the same time, efficiently match with people who share similar hobbies and interests. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system as follows. The system includes a means for a user to input profile information and interest information, and a means for collecting and saving the user information. It also provides a means for placing a generated avatar in a chat room and connecting with the user. The system has a means for searching for other users similar to the user in the background, and a means for notifying the user when other users are found. The system also includes a means for switching the chat partner from the avatar to the other user if the user approves. Such a system allows a user to first begin a safe and enjoyable conversation with a generated avatar, and then naturally match with real users who share common interests.

[0006] "User" refers to a person who uses a system or application.

[0007] "Profile information" refers to personal information about a user, such as information including name, age, gender, hobbies, interests, etc.

[0008] "Interest information" refers to information such as activities or themes in which a user is particularly interested.

[0009] "Generated avatar" refers to a virtual human character that chats on behalf of a user using AI technology or other generational means.

[0010] "Chat room" refers to a digital space or interface for users to interact with avatars or other users.

[0011] "Background searching" refers to information processing and search processes that occur in parallel and invisibly while users are chatting.

[0012] "Other users" refers to other people in the system other than the current chat user.

[0013] "Means for notifying" refers to a method or system for notifying a user of specific information.

[0014] "Acceptance" refers to a user's affirmative response to a suggestion or notification.

[0015] "Means for switching chat partners" refers to a method or system for changing chat partners from an avatar to a real person. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently match with other users who share common interests.

[0038] 1. User Registration

[0039] Users: To register with the application, they enter their name, profile information (age, gender, hobbies, etc.) and interests. They can also upload a profile photo or ask the generative AI to create an avatar for them.

[0040] Terminal: Sends the information entered by the user to the server.

[0041] Server: Stores the received user information in a database, and if necessary, uses a generation AI to create an avatar based on the user information. The generated avatar data is also stored in the database.

[0042] 2. Start a chat room

[0043] User: Presses the "Start Chat" button in the application.

[0044] Server: Creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to find other users similar to the user based on relevant information.

[0045] 3. Chat Progression

[0046] User: Start a chat with an avatar, type a message and send it.

[0047] Terminal: Sends a message to the server.

[0048] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[0049] 4. Proceeding with the Matching Process

[0050] Server: Runs a background process to search for similar users based on the current chat, the user's profile information, and interests. If a match is found, stores the user's information in a queue.

[0051] 5. User Notifications and Chat Switching

[0052] Server: If a match is successful, a notification is sent to the user, informing them of the other user's information.

[0053] On your device: Show a notification and offer a "yes" or "no" option.

[0054] User: If you approve the match, press the "Yes" button.

[0055] Server: If approved, terminate the current chat session and connect the new user, create a new chat room and transfer the existing chat history to the new user, or start a new session.

[0056] Specific examples

[0057] 1. User Registration

[0058] User: For example, a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or chooses to have an avatar generated by AI.

[0059] Terminal: Sends input information to the server.

[0060] Server: Stores the received information in a database, generates and saves avatars.

[0061] 2. Start a chat room

[0062] User: Presses the "Start Chat" button.

[0063] Server: Create a new chat room, add Hanako Yamada and the generated avatar to the room, and start the matching process in the background.

[0064] 3. Chat Progression

[0065] User: Sends a message to the avatar saying "Hi, I love reading."

[0066] Terminal: Sends a message to the server.

[0067] Server: Passes the received message to the generation AI, generates a response, and sends back "Reading is great, what book have you read recently?"

[0068] 4. Proceeding with the Matching Process

[0069] Server: Search for other users who are interested in reading or traveling based on chat content and user information.

[0070] 5. User Notifications and Chat Switching

[0071] Server: If the match is successful, notify Hanako Yamada, "Similar users found. Do you want to continue?"

[0072] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0073] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[0074] This system allows users to first engage in safe and enjoyable communication with their generated avatars, and then effectively match with other real users who share common interests.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] User: Opens the application and navigates to the registration screen. Fills in the required information (username, profile, interests, etc.). Uploads a profile photo or selects the option to create an avatar using generative AI.

[0078] Step 2:

[0079] Terminal: Sends the information entered by the user to the server.

[0080] Step 3:

[0081] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[0082] Step 4:

[0083] User: Presses the "Start Chat" button within the application.

[0084] Step 5:

[0085] Server: Create a new chat room, get the user's avatar information from the database, and add the avatar to the chat room.

[0086] Step 6:

[0087] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0088] Step 7:

[0089] User: Start a chat with an avatar. Type your message and press the "Send" button.

[0090] Step 8:

[0091] Terminal: Sends messages typed by the user to the server.

[0092] Step 9:

[0093] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[0094] Step 10:

[0095] Terminal: Receives response messages from the server and displays them to the user.

[0096] Step 11:

[0097] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[0098] Step 12:

[0099] Server: If the match is successful, notify the user. Send a message saying "We've found a user similar to you. Do you want to continue?"

[0100] Step 13:

[0101] On your device: Display a notification and offer a choice (yes or no).

[0102] Step 14:

[0103] User: Check the notification and press the "Yes" button to approve the match.

[0104] Step 15:

[0105] Server: If approved, end the current chat session and start a session with the new user.

[0106] Step 16:

[0107] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0108] Step 17:

[0109] Terminal: Displays new chat room information to the user and starts a new chat.

[0110] Step 18:

[0111] User: Continue chatting with a new user.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] In conventional online chat systems, users interact with other real users from the beginning, which can cause tension and anxiety when meeting for the first time. Furthermore, there is a problem that the accuracy of matching users with similar interests is low, making efficient matching difficult. This has led to a need for a system that efficiently matches users with common hobbies and interests so that they can begin communicating with ease.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for a user to input profile information and interest information, means for transmitting the user information to the server, means for storing the received user information in a database and creating an avatar using a generation AI, means for placing the generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when the other user is found, means for switching the chat partner from the avatar to the other user if the user approves, means for generating responses to chat messages in real time using the generated avatar, and means for searching for similar users based on the user's profile information, interest information, and chat content. This allows users to have initial chats in a safe and relaxed state and enables efficient matching with other users who share common interests.

[0117] A "user" is an individual who uses the system to chat and match.

[0118] "Profile information" refers to personal information such as a user's name, age, gender, and hobbies.

[0119] "Interest information" refers to information about activities and topics that interest a user.

[0120] "Server" refers to the computer system that processes information sent by users, stores it in a database, and creates avatars using generative AI.

[0121] "Generative AI" refers to artificial intelligence that automatically generates an avatar based on input information.

[0122] An "avatar" refers to a virtual character that participates in chats on behalf of a user.

[0123] A "chat room" refers to an online space for users and avatars to interact.

[0124] A "background process" refers to work that is carried out in parallel and invisible to the user.

[0125] "Matching" refers to the process of identifying and connecting with other users who share common hobbies and interests.

[0126] "Means for notifying" refers to a function for conveying messages or information to the user.

[0127] "Real-time" refers to a situation where a user sends a message and receives an immediate response.

[0128] "Chat content" refers to messages and conversations exchanged between a user and an avatar or other users.

[0129] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently matches with other users who share common interests.

[0130] User Registration

[0131] The user launches the application and enters their name, profile information (age, gender, hobbies), and interests. They can then upload a profile photo or request the generation AI to create an avatar. The device sends the information entered by the user to the server. The server stores the received user information in a database and, if necessary, uses the generation AI to create an avatar based on the user information. The data of the generated avatar is also stored in the database.

[0132] For example, if a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or asks the AI ​​to create an avatar, the device sends the input information to the server, which then stores the information and generates the avatar.

[0133] Starting a Chat Room

[0134] The user presses the "Start Chat" button in the application. The server creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to search for other users similar to the user based on relevant information.

[0135] Chat progress

[0136] The user starts a chat with the generated avatar, inputs a message, and sends it. The device sends the user's message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device.

[0137] For example, if a user sends a message to an avatar saying, "Hello, I love reading," the device sends the message to the server, which passes it on to the generation AI, which generates a response saying, "Reading is great, what book have you read recently?"

[0138] Progressing through the matching process

[0139] The server runs a background process to search for similar users based on the current chat content, the user's profile information, and their interests. If a match is found, the user's information is stored in a queue.

[0140] User notifications and chat switching

[0141] If the match is successful, the server will notify the user and provide information about the other user. The device will display the notification and provide a "Yes / No" option. The user can press the "Yes" button to approve the match. If approval is granted, the server will end the current chat session and connect with the new user, creating a new chat room and either inheriting the existing chat history or starting a new session.

[0142] For example, after a successful match, the server notifies Hanako Yamada, "Similar users found. Do you want to continue?" and the device displays a "Yes / No" option. If the user selects "Yes," the chat partner will switch to the new user.

[0143] Example of input prompt for generative AI model

[0144] An example of a prompt is, "My name is Hanako Yamada and my hobbies are reading and traveling. Please tell me about some recent books you recommend." In response to this prompt, the generative AI model generates a response such as, "Reading is a great hobby! Of the books you've read recently, the one that made the biggest impression on you was 'XXX'. What books have you read recently?"

[0145] The system allows users to have initial chats in a safe and relaxed environment, and efficiently matches them with other users who share common interests.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Step 1:

[0148] Users: Enter their profile information (name, age, gender, hobbies) and interests, and upload a profile photo or ask the generative AI to create an avatar for them.

[0149] Input: Information entered by a user into an input form.

[0150] Output: When the input form is completed and the submit button is pressed, the data is sent to the device.

[0151] Specific actions: Enter the required information in the form and press the "Submit" button.

[0152] Step 2:

[0153] Terminal: Sends the information entered by the user to the server.

[0154] Input: Request data, including information entered by the user into an input form.

[0155] Output: Sends a POST request to the server in JSON format.

[0156] Specific operation: Converts form data into JSON format and sends it to the server via an HTTP request.

[0157] Step 3:

[0158] Server: Stores the received user information in a database and creates an avatar using a generation AI.

[0159] Input: Profile and interest information submitted by you.

[0160] Output: The new user information and the generated avatar information are saved in the database.

[0161] Specific operations: Parse and validate the incoming data, save the profile information to the database, call the generation AI if necessary to generate an avatar, and save the data.

[0162] Step 4:

[0163] User: Presses the "Start Chat" button in the application.

[0164] Input: "Start Chat" button click event.

[0165] Output: A chat room creation request is sent to the server.

[0166] Specific actions: Click the "Start Chat" button on the application interface.

[0167] Step 5:

[0168] Server: Create a new chat room, add the generated avatar to the room, and start searching for similar users in the background.

[0169] Input: Chat room creation request, user and avatar information.

[0170] Output: A new chat room is created, with a user and avatar added.

[0171] What it does: Creates a new chat room record in the database and launches a background job to start the matching process.

[0172] Step 6:

[0173] User: Start a chat with an avatar, type a message and send it.

[0174] Input: Messages typed in the chat interface.

[0175] Output: The message is sent from the terminal to the server.

[0176] Specific action: Enter text in the chat interface and press the "Send" button.

[0177] Step 7:

[0178] Terminal: Sends a message to the server.

[0179] Input: Message data entered by the user.

[0180] Output: A request containing the message data is sent to the server.

[0181] Specific behavior: Sends message data in JSON format to the server.

[0182] Step 8:

[0183] Server: Passes the received message to the generative AI model, generates an appropriate response, and returns the generated response message to the user, displaying it on the device.

[0184] Input: The message data sent by the user.

[0185] Output: A response message from the generative AI model is generated and sent back to the user.

[0186] Specific operations: Receive and parse message data, call the generation AI, generate a response message, and return the response to the terminal in JSON format.

[0187] Step 9:

[0188] Server: Runs a background process to find similar users based on chat content, user profile information, and interests.

[0189] Input: User profile information, interests, chat content.

[0190] Output: User information of potential matches is stored in a queue.

[0191] What it does: Searches the database to evaluate potential matches and adds them to the queue.

[0192] Step 10:

[0193] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0194] Input: User information of potential matches.

[0195] Output: A notification is sent to the user.

[0196] What it does: Sends notifications via email and in-app notifications.

[0197] Step 11:

[0198] On your device: Show a notification and offer a "yes" or "no" option.

[0199] Input: Notification data from the server.

[0200] Output: A "Yes / No" option is displayed in the user interface.

[0201] Specific behavior: Display a notification message on the interface and display a selection button.

[0202] Step 12:

[0203] User: If you approve the match, press the "Yes" button.

[0204] Input: "Yes" button click event.

[0205] Output: The authorization information is sent to the server.

[0206] Specific action: Click the "Yes" button.

[0207] Step 13:

[0208] Server: If approved, terminate the current chat session and connect the new user. Create a new chat room and continue the existing chat history or start a new session.

[0209] Input: Authorization information, new chat partner's user information, existing chat history.

[0210] Output: A new chat room is created and the user is connected to the new chat partner.

[0211] Specific actions: Create a new chat room, update the database, carry over chat history or start a new session.

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] In modern society, people are increasingly enjoying interactions in virtual environments. However, many users feel uneasy about communicating directly with others they meet for the first time, making it difficult to establish initial connections. Furthermore, in commercial virtual stores, users are not effectively matched, resulting in inactive interactions and limiting the full potential of virtual stores.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes a means for users to input profile information and interest information, a means for collecting and saving the user information, and a means for placing the generated avatar in a chat room and connecting with the user. This allows users to start communication with others for the first time using the indirect means of the avatar, allowing users to engage in conversation with peace of mind. The server also includes a means for matching users within the virtual store, allowing users with common interests to quickly meet and deepen their interactions. This further enhances the experience in the virtual store and is expected to have a commercial impact due to an increase in users.

[0217] A "user" is a person who enters profile information and interest information into the system and interacts within the virtual store.

[0218] "Profile information" is detailed personal information such as a user's name, age, gender, hobbies, interests, etc.

[0219] "Interest information" is information about areas or activities in which a user is particularly interested.

[0220] An "avatar" is a virtual 3D character that represents a user and is created by a generative AI based on the user's profile information.

[0221] A "chat room" is a virtual space where users and avatars can communicate in real time using text and voice.

[0222] A "virtual store" is a virtual commercial facility that users can access in a virtual space, and is a place that offers a variety of products and services.

[0223] "Matching" is the process by which the system identifies other users who share common hobbies and interests with the user and encourages interaction.

[0224] "Generative AI" is an artificial intelligence system that generates avatars based on user information and generates real-time responses to text messages.

[0225] The system for implementing this invention allows users to input profile information and interest information, and then uses a generative AI model to generate an avatar based on that information, enabling users to safely and effectively match and interact with other users within a virtual store.

[0226] The server is built using Node.js and stores user profile information and interest information in a MySQL database. Avatars are generated using OpenAI's GPT-4 model, and the generated avatars are placed in chat rooms on behalf of the users.

[0227] When a user enters a virtual store, they first start chatting with an avatar. The initial communication is conducted in real time using DialogFlow, and the avatar responds to the user's questions and messages.

[0228] Meanwhile, the server searches for similar users in the background based on other users' profile information, interests, and chat content. If the search process is successful and a match is made, the server sends a notification to the user saying, "Similar users have been found. Would you like to interact?" If the user accepts, the chat partner will switch from the avatar to the matched real user.

[0229] A concrete example of this within a virtual store could be the following scenario:

[0230] Examples:

[0231] User Registration:

[0232] Customers enter information such as "Hobbies: music, cooking" and select to have an avatar generated by AI.

[0233] Entering the Virtual Store:

[0234] A customer enters a virtual cafe and begins a conversation with an avatar.

[0235] Chat content:

[0236] What is the most popular menu item at this cafe?

[0237] Matching Notification:

[0238] The server sends a notification saying, "There are users with the same interests. Would you like to interact with them?"

[0239] Chat Toggle:

[0240] Once the customer approves, they can begin chatting with other users and exchange information about the cafe and recommended dishes.

[0241] Prompt for the generative AI model:

[0242] What is the most popular menu item at this cafe?

[0243] "Please tell me some recommended cafes you've visited recently."

[0244] This invention allows users to enjoy initial communication in a safe virtual environment and quickly and effectively match with other users who share common interests, improving the user experience in the virtual store and potentially bringing about commercial benefits.

[0245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0246] Step 1:

[0247] The user enters profile information and interest information.

[0248] Input: The user enters information into the device, including name, age, gender, hobbies, and interests.

[0249] How it works: The terminal parses the entered data and sends it to the server.

[0250] Output: The server stores the received data in a MySQL database.

[0251] Step 2:

[0252] The server generates an avatar using generation AI.

[0253] Input: User profile and interest information stored in a database.

[0254] How it works: The server feeds data into OpenAI's GPT-4 model and sends prompts to generate an avatar.

[0255] Output: The generated avatar information is returned and stored in a database.

[0256] Step 3:

[0257] A user enters a virtual store.

[0258] Input: The user selects a virtual store on the terminal and presses the enter button.

[0259] Operation: The terminal sends a store entry request to the server.

[0260] Output: The server creates a new chat room and places the user and the generated avatar in it.

[0261] Step 4:

[0262] Start an initial chat with your avatar.

[0263] Input: An initial message from the user (e.g., "What's the popular menu item at this cafe?").

[0264] How it works: The device sends a message to the server, which then asks the generating AI for a response via DialogFlow.

[0265] Output: The response from the generative AI (e.g., "This cafe's popular items are cafe lattes and cheesecake") is displayed on the user's device.

[0266] Step 5:

[0267] The server searches in the background for other users similar to the user.

[0268] Input: User profile information, interests, and chat content.

[0269] How it works: The server queries the database to find other users with common interests.

[0270] Output: A list of candidate matches is generated and stored in a queue.

[0271] Step 6:

[0272] If a match is made, a notification is sent to the user.

[0273] Input: A list of possible matches.

[0274] How it works: The server sends a notification to the user saying, "Similar customers found. Would you like to interact?"

[0275] Output: A notification is displayed on the user's device.

[0276] Step 7:

[0277] If the user accepts the notification, the chat partner is switched from the avatar to another user.

[0278] Input: The user's response ("Yes").

[0279] What it does: The server updates the current chat room and sets another real user as the new chat partner.

[0280] Output: The user switches chat partners and starts a new chat.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] The present invention relates to a system that allows a user to start a chat with a generated avatar, searches for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. Furthermore, this system has an emotion engine built in, which can recognize the user's emotions and use them in chat responses and matching.

[0283] 1. User Registration

[0284] User: Opens the application and navigates to the registration screen. Enters name, profile information (age, gender, hobbies, etc.) and interests. Uploads a profile photo or asks for an avatar to be created by generative AI.

[0285] Terminal: Sends the information entered by the user to the server.

[0286] Server: The received user information is stored in a database, and if necessary, a generation AI is used to create an avatar based on the user information, and the data of the generated avatar is also stored in the database.

[0287] 2. Start a chat room

[0288] User: Presses the "Start Chat" button within the application.

[0289] Server: Create a new chat room and add the user and the generated avatar to the room.

[0290] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0291] 3. Chat Progression

[0292] User: Start a chat with an avatar, type a message and send it.

[0293] Terminal: Sends a message to the server.

[0294] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[0295] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[0296] 4. Proceeding with the Matching Process

[0297] Server: Runs a background process that searches for similar users based on the current chat content, user profile information, interests, and sentiment data. If a match is found, the user's information is added to the matching queue.

[0298] 5. User Notifications and Chat Switching

[0299] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0300] On your device: Display a notification and offer a choice (yes or no).

[0301] User: Check the notification and press the "Yes" button to approve the match.

[0302] Server: If approved, end the current chat session and start a session with the new user.

[0303] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0304] Terminal: Displays new chat room information to the user and starts a new chat.

[0305] Specific examples

[0306] 1. User Registration

[0307] User: For example, a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and then uploads a profile photo or chooses to have an avatar generated by AI.

[0308] Terminal: Sends input information to the server.

[0309] Server: Stores the received information in a database, generates and saves avatars.

[0310] 2. Start a chat room

[0311] User: Presses the "Start Chat" button.

[0312] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[0313] 3. Chat Progression

[0314] User: Sends a message to the avatar saying "Hi, I love movies."

[0315] Terminal: Sends a message to the server.

[0316] Server: Passes received messages to the generation AI and generates a response.

[0317] Server: Analyzes the user's emotions using an emotion engine and sends back a tone-adjusted response such as "The movie is great, what movie have you seen recently?"

[0318] 4. Proceeding with the Matching Process

[0319] Server: Search for other users who are interested in movies or sports based on chat content and user information.

[0320] 5. User Notifications and Chat Switching

[0321] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[0322] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0323] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[0324] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[0325] The processing flow will be explained below.

[0326] Step 1:

[0327] User: Opens the application and navigates to the registration screen. Enters the required information (name, profile information, interests), and chooses to upload a profile photo or have a generative AI create an avatar for them.

[0328] Step 2:

[0329] Terminal: Sends the information entered by the user to the server.

[0330] Step 3:

[0331] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[0332] Step 4:

[0333] User: Presses the "Start Chat" button within the application.

[0334] Step 5:

[0335] Server: Create a new chat room and add the user and the generated avatar to the room.

[0336] Step 6:

[0337] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0338] Step 7:

[0339] User: Start a chat with an avatar. Type and send a message.

[0340] Step 8:

[0341] Terminal: Sends a message to the server.

[0342] Step 9:

[0343] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[0344] Step 10:

[0345] Terminal: Receives response messages from the server and displays them to the user.

[0346] Step 11:

[0347] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[0348] Step 12:

[0349] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[0350] Step 13:

[0351] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0352] Step 14:

[0353] On your device: Show a notification and offer a "yes" or "no" option.

[0354] Step 15:

[0355] User: Check the notification and press the "Yes" button if you approve the match.

[0356] Step 16:

[0357] Server: If approved, end the current chat session and start a session with the new user.

[0358] Step 17:

[0359] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0360] Step 18:

[0361] Terminal: Displays new chat room information to the user and starts a new chat.

[0362] Step 19:

[0363] User: Continue chatting with a new user.

[0364] Example 2

[0365] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0366] Conventional online chat systems lack a means for users to communicate with other users with confidence and an efficient way to match users with common interests. Furthermore, they face challenges in generating responses that take users' emotions into account and smoothly switching between chat partners. To address these challenges, a system is needed that recognizes users' emotions to provide more natural dialogue and matches users with common interests in the background.

[0367] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0368] In this invention, the server includes means for users to input profile information and interest information, means for collecting and saving user information, means for placing a generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when another user is found, means for switching the chat partner from the avatar to another user if the user approves, means for recognizing emotions from the user's input message, facial expression, and voice data, and means for adjusting the content and tone of the chat message responses generated by the generation AI based on the recognized emotions. This not only allows users to enjoy natural conversations with other users with peace of mind, but also enables effective matching with other users who share common interests.

[0369] "User" refers to an individual or corporation that uses the system and provides profile information and interest information.

[0370] "Profile information" is basic information about a user, including data such as name, age, sex, address, and occupation.

[0371] "Interest information" is information about activities and topics in which a user is interested, including hobbies and favorite things.

[0372] "Generative AI" refers to artificial intelligence that automatically generates content such as text and images using generative models.

[0373] "Avatar" refers to a digital image or character that represents a user in a virtual world.

[0374] "Chat room" refers to a virtual room for exchanging messages online.

[0375] "Server" refers to the central computer of the system, a device that manages data storage, processing, and communication with other devices and terminals.

[0376] "Background" refers to processing that the system executes automatically, independent of user operations.

[0377] An "emotion engine" refers to a program and system that analyzes and recognizes emotions from a user's messages, voice, facial expressions, etc.

[0378] "Notification" refers to the act of the system sending a message or information to the user.

[0379] "Chat message" refers to text or audio information exchanged between users in a chat room.

[0380] "Matching" refers to the process of connecting users with common interests.

[0381] "Response content" refers to the text of the response that the generation AI generates in response to the user's message.

[0382] "Tone" refers to the style and expression of the chat messages created by the generative AI.

[0383] A "chat partner" refers to another user or avatar with whom a user communicates in a chat room.

[0384] This invention is a system in which a user starts a chat with a generated avatar, searches for similar users in the background, and if a match is successful, switches the chat partner from the avatar to a real person. The system incorporates an emotion engine that can recognize the user's emotions and use them in chat responses and matching. A server, a terminal, a generative AI model, and an emotion engine are required to implement this system.

[0385] System Hardware and Software

[0386] The hardware required to implement the system includes smartphones, tablets, and PCs. Python and Node.js are used for server-side programming. Databases such as MySQL and PostgreSQL are used to store data. GPT-3 and other technologies are used for generative AI, and dedicated emotion analysis software is used as the emotion recognition engine.

[0387] Specific explanation of the process

[0388] 1. User Registration

[0389] Users open the application and enter their name, profile information (age, gender, hobbies, etc.) and interests on the registration screen. They can then upload a profile photo or choose to have their avatar created by generative AI.

[0390] The device converts the information entered by the user into JSON format and sends it to the server via an HTTP POST request.

[0391] The server stores the received user information in a database. If there is a request to create an avatar, it sends an API request to the generation AI and stores the generated avatar data in the database.

[0392] 2. Start a chat room

[0393] The user presses the "Start Chat" button within the application.

[0394] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[0395] 3. Chat Progression

[0396] Users can start chatting with their avatar, typing and sending messages, such as "Hi, I love movies."

[0397] The device sends messages in JSON format to the server.

[0398] The server passes the received message to the generation AI, which generates an appropriate response, sends the generated response back to the user, and displays a response such as "That movie is great. What movie have you seen recently?" on the device.

[0399] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[0400] 4. Proceeding with the Matching Process

[0401] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data. If a match is found, the user's information is added to the matching queue.

[0402] 5. User Notifications and Chat Switching

[0403] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[0404] The device will display a notification and offer a choice ("Yes" or "No").

[0405] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[0406] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[0407] The terminal displays the information about the new chat room to the user and starts a new chat.

[0408] Specific examples

[0409] 1. User Registration

[0410] User: Enter "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports" and choose to upload a profile photo or have an avatar generated by AI.

[0411] Terminal: Sends input information to the server.

[0412] Server: Stores the received information in a database, generates and saves avatars.

[0413] 2. Start a chat room

[0414] User: Presses the "Start Chat" button.

[0415] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[0416] 3. Chat Progression

[0417] User: Sends a message saying "Hi, I love movies."

[0418] Terminal: Sends a message to the server.

[0419] Server: Passes the received message to the generation AI and generates a response. It responds with "That movie is great, what movie have you seen recently?"

[0420] 4. Proceeding with the Matching Process

[0421] Server: Searches for other users who are interested in movies or sports based on chat content and user information.

[0422] 5. User Notifications and Chat Switching

[0423] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[0424] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0425] Server: Create a new chat room and either carry over existing chat history or start a new session.

[0426] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[0427] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0428] Step 1: User Registration

[0429] Users open the application, enter their name, profile information (age, gender, hobbies, etc.), and interests, and choose to upload a profile photo or have a generative AI create an avatar for them.

[0430] Specific actions: Enter your name ("Yamada Taro") and hobbies ("watching movies and sports"), then select and upload a profile picture or tap the "Generate Avatar" button.

[0431] The device converts the entered information into JSON format and sends it to the server via an HTTP POST request.

[0432] Input: Name, Profile Information, Interests, Profile Picture.

[0433] Output: User information in JSON format.

[0434] Specific operation: Enter data into the input form, format it into JSON, and send it to the endpoint " / register".

[0435] The server receives user information in JSON format and stores it in a database. When a request to create an avatar is received, it sends an API request to the AI ​​generator and stores the generated avatar image in the database.

[0436] Input: JSON formatted user information sent from the terminal.

[0437] Output: User information stored in the database, generated avatar.

[0438] Specific operation: The received information is inserted into the database, a request is sent to the generation AI, and the generated avatar data is saved back into the database.

[0439] Step 2: Start a chat room

[0440] The user presses the "Start Chat" button within the application.

[0441] Specific actions: Press the "Start chat" button to begin the process.

[0442] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[0443] Input: User's request to start a chat room.

[0444] Output: New chat room, background search process.

[0445] Specific actions: Generate a new chat room ID, add the user and avatar to the chat room, and start searching for similar users.

[0446] Step 3: Chat Progression

[0447] Users can start chatting with their avatar, typing and sending messages such as "Hi, I love movies."

[0448] Specific actions: Enter information into the text box and click the send button.

[0449] The device sends messages in JSON format to the server.

[0450] Input: The message entered by the user.

[0451] Output: Message data in JSON format.

[0452] Specific operation: Convert the input message into JSON and POST it to the endpoint " / send_message".

[0453] The server passes the received message to the generation AI, which generates an appropriate response, which is then sent back to the user and displayed on the screen.

[0454] Input: Message received from the terminal.

[0455] Output: Response message from the generating AI.

[0456] Specific operation: The received message is passed to the generation AI, which generates a response saying, "The movie is great. What movie have you seen recently?" and sends it back to the device and displays it on the screen.

[0457] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[0458] Input: User message, emotion data.

[0459] Output: The adjusted response message.

[0460] Specific operation: Emotion analysis is performed using the emotion engine, and the generative AI creates a tone-adjusted response.

[0461] Step 4: Proceed with the matching process

[0462] The server runs a background process to search for similar users based on chat content, user profile information, interests, and emotional data.

[0463] Input: User chat content, profile information, interest data, and emotional data.

[0464] Output: Potential matching users.

[0465] What it does: It uses a scoring algorithm to query the database to find similar users and adds potential matches to a queue.

[0466] Step 5: Notify users and switch chat partners

[0467] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[0468] Input: The result of the matching process.

[0469] Output: Informational message.

[0470] Specific behavior: If a new match is found, create a notification message and send it to the user.

[0471] The device will display a notification and offer a choice ("Yes" or "No").

[0472] Input: Notification message from the server.

[0473] Output: The displayed notification and selection options.

[0474] Specific operation: Implement a mechanism to display a pop-up screen and provide the user with options.

[0475] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[0476] Specific action: Tap the "Yes" button to approve.

[0477] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[0478] Input: Acknowledgement from the user.

[0479] Output: New chat session, new chat room.

[0480] Specific actions: Close the old chat room, generate a new chat room ID, and carry over the history if necessary.

[0481] The terminal displays the information about the new chat room to the user and starts a new chat.

[0482] Input: New chat room information.

[0483] Output: Updated chat screen.

[0484] Specific operation: The new chat room information is reflected on the screen and displayed to the user.

[0485] (Application example 2)

[0486] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] Conventional chat systems lack the means to safely and effectively match users with other users. They also struggle to provide a natural, emotion-based chat experience, and are unable to recommend products and services that match the user's interests. The objective of this invention is to provide a system that improves the user experience, enables more natural and personal communication, and recommends appropriate products and services.

[0488] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input profile information and interest information; a means for collecting and saving the user information; a means for placing a generated avatar in a chat room and connecting with the user; a means for searching for other users similar to the user in the background; a means for notifying the user when the other user is found; a means for switching the chat partner from the avatar to the other user if the user approves; a means for analyzing the user's emotions and adjusting the tone of chat messages based on the emotion data; and a means for analyzing the user's input text in real time and recommending appropriate products or services. This allows users to be safely and effectively matched with other users, enjoy a natural chat experience based on their emotions, and receive recommendations for products and services that match their interests.

[0489] "User profile" refers to personal information entered by a user, such as name, age, gender, hobbies, and interests.

[0490] "Generated avatar" refers to a virtual character created by the generation AI based on the user's profile information.

[0491] "Chat room" refers to a virtual space where a user, an avatar, or other users can exchange messages online.

[0492] An "emotion engine" refers to software that recognizes emotions from the user's input text, voice, facial expressions, etc., and uses the results.

[0493] "Searching in the background" refers to the process by which the system automatically searches for other similar users without any explicit user interaction.

[0494] "Matching" refers to the process of finding other users with whom you have common ground and compatibility based on your profile information, interests, and emotional data.

[0495] "Notification" refers to a message that the system displays to inform the user.

[0496] "Adjusting the tone of a chat message" refers to changing the tone and expression of a generated message based on the user's emotional data.

[0497] "Recommending products or services" refers to the system suggesting related products or services based on the user's input text and sentiment.

[0498] "Real-time" refers to immediate response or processing of user input.

[0499] The embodiment of the present invention is based on building a system in which a user starts a chat with a generated avatar, while searching for other users similar to the user in the background, and switching the chat partner from the avatar to a real person if a match is successful, according to the following steps:

[0500] 1. User Registration

[0501] The user opens the application, goes to the registration screen, and enters their name, profile information (age, gender, hobbies, etc.), and interests. They can then upload a profile photo or request that the AI ​​create an avatar for them. The device sends the information the user entered to the server, which stores the received user information in a database and, if necessary, uses the AI ​​to create an avatar based on the user information, storing the generated avatar data in the database.

[0502] 2. Start a chat room

[0503] The user presses a "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[0504] 3. Chat Progression

[0505] The user starts a chat with an avatar, inputs a message, and sends it. The device sends the message to the server, which passes the received message to the generation AI and generates an appropriate response. The generated response message is returned to the user and displayed on the device. The server also uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data. Based on the recognized emotions, the server adjusts the content and tone of the generation AI's response to adapt to the user, providing a more natural chat experience.

[0506] 4. Proceeding with the Matching Process

[0507] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[0508] 5. User Notifications and Chat Switching

[0509] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and provides a choice ("Yes" or "No"). The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays the new chat room information to the user and starts a new chat.

[0510] Hardware and software used

[0511] Generative AI models (e.g., GPT-type language models)

[0512] Emotion engine (e.g., emotion analysis AI)

[0513] Server (e.g. cloud server)

[0514] Devices (e.g. smartphones, head-mounted displays)

[0515] Specific examples

[0516] Consider a scenario where a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and either uploads a profile picture or selects to have an avatar generated by a generative AI. In this case, the user presses the "Start Chat" button to start a chat room and sends the message "Hello, I love movies" to the avatar. The server passes the received message to the generative AI, which generates a response. For example, an emotion engine could analyze the user's emotions and return a tone-adjusted response such as "Movies are great. What movies have you seen recently?"

[0517] Prompt Sentence Examples

[0518] Input sentence: "I've been liking movies lately"

[0519] Prompt: "The user is talking about movies. Analyze the given input for emotions and respond in a friendly manner."

[0520] AI Response: "That's great! What movie did you see recently?"

[0521] As described above, this system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar. Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1: User Registration

[0524] The user opens the application, enters their name, profile information (age, gender, hobbies, etc.) and interests, and either uploads a profile photo or requests that the AI ​​create an avatar. The device sends the entered information to the server. The server stores the received user information in a database, uses the AI ​​to create an avatar based on the user information, and also stores the data of the generated avatar in the database.

[0525] Input: Name, profile information, interests, profile picture or avatar creation request

[0526] Output: Generated avatar data

[0527] Specific behavior:

[0528] User enters information via application

[0529] Send information from the device to the server

[0530] The server stores the information in a database and generates an avatar using a generative AI.

[0531] Step 2: Start a chat room

[0532] The user presses the "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[0533] Input: Press the "Start Chat" button

[0534] Output: Creates a new chat room and populates it with users and avatars.

[0535] Specific behavior:

[0536] User presses "Start Chat" button

[0537] The server creates a new chat room and places users and avatars in it.

[0538] Start searching for other similar users in the background

[0539] Step 3: Chat Progression

[0540] The user starts a chat with the avatar, inputs a message, and sends it. The device sends the message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device. The server uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data, and adjusts the content and tone of the generation AI's response based on the recognized emotion.

[0541] Input: User input message

[0542] Output: The generated response message

[0543] Specific behavior:

[0544] User types and sends message

[0545] The device sends a message to the server

[0546] The server passes the message to the generation AI and generates a response.

[0547] A response message is returned to the user and displayed on the terminal.

[0548] Emotion engine recognizes user emotions and adjusts response tone

[0549] Step 4: Proceed with the matching process

[0550] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[0551] Input: Current chat content, profile information, interests, emotional data

[0552] Output: User information of match candidates

[0553] Specific behavior:

[0554] The server analyzes chat content and user information in the background

[0555] Find other similar users

[0556] If a match is found, add the information to the match queue

[0557] Step 5: Notify users and switch chat partners

[0558] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and offers the option of "Yes" or "No." The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays information about the new chat room to the user and starts a new chat.

[0559] Input: User information of match candidates, user approval

[0560] Output: Create a new chat room and switch chat partners

[0561] Specific behavior:

[0562] The server notifies the user when a match is successful

[0563] Your device will display a notification and offer you the option to choose "Yes" or "No."

[0564] User selects "Yes"

[0565] The server ends the chat session and starts a new session with the user.

[0566] Create a new chat room and take over the information

[0567] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0568] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0569] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0570] [Second embodiment]

[0571] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0572] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0573] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0574] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0575] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0576] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0577] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0578] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0579] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0580] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0581] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0582] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0583] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently match with other users who share common interests.

[0584] 1. User Registration

[0585] Users: To register with the application, they enter their name, profile information (age, gender, hobbies, etc.) and interests. They can also upload a profile photo or ask the generative AI to create an avatar for them.

[0586] Terminal: Sends the information entered by the user to the server.

[0587] Server: Stores the received user information in a database, and if necessary, uses a generation AI to create an avatar based on the user information. The generated avatar data is also stored in the database.

[0588] 2. Start a chat room

[0589] User: Presses the "Start Chat" button in the application.

[0590] Server: Creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to find other users similar to the user based on relevant information.

[0591] 3. Chat Progression

[0592] User: Start a chat with an avatar, type a message and send it.

[0593] Terminal: Sends a message to the server.

[0594] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[0595] 4. Proceeding with the Matching Process

[0596] Server: Runs a background process to search for similar users based on the current chat, the user's profile information, and interests. If a match is found, stores the user's information in a queue.

[0597] 5. User Notifications and Chat Switching

[0598] Server: If a match is successful, a notification is sent to the user, informing them of the other user's information.

[0599] On your device: Show a notification and offer a "yes" or "no" option.

[0600] User: If you approve the match, press the "Yes" button.

[0601] Server: If approved, terminate the current chat session and connect the new user, create a new chat room and transfer the existing chat history to the new user, or start a new session.

[0602] Specific examples

[0603] 1. User Registration

[0604] User: For example, a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or chooses to have an avatar generated by AI.

[0605] Terminal: Sends input information to the server.

[0606] Server: Stores the received information in a database, generates and saves avatars.

[0607] 2. Start a chat room

[0608] User: Presses the "Start Chat" button.

[0609] Server: Create a new chat room, add Hanako Yamada and the generated avatar to the room, and start the matching process in the background.

[0610] 3. Chat Progression

[0611] User: Sends a message to the avatar saying "Hi, I love reading."

[0612] Terminal: Sends a message to the server.

[0613] Server: Passes the received message to the generation AI, generates a response, and sends back "Reading is great, what book have you read recently?"

[0614] 4. Proceeding with the Matching Process

[0615] Server: Search for other users who are interested in reading or traveling based on chat content and user information.

[0616] 5. User Notifications and Chat Switching

[0617] Server: If the match is successful, notify Hanako Yamada, "Similar users found. Do you want to continue?"

[0618] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0619] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[0620] This system allows users to first engage in safe and enjoyable communication with their generated avatars, and then effectively match with other real users who share common interests.

[0621] The processing flow will be explained below.

[0622] Step 1:

[0623] User: Opens the application and navigates to the registration screen. Fills in the required information (username, profile, interests, etc.). Uploads a profile photo or selects the option to create an avatar using generative AI.

[0624] Step 2:

[0625] Terminal: Sends the information entered by the user to the server.

[0626] Step 3:

[0627] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[0628] Step 4:

[0629] User: Presses the "Start Chat" button within the application.

[0630] Step 5:

[0631] Server: Create a new chat room, get the user's avatar information from the database, and add the avatar to the chat room.

[0632] Step 6:

[0633] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0634] Step 7:

[0635] User: Start a chat with an avatar. Type your message and press the "Send" button.

[0636] Step 8:

[0637] Terminal: Sends messages typed by the user to the server.

[0638] Step 9:

[0639] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[0640] Step 10:

[0641] Terminal: Receives response messages from the server and displays them to the user.

[0642] Step 11:

[0643] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[0644] Step 12:

[0645] Server: If the match is successful, notify the user. Send a message saying "We've found a user similar to you. Do you want to continue?"

[0646] Step 13:

[0647] On your device: Display a notification and offer a choice (yes or no).

[0648] Step 14:

[0649] User: Check the notification and press the "Yes" button to approve the match.

[0650] Step 15:

[0651] Server: If approved, end the current chat session and start a session with the new user.

[0652] Step 16:

[0653] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0654] Step 17:

[0655] Terminal: Displays new chat room information to the user and starts a new chat.

[0656] Step 18:

[0657] User: Continue chatting with a new user.

[0658] Example 1

[0659] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0660] In conventional online chat systems, users interact with other real users from the beginning, which can cause tension and anxiety when meeting for the first time. Furthermore, there is a problem that the accuracy of matching users with similar interests is low, making efficient matching difficult. This has led to a need for a system that efficiently matches users with common hobbies and interests so that they can begin communicating with ease.

[0661] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0662] In this invention, the server includes means for a user to input profile information and interest information, means for transmitting the user information to the server, means for storing the received user information in a database and creating an avatar using a generation AI, means for placing the generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when the other user is found, means for switching the chat partner from the avatar to the other user if the user approves, means for generating responses to chat messages in real time using the generated avatar, and means for searching for similar users based on the user's profile information, interest information, and chat content. This allows users to have initial chats in a safe and relaxed state and enables efficient matching with other users who share common interests.

[0663] A "user" is an individual who uses the system to chat and match.

[0664] "Profile information" refers to personal information such as a user's name, age, gender, and hobbies.

[0665] "Interest information" refers to information about activities and topics that interest a user.

[0666] "Server" refers to the computer system that processes information sent by users, stores it in a database, and creates avatars using generative AI.

[0667] "Generative AI" refers to artificial intelligence that automatically generates an avatar based on input information.

[0668] An "avatar" refers to a virtual character that participates in chats on behalf of a user.

[0669] A "chat room" refers to an online space for users and avatars to interact.

[0670] A "background process" refers to work that is carried out in parallel and invisible to the user.

[0671] "Matching" refers to the process of identifying and connecting with other users who share common hobbies and interests.

[0672] "Means for notifying" refers to a function for conveying messages or information to the user.

[0673] "Real-time" refers to a situation where a user sends a message and receives an immediate response.

[0674] "Chat content" refers to messages and conversations exchanged between a user and an avatar or other users.

[0675] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently matches with other users who share common interests.

[0676] User Registration

[0677] The user launches the application and enters their name, profile information (age, gender, hobbies), and interests. They can then upload a profile photo or request the generation AI to create an avatar. The device sends the information entered by the user to the server. The server stores the received user information in a database and, if necessary, uses the generation AI to create an avatar based on the user information. The data of the generated avatar is also stored in the database.

[0678] For example, if a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or asks the AI ​​to create an avatar, the device sends the input information to the server, which then stores the information and generates the avatar.

[0679] Starting a Chat Room

[0680] The user presses the "Start Chat" button in the application. The server creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to search for other users similar to the user based on relevant information.

[0681] Chat progress

[0682] The user starts a chat with the generated avatar, inputs a message, and sends it. The device sends the user's message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device.

[0683] For example, if a user sends a message to an avatar saying, "Hello, I love reading," the device sends the message to the server, which passes it on to the generation AI, which generates a response saying, "Reading is great, what book have you read recently?"

[0684] Progressing through the matching process

[0685] The server runs a background process to search for similar users based on the current chat content, the user's profile information, and their interests. If a match is found, the user's information is stored in a queue.

[0686] User notifications and chat switching

[0687] If the match is successful, the server will notify the user and provide information about the other user. The device will display the notification and provide a "Yes / No" option. The user can press the "Yes" button to approve the match. If approval is granted, the server will end the current chat session and connect with the new user, creating a new chat room and either inheriting the existing chat history or starting a new session.

[0688] For example, after a successful match, the server notifies Hanako Yamada, "Similar users found. Do you want to continue?" and the device displays a "Yes / No" option. If the user selects "Yes," the chat partner will switch to the new user.

[0689] Example of input prompt for generative AI model

[0690] An example of a prompt is, "My name is Hanako Yamada and my hobbies are reading and traveling. Please tell me about some recent books you recommend." In response to this prompt, the generative AI model generates a response such as, "Reading is a great hobby! Of the books you've read recently, the one that made the biggest impression on you was 'XXX'. What books have you read recently?"

[0691] The system allows users to have initial chats in a safe and relaxed environment, and efficiently matches them with other users who share common interests.

[0692] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0693] Step 1:

[0694] Users: Enter their profile information (name, age, gender, hobbies) and interests, and upload a profile photo or ask the generative AI to create an avatar for them.

[0695] Input: Information entered by a user into an input form.

[0696] Output: When the input form is completed and the submit button is pressed, the data is sent to the device.

[0697] Specific actions: Enter the required information in the form and press the "Submit" button.

[0698] Step 2:

[0699] Terminal: Sends the information entered by the user to the server.

[0700] Input: Request data, including information entered by the user into an input form.

[0701] Output: Sends a POST request to the server in JSON format.

[0702] Specific operation: Converts form data into JSON format and sends it to the server via an HTTP request.

[0703] Step 3:

[0704] Server: Stores the received user information in a database and creates an avatar using a generation AI.

[0705] Input: Profile and interest information submitted by you.

[0706] Output: The new user information and the generated avatar information are saved in the database.

[0707] Specific operations: Parse and validate the incoming data, save the profile information to the database, call the generation AI if necessary to generate an avatar, and save the data.

[0708] Step 4:

[0709] User: Presses the "Start Chat" button in the application.

[0710] Input: "Start Chat" button click event.

[0711] Output: A chat room creation request is sent to the server.

[0712] Specific actions: Click the "Start Chat" button on the application interface.

[0713] Step 5:

[0714] Server: Create a new chat room, add the generated avatar to the room, and start searching for similar users in the background.

[0715] Input: Chat room creation request, user and avatar information.

[0716] Output: A new chat room is created, with a user and avatar added.

[0717] What it does: Creates a new chat room record in the database and launches a background job to start the matching process.

[0718] Step 6:

[0719] User: Start a chat with an avatar, type a message and send it.

[0720] Input: Messages typed in the chat interface.

[0721] Output: The message is sent from the terminal to the server.

[0722] Specific action: Enter text in the chat interface and press the "Send" button.

[0723] Step 7:

[0724] Terminal: Sends a message to the server.

[0725] Input: Message data entered by the user.

[0726] Output: A request containing the message data is sent to the server.

[0727] Specific behavior: Sends message data in JSON format to the server.

[0728] Step 8:

[0729] Server: Passes the received message to the generative AI model, generates an appropriate response, and returns the generated response message to the user, displaying it on the device.

[0730] Input: The message data sent by the user.

[0731] Output: A response message from the generative AI model is generated and sent back to the user.

[0732] Specific operations: Receive and parse message data, call the generation AI, generate a response message, and return the response to the terminal in JSON format.

[0733] Step 9:

[0734] Server: Runs a background process to find similar users based on chat content, user profile information, and interests.

[0735] Input: User profile information, interests, chat content.

[0736] Output: User information of potential matches is stored in a queue.

[0737] What it does: Searches the database to evaluate potential matches and adds them to the queue.

[0738] Step 10:

[0739] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0740] Input: User information of potential matches.

[0741] Output: A notification is sent to the user.

[0742] What it does: Sends notifications via email and in-app notifications.

[0743] Step 11:

[0744] On your device: Show a notification and offer a "yes" or "no" option.

[0745] Input: Notification data from the server.

[0746] Output: A "Yes / No" option is displayed in the user interface.

[0747] Specific behavior: Display a notification message on the interface and display a selection button.

[0748] Step 12:

[0749] User: If you approve the match, press the "Yes" button.

[0750] Input: "Yes" button click event.

[0751] Output: The authorization information is sent to the server.

[0752] Specific action: Click the "Yes" button.

[0753] Step 13:

[0754] Server: If approved, terminate the current chat session and connect the new user. Create a new chat room and continue the existing chat history or start a new session.

[0755] Input: Authorization information, new chat partner's user information, existing chat history.

[0756] Output: A new chat room is created and the user is connected to the new chat partner.

[0757] Specific actions: Create a new chat room, update the database, carry over chat history or start a new session.

[0758] (Application example 1)

[0759] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] In modern society, people are increasingly enjoying interactions in virtual environments. However, many users feel uneasy about communicating directly with others they meet for the first time, making it difficult to establish initial connections. Furthermore, in commercial virtual stores, users are not effectively matched, resulting in inactive interactions and limiting the full potential of virtual stores.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0762] In this invention, the server includes a means for users to input profile information and interest information, a means for collecting and saving the user information, and a means for placing the generated avatar in a chat room and connecting with the user. This allows users to start communication with others for the first time using the indirect means of the avatar, allowing users to engage in conversation with peace of mind. The server also includes a means for matching users within the virtual store, allowing users with common interests to quickly meet and deepen their interactions. This further enhances the experience in the virtual store and is expected to have a commercial impact due to an increase in users.

[0763] A "user" is a person who enters profile information and interest information into the system and interacts within the virtual store.

[0764] "Profile information" is detailed personal information such as a user's name, age, gender, hobbies, interests, etc.

[0765] "Interest information" is information about areas or activities in which a user is particularly interested.

[0766] An "avatar" is a virtual 3D character that represents a user and is created by a generative AI based on the user's profile information.

[0767] A "chat room" is a virtual space where users and avatars can communicate in real time using text and voice.

[0768] A "virtual store" is a virtual commercial facility that users can access in a virtual space, and is a place that offers a variety of products and services.

[0769] "Matching" is the process by which the system identifies other users who share common hobbies and interests with the user and encourages interaction.

[0770] "Generative AI" is an artificial intelligence system that generates avatars based on user information and generates real-time responses to text messages.

[0771] The system for implementing this invention allows users to input profile information and interest information, and then uses a generative AI model to generate an avatar based on that information, enabling users to safely and effectively match and interact with other users within a virtual store.

[0772] The server is built using Node.js and stores user profile information and interest information in a MySQL database. Avatars are generated using OpenAI's GPT-4 model, and the generated avatars are placed in chat rooms on behalf of the users.

[0773] When a user enters a virtual store, they first start chatting with an avatar. The initial communication is conducted in real time using DialogFlow, and the avatar responds to the user's questions and messages.

[0774] Meanwhile, the server searches for similar users in the background based on other users' profile information, interests, and chat content. If the search process is successful and a match is made, the server sends a notification to the user saying, "Similar users have been found. Would you like to interact?" If the user accepts, the chat partner will switch from the avatar to the matched real user.

[0775] A concrete example of this within a virtual store could be the following scenario:

[0776] Examples:

[0777] User Registration:

[0778] Customers enter information such as "Hobbies: music, cooking" and select to have an avatar generated by AI.

[0779] Entering the Virtual Store:

[0780] A customer enters a virtual cafe and begins a conversation with an avatar.

[0781] Chat content:

[0782] What is the most popular menu item at this cafe?

[0783] Matching Notification:

[0784] The server sends a notification saying, "There are users with the same interests. Would you like to interact with them?"

[0785] Chat Toggle:

[0786] Once the customer approves, they can begin chatting with other users and exchange information about the cafe and recommended dishes.

[0787] Prompt for the generative AI model:

[0788] What is the most popular menu item at this cafe?

[0789] "Please tell me some recommended cafes you've visited recently."

[0790] This invention allows users to enjoy initial communication in a safe virtual environment and quickly and effectively match with other users who share common interests, improving the user experience in the virtual store and potentially bringing about commercial benefits.

[0791] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0792] Step 1:

[0793] The user enters profile information and interest information.

[0794] Input: The user enters information into the device, including name, age, gender, hobbies, and interests.

[0795] How it works: The terminal parses the entered data and sends it to the server.

[0796] Output: The server stores the received data in a MySQL database.

[0797] Step 2:

[0798] The server generates an avatar using generation AI.

[0799] Input: User profile and interest information stored in a database.

[0800] How it works: The server feeds data into OpenAI's GPT-4 model and sends prompts to generate an avatar.

[0801] Output: The generated avatar information is returned and stored in a database.

[0802] Step 3:

[0803] A user enters a virtual store.

[0804] Input: The user selects a virtual store on the terminal and presses the enter button.

[0805] Operation: The terminal sends a store entry request to the server.

[0806] Output: The server creates a new chat room and places the user and the generated avatar in it.

[0807] Step 4:

[0808] Start an initial chat with your avatar.

[0809] Input: An initial message from the user (e.g., "What's the popular menu item at this cafe?").

[0810] How it works: The device sends a message to the server, which then asks the generating AI for a response via DialogFlow.

[0811] Output: The response from the generative AI (e.g., "This cafe's popular items are cafe lattes and cheesecake") is displayed on the user's device.

[0812] Step 5:

[0813] The server searches in the background for other users similar to the user.

[0814] Input: User profile information, interests, and chat content.

[0815] How it works: The server queries the database to find other users with common interests.

[0816] Output: A list of candidate matches is generated and stored in a queue.

[0817] Step 6:

[0818] If a match is made, a notification is sent to the user.

[0819] Input: A list of possible matches.

[0820] How it works: The server sends a notification to the user saying, "Similar customers found. Would you like to interact?"

[0821] Output: A notification is displayed on the user's device.

[0822] Step 7:

[0823] If the user accepts the notification, the chat partner is switched from the avatar to another user.

[0824] Input: The user's response ("Yes").

[0825] What it does: The server updates the current chat room and sets another real user as the new chat partner.

[0826] Output: The user switches chat partners and starts a new chat.

[0827] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0828] The present invention relates to a system that allows a user to start a chat with a generated avatar, searches for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. Furthermore, this system has an emotion engine built in, which can recognize the user's emotions and use them in chat responses and matching.

[0829] 1. User Registration

[0830] User: Opens the application and navigates to the registration screen. Enters name, profile information (age, gender, hobbies, etc.) and interests. Uploads a profile photo or asks for an avatar to be created by generative AI.

[0831] Terminal: Sends the information entered by the user to the server.

[0832] Server: The received user information is stored in a database, and if necessary, a generation AI is used to create an avatar based on the user information, and the data of the generated avatar is also stored in the database.

[0833] 2. Start a chat room

[0834] User: Presses the "Start Chat" button within the application.

[0835] Server: Create a new chat room and add the user and the generated avatar to the room.

[0836] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0837] 3. Chat Progression

[0838] User: Start a chat with an avatar, type a message and send it.

[0839] Terminal: Sends a message to the server.

[0840] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[0841] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[0842] 4. Proceeding with the Matching Process

[0843] Server: Runs a background process that searches for similar users based on the current chat content, user profile information, interests, and sentiment data. If a match is found, the user's information is added to the matching queue.

[0844] 5. User Notifications and Chat Switching

[0845] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0846] On your device: Display a notification and offer a choice (yes or no).

[0847] User: Check the notification and press the "Yes" button to approve the match.

[0848] Server: If approved, end the current chat session and start a session with the new user.

[0849] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0850] Terminal: Displays new chat room information to the user and starts a new chat.

[0851] Specific examples

[0852] 1. User Registration

[0853] User: For example, a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and then uploads a profile photo or chooses to have an avatar generated by AI.

[0854] Terminal: Sends input information to the server.

[0855] Server: Stores the received information in a database, generates and saves avatars.

[0856] 2. Start a chat room

[0857] User: Presses the "Start Chat" button.

[0858] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[0859] 3. Chat Progression

[0860] User: Sends a message to the avatar saying "Hi, I love movies."

[0861] Terminal: Sends a message to the server.

[0862] Server: Passes received messages to the generation AI and generates a response.

[0863] Server: Analyzes the user's emotions using an emotion engine and sends back a tone-adjusted response such as "The movie is great, what movie have you seen recently?"

[0864] 4. Proceeding with the Matching Process

[0865] Server: Search for other users who are interested in movies or sports based on chat content and user information.

[0866] 5. User Notifications and Chat Switching

[0867] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[0868] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0869] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[0870] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[0871] The processing flow will be explained below.

[0872] Step 1:

[0873] User: Opens the application and navigates to the registration screen. Enters the required information (name, profile information, interests), and chooses to upload a profile photo or have a generative AI create an avatar for them.

[0874] Step 2:

[0875] Terminal: Sends the information entered by the user to the server.

[0876] Step 3:

[0877] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[0878] Step 4:

[0879] User: Presses the "Start Chat" button within the application.

[0880] Step 5:

[0881] Server: Create a new chat room and add the user and the generated avatar to the room.

[0882] Step 6:

[0883] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[0884] Step 7:

[0885] User: Start a chat with an avatar. Type and send a message.

[0886] Step 8:

[0887] Terminal: Sends a message to the server.

[0888] Step 9:

[0889] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[0890] Step 10:

[0891] Terminal: Receives response messages from the server and displays them to the user.

[0892] Step 11:

[0893] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[0894] Step 12:

[0895] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[0896] Step 13:

[0897] Server: If a match is successful, the server notifies the user and provides the other user's information.

[0898] Step 14:

[0899] On your device: Show a notification and offer a "yes" or "no" option.

[0900] Step 15:

[0901] User: Check the notification and press the "Yes" button if you approve the match.

[0902] Step 16:

[0903] Server: If approved, end the current chat session and start a session with the new user.

[0904] Step 17:

[0905] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[0906] Step 18:

[0907] Terminal: Displays new chat room information to the user and starts a new chat.

[0908] Step 19:

[0909] User: Continue chatting with a new user.

[0910] Example 2

[0911] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] Conventional online chat systems lack a means for users to communicate with other users with confidence and an efficient way to match users with common interests. Furthermore, they face challenges in generating responses that take users' emotions into account and smoothly switching between chat partners. To address these challenges, a system is needed that recognizes users' emotions to provide more natural dialogue and matches users with common interests in the background.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0914] In this invention, the server includes means for users to input profile information and interest information, means for collecting and saving user information, means for placing a generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when another user is found, means for switching the chat partner from the avatar to another user if the user approves, means for recognizing emotions from the user's input message, facial expression, and voice data, and means for adjusting the content and tone of the chat message responses generated by the generation AI based on the recognized emotions. This not only allows users to enjoy natural conversations with other users with peace of mind, but also enables effective matching with other users who share common interests.

[0915] "User" refers to an individual or corporation that uses the system and provides profile information and interest information.

[0916] "Profile information" is basic information about a user, including data such as name, age, sex, address, and occupation.

[0917] "Interest information" is information about activities and topics in which a user is interested, including hobbies and favorite things.

[0918] "Generative AI" refers to artificial intelligence that automatically generates content such as text and images using generative models.

[0919] "Avatar" refers to a digital image or character that represents a user in a virtual world.

[0920] "Chat room" refers to a virtual room for exchanging messages online.

[0921] "Server" refers to the central computer of the system, a device that manages data storage, processing, and communication with other devices and terminals.

[0922] "Background" refers to processing that the system executes automatically, independent of user operations.

[0923] An "emotion engine" refers to a program and system that analyzes and recognizes emotions from a user's messages, voice, facial expressions, etc.

[0924] "Notification" refers to the act of the system sending a message or information to the user.

[0925] "Chat message" refers to text or audio information exchanged between users in a chat room.

[0926] "Matching" refers to the process of connecting users with common interests.

[0927] "Response content" refers to the text of the response that the generation AI generates in response to the user's message.

[0928] "Tone" refers to the style and expression of the chat messages created by the generative AI.

[0929] A "chat partner" refers to another user or avatar with whom a user communicates in a chat room.

[0930] This invention is a system in which a user starts a chat with a generated avatar, searches for similar users in the background, and if a match is successful, switches the chat partner from the avatar to a real person. The system incorporates an emotion engine that can recognize the user's emotions and use them in chat responses and matching. A server, a terminal, a generative AI model, and an emotion engine are required to implement this system.

[0931] System Hardware and Software

[0932] The hardware required to implement the system includes smartphones, tablets, and PCs. Python and Node.js are used for server-side programming. Databases such as MySQL and PostgreSQL are used to store data. GPT-3 and other technologies are used for generative AI, and dedicated emotion analysis software is used as the emotion recognition engine.

[0933] Specific explanation of the process

[0934] 1. User Registration

[0935] Users open the application and enter their name, profile information (age, gender, hobbies, etc.) and interests on the registration screen. They can then upload a profile photo or choose to have their avatar created by generative AI.

[0936] The device converts the information entered by the user into JSON format and sends it to the server via an HTTP POST request.

[0937] The server stores the received user information in a database. If there is a request to create an avatar, it sends an API request to the generation AI and stores the generated avatar data in the database.

[0938] 2. Start a chat room

[0939] The user presses the "Start Chat" button within the application.

[0940] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[0941] 3. Chat Progression

[0942] Users can start chatting with their avatar, typing and sending messages, such as "Hi, I love movies."

[0943] The device sends messages in JSON format to the server.

[0944] The server passes the received message to the generation AI, which generates an appropriate response, sends the generated response back to the user, and displays a response such as "That movie is great. What movie have you seen recently?" on the device.

[0945] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[0946] 4. Proceeding with the Matching Process

[0947] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data. If a match is found, the user's information is added to the matching queue.

[0948] 5. User Notifications and Chat Switching

[0949] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[0950] The device will display a notification and offer a choice ("Yes" or "No").

[0951] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[0952] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[0953] The terminal displays the information about the new chat room to the user and starts a new chat.

[0954] Specific examples

[0955] 1. User Registration

[0956] User: Enter "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports" and choose to upload a profile photo or have an avatar generated by AI.

[0957] Terminal: Sends input information to the server.

[0958] Server: Stores the received information in a database, generates and saves avatars.

[0959] 2. Start a chat room

[0960] User: Presses the "Start Chat" button.

[0961] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[0962] 3. Chat Progression

[0963] User: Sends a message saying "Hi, I love movies."

[0964] Terminal: Sends a message to the server.

[0965] Server: Passes the received message to the generation AI and generates a response. It responds with "That movie is great, what movie have you seen recently?"

[0966] 4. Proceeding with the Matching Process

[0967] Server: Searches for other users who are interested in movies or sports based on chat content and user information.

[0968] 5. User Notifications and Chat Switching

[0969] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[0970] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[0971] Server: Create a new chat room and either carry over existing chat history or start a new session.

[0972] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[0973] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0974] Step 1: User Registration

[0975] Users open the application, enter their name, profile information (age, gender, hobbies, etc.), and interests, and choose to upload a profile photo or have a generative AI create an avatar for them.

[0976] Specific actions: Enter your name ("Yamada Taro") and hobbies ("watching movies and sports"), then select and upload a profile picture or tap the "Generate Avatar" button.

[0977] The device converts the entered information into JSON format and sends it to the server via an HTTP POST request.

[0978] Input: Name, Profile Information, Interests, Profile Picture.

[0979] Output: User information in JSON format.

[0980] Specific operation: Enter data into the input form, format it into JSON, and send it to the endpoint " / register".

[0981] The server receives user information in JSON format and stores it in a database. When a request to create an avatar is received, it sends an API request to the AI ​​generator and stores the generated avatar image in the database.

[0982] Input: JSON formatted user information sent from the terminal.

[0983] Output: User information stored in the database, generated avatar.

[0984] Specific operation: The received information is inserted into the database, a request is sent to the generation AI, and the generated avatar data is saved back into the database.

[0985] Step 2: Start a chat room

[0986] The user presses the "Start Chat" button within the application.

[0987] Specific actions: Press the "Start chat" button to begin the process.

[0988] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[0989] Input: User's request to start a chat room.

[0990] Output: New chat room, background search process.

[0991] Specific actions: Generate a new chat room ID, add the user and avatar to the chat room, and start searching for similar users.

[0992] Step 3: Chat Progression

[0993] Users can start chatting with their avatar, typing and sending messages such as "Hi, I love movies."

[0994] Specific actions: Enter information into the text box and click the send button.

[0995] The device sends messages in JSON format to the server.

[0996] Input: The message entered by the user.

[0997] Output: Message data in JSON format.

[0998] Specific operation: Convert the input message into JSON and POST it to the endpoint " / send_message".

[0999] The server passes the received message to the generation AI, which generates an appropriate response, which is then sent back to the user and displayed on the screen.

[1000] Input: Message received from the terminal.

[1001] Output: Response message from the generating AI.

[1002] Specific operation: The received message is passed to the generation AI, which generates a response saying, "The movie is great. What movie have you seen recently?" and sends it back to the device and displays it on the screen.

[1003] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[1004] Input: User message, emotion data.

[1005] Output: The adjusted response message.

[1006] Specific operation: Emotion analysis is performed using the emotion engine, and the generative AI creates a tone-adjusted response.

[1007] Step 4: Proceed with the matching process

[1008] The server runs a background process to search for similar users based on chat content, user profile information, interests, and emotional data.

[1009] Input: User chat content, profile information, interest data, and emotional data.

[1010] Output: Potential matching users.

[1011] What it does: It uses a scoring algorithm to query the database to find similar users and adds potential matches to a queue.

[1012] Step 5: Notify users and switch chat partners

[1013] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[1014] Input: The result of the matching process.

[1015] Output: Informational message.

[1016] Specific behavior: If a new match is found, create a notification message and send it to the user.

[1017] The device will display a notification and offer a choice ("Yes" or "No").

[1018] Input: Notification message from the server.

[1019] Output: The displayed notification and selection options.

[1020] Specific operation: Implement a mechanism to display a pop-up screen and provide the user with options.

[1021] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[1022] Specific action: Tap the "Yes" button to approve.

[1023] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[1024] Input: Acknowledgement from the user.

[1025] Output: New chat session, new chat room.

[1026] Specific actions: Close the old chat room, generate a new chat room ID, and carry over the history if necessary.

[1027] The terminal displays the information about the new chat room to the user and starts a new chat.

[1028] Input: New chat room information.

[1029] Output: Updated chat screen.

[1030] Specific operation: The new chat room information is reflected on the screen and displayed to the user.

[1031] (Application example 2)

[1032] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1033] Conventional chat systems lack the means to safely and effectively match users with other users. They also struggle to provide a natural, emotion-based chat experience, and are unable to recommend products and services that match the user's interests. The objective of this invention is to provide a system that improves the user experience, enables more natural and personal communication, and recommends appropriate products and services.

[1034] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input profile information and interest information; a means for collecting and saving the user information; a means for placing a generated avatar in a chat room and connecting with the user; a means for searching for other users similar to the user in the background; a means for notifying the user when the other user is found; a means for switching the chat partner from the avatar to the other user if the user approves; a means for analyzing the user's emotions and adjusting the tone of chat messages based on the emotion data; and a means for analyzing the user's input text in real time and recommending appropriate products or services. This allows users to be safely and effectively matched with other users, enjoy a natural chat experience based on their emotions, and receive recommendations for products and services that match their interests.

[1035] "User profile" refers to personal information entered by a user, such as name, age, gender, hobbies, and interests.

[1036] "Generated avatar" refers to a virtual character created by the generation AI based on the user's profile information.

[1037] "Chat room" refers to a virtual space where a user, an avatar, or other users can exchange messages online.

[1038] An "emotion engine" refers to software that recognizes emotions from the user's input text, voice, facial expressions, etc., and uses the results.

[1039] "Searching in the background" refers to the process by which the system automatically searches for other similar users without any explicit user interaction.

[1040] "Matching" refers to the process of finding other users with whom you have common ground and compatibility based on your profile information, interests, and emotional data.

[1041] "Notification" refers to a message that the system displays to inform the user.

[1042] "Adjusting the tone of a chat message" refers to changing the tone and expression of a generated message based on the user's emotional data.

[1043] "Recommending products or services" refers to the system suggesting related products or services based on the user's input text and sentiment.

[1044] "Real-time" refers to immediate response or processing of user input.

[1045] The embodiment of the present invention is based on building a system in which a user starts a chat with a generated avatar, while searching for other users similar to the user in the background, and switching the chat partner from the avatar to a real person if a match is successful, according to the following steps:

[1046] 1. User Registration

[1047] The user opens the application, goes to the registration screen, and enters their name, profile information (age, gender, hobbies, etc.), and interests. They can then upload a profile photo or request that the AI ​​create an avatar for them. The device sends the information the user entered to the server, which stores the received user information in a database and, if necessary, uses the AI ​​to create an avatar based on the user information, storing the generated avatar data in the database.

[1048] 2. Start a chat room

[1049] The user presses a "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[1050] 3. Chat Progression

[1051] The user starts a chat with an avatar, inputs a message, and sends it. The device sends the message to the server, which passes the received message to the generation AI and generates an appropriate response. The generated response message is returned to the user and displayed on the device. The server also uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data. Based on the recognized emotions, the server adjusts the content and tone of the generation AI's response to adapt to the user, providing a more natural chat experience.

[1052] 4. Proceeding with the Matching Process

[1053] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[1054] 5. User Notifications and Chat Switching

[1055] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and provides a choice ("Yes" or "No"). The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays the new chat room information to the user and starts a new chat.

[1056] Hardware and software used

[1057] Generative AI models (e.g., GPT-type language models)

[1058] Emotion engine (e.g., emotion analysis AI)

[1059] Server (e.g. cloud server)

[1060] Devices (e.g. smartphones, head-mounted displays)

[1061] Specific examples

[1062] Consider a scenario where a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and either uploads a profile picture or selects to have an avatar generated by a generative AI. In this case, the user presses the "Start Chat" button to start a chat room and sends the message "Hello, I love movies" to the avatar. The server passes the received message to the generative AI, which generates a response. For example, an emotion engine could analyze the user's emotions and return a tone-adjusted response such as "Movies are great. What movies have you seen recently?"

[1063] Prompt Sentence Examples

[1064] Input sentence: "I've been liking movies lately"

[1065] Prompt: "The user is talking about movies. Analyze the given input for emotions and respond in a friendly manner."

[1066] AI Response: "That's great! What movie did you see recently?"

[1067] As described above, this system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar. Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[1068] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1069] Step 1: User Registration

[1070] The user opens the application, enters their name, profile information (age, gender, hobbies, etc.) and interests, and either uploads a profile photo or requests that the AI ​​create an avatar. The device sends the entered information to the server. The server stores the received user information in a database, uses the AI ​​to create an avatar based on the user information, and also stores the data of the generated avatar in the database.

[1071] Input: Name, profile information, interests, profile picture or avatar creation request

[1072] Output: Generated avatar data

[1073] Specific behavior:

[1074] User enters information via application

[1075] Send information from the device to the server

[1076] The server stores the information in a database and generates an avatar using a generative AI.

[1077] Step 2: Start a chat room

[1078] The user presses the "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[1079] Input: Press the "Start Chat" button

[1080] Output: Creates a new chat room and populates it with users and avatars.

[1081] Specific behavior:

[1082] User presses "Start Chat" button

[1083] The server creates a new chat room and places users and avatars in it.

[1084] Start searching for other similar users in the background

[1085] Step 3: Chat Progression

[1086] The user starts a chat with the avatar, inputs a message, and sends it. The device sends the message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device. The server uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data, and adjusts the content and tone of the generation AI's response based on the recognized emotion.

[1087] Input: User input message

[1088] Output: The generated response message

[1089] Specific behavior:

[1090] User types and sends message

[1091] The device sends a message to the server

[1092] The server passes the message to the generation AI and generates a response.

[1093] A response message is returned to the user and displayed on the terminal.

[1094] Emotion engine recognizes user emotions and adjusts response tone

[1095] Step 4: Proceed with the matching process

[1096] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[1097] Input: Current chat content, profile information, interests, emotional data

[1098] Output: User information of match candidates

[1099] Specific behavior:

[1100] The server analyzes chat content and user information in the background

[1101] Find other similar users

[1102] If a match is found, add the information to the match queue

[1103] Step 5: Notify users and switch chat partners

[1104] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and offers the option of "Yes" or "No." The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays the information about the new chat room to the user and starts a new chat.

[1105] Input: User information of match candidates, user approval

[1106] Output: Create a new chat room and switch chat partners

[1107] Specific behavior:

[1108] The server notifies the user when a match is successful

[1109] Your device will display a notification and offer you the option to choose "Yes" or "No."

[1110] User selects "Yes"

[1111] The server ends the chat session and starts a new session with the user.

[1112] Create a new chat room and take over the information

[1113] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1114] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1115] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1116] [Third embodiment]

[1117] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1118] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1119] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1120] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1121] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1122] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1123] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1124] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1125] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1126] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1127] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1128] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1129] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently match with other users who share common interests.

[1130] 1. User Registration

[1131] Users: To register with the application, they enter their name, profile information (age, gender, hobbies, etc.) and interests. They can also upload a profile photo or ask the generative AI to create an avatar for them.

[1132] Terminal: Sends the information entered by the user to the server.

[1133] Server: Stores the received user information in a database, and if necessary, uses a generation AI to create an avatar based on the user information. The generated avatar data is also stored in the database.

[1134] 2. Start a chat room

[1135] User: Presses the "Start Chat" button in the application.

[1136] Server: Creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to find other users similar to the user based on relevant information.

[1137] 3. Chat Progression

[1138] User: Start a chat with an avatar, type a message and send it.

[1139] Terminal: Sends a message to the server.

[1140] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[1141] 4. Proceeding with the Matching Process

[1142] Server: Runs a background process to search for similar users based on the current chat, the user's profile information, and interests. If a match is found, stores the user's information in a queue.

[1143] 5. User Notifications and Chat Switching

[1144] Server: If a match is successful, a notification is sent to the user, informing them of the other user's information.

[1145] On your device: Show a notification and offer a "yes" or "no" option.

[1146] User: If you approve the match, press the "Yes" button.

[1147] Server: If approved, terminate the current chat session and connect the new user, create a new chat room and transfer the existing chat history to the new user, or start a new session.

[1148] Specific examples

[1149] 1. User Registration

[1150] User: For example, a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or chooses to have an avatar generated by AI.

[1151] Terminal: Sends input information to the server.

[1152] Server: Stores the received information in a database, generates and saves avatars.

[1153] 2. Start a chat room

[1154] User: Presses the "Start Chat" button.

[1155] Server: Create a new chat room, add Hanako Yamada and the generated avatar to the room, and start the matching process in the background.

[1156] 3. Chat Progression

[1157] User: Sends a message to the avatar saying "Hi, I love reading."

[1158] Terminal: Sends a message to the server.

[1159] Server: Passes the received message to the generation AI, generates a response, and sends back "Reading is great, what book have you read recently?"

[1160] 4. Proceeding with the Matching Process

[1161] Server: Search for other users who are interested in reading or traveling based on chat content and user information.

[1162] 5. User Notifications and Chat Switching

[1163] Server: If the match is successful, notify Hanako Yamada, "Similar users found. Do you want to continue?"

[1164] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[1165] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[1166] This system allows users to first engage in safe and enjoyable communication with their generated avatars, and then effectively match with other real users who share common interests.

[1167] The processing flow will be explained below.

[1168] Step 1:

[1169] User: Opens the application and navigates to the registration screen. Fills in the required information (username, profile, interests, etc.). Uploads a profile photo or selects the option to create an avatar using generative AI.

[1170] Step 2:

[1171] Terminal: Sends the information entered by the user to the server.

[1172] Step 3:

[1173] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[1174] Step 4:

[1175] User: Presses the "Start Chat" button within the application.

[1176] Step 5:

[1177] Server: Create a new chat room, get the user's avatar information from the database, and add the avatar to the chat room.

[1178] Step 6:

[1179] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1180] Step 7:

[1181] User: Start a chat with an avatar. Type your message and press the "Send" button.

[1182] Step 8:

[1183] Terminal: Sends messages typed by the user to the server.

[1184] Step 9:

[1185] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[1186] Step 10:

[1187] Terminal: Receives response messages from the server and displays them to the user.

[1188] Step 11:

[1189] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[1190] Step 12:

[1191] Server: If the match is successful, notify the user. Send a message saying "We've found a user similar to you. Do you want to continue?"

[1192] Step 13:

[1193] On your device: Display a notification and offer a choice (yes or no).

[1194] Step 14:

[1195] User: Check the notification and press the "Yes" button to approve the match.

[1196] Step 15:

[1197] Server: If approved, end the current chat session and start a session with the new user.

[1198] Step 16:

[1199] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1200] Step 17:

[1201] Terminal: Displays new chat room information to the user and starts a new chat.

[1202] Step 18:

[1203] User: Continue chatting with a new user.

[1204] Example 1

[1205] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1206] In conventional online chat systems, users interact with other real users from the beginning, which can cause tension and anxiety when meeting for the first time. Furthermore, there is a problem that the accuracy of matching users with similar interests is low, making efficient matching difficult. This has led to a need for a system that efficiently matches users with common hobbies and interests so that they can begin communicating with ease.

[1207] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1208] In this invention, the server includes means for a user to input profile information and interest information, means for transmitting the user information to the server, means for storing the received user information in a database and creating an avatar using a generation AI, means for placing the generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when the other user is found, means for switching the chat partner from the avatar to the other user if the user approves, means for generating responses to chat messages in real time using the generated avatar, and means for searching for similar users based on the user's profile information, interest information, and chat content. This allows users to have initial chats in a safe and relaxed state and enables efficient matching with other users who share common interests.

[1209] A "user" is an individual who uses the system to chat and match.

[1210] "Profile information" refers to personal information such as a user's name, age, gender, and hobbies.

[1211] "Interest information" refers to information about activities and topics that interest a user.

[1212] "Server" refers to the computer system that processes information sent by users, stores it in a database, and creates avatars using generative AI.

[1213] "Generative AI" refers to artificial intelligence that automatically generates an avatar based on input information.

[1214] An "avatar" refers to a virtual character that participates in chats on behalf of a user.

[1215] A "chat room" refers to an online space for users and avatars to interact.

[1216] A "background process" refers to work that is carried out in parallel and invisible to the user.

[1217] "Matching" refers to the process of identifying and connecting with other users who share common hobbies and interests.

[1218] "Means for notifying" refers to a function for conveying messages or information to the user.

[1219] "Real-time" refers to a situation where a user sends a message and receives an immediate response.

[1220] "Chat content" refers to messages and conversations exchanged between a user and an avatar or other users.

[1221] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently matches with other users who share common interests.

[1222] User Registration

[1223] The user launches the application and enters their name, profile information (age, gender, hobbies), and interests. They can then upload a profile photo or request the generation AI to create an avatar. The device sends the information entered by the user to the server. The server stores the received user information in a database and, if necessary, uses the generation AI to create an avatar based on the user information. The data of the generated avatar is also stored in the database.

[1224] For example, if a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or asks the AI ​​to create an avatar, the device sends the input information to the server, which then stores the information and generates the avatar.

[1225] Starting a Chat Room

[1226] The user presses the "Start Chat" button in the application. The server creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to search for other users similar to the user based on relevant information.

[1227] Chat progress

[1228] The user starts a chat with the generated avatar, inputs a message, and sends it. The device sends the user's message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device.

[1229] For example, if a user sends a message to an avatar saying, "Hello, I love reading," the device sends the message to the server, which passes it on to the generation AI, which generates a response saying, "Reading is great, what book have you read recently?"

[1230] Progressing through the matching process

[1231] The server runs a background process to search for similar users based on the current chat content, the user's profile information, and their interests. If a match is found, the user's information is stored in a queue.

[1232] User notifications and chat switching

[1233] If the match is successful, the server will notify the user and provide information about the other user. The device will display the notification and provide a "Yes / No" option. The user can press the "Yes" button to approve the match. If approval is granted, the server will end the current chat session and connect with the new user, creating a new chat room and either inheriting the existing chat history or starting a new session.

[1234] For example, after a successful match, the server notifies Hanako Yamada, "Similar users found. Do you want to continue?" and the device displays a "Yes / No" option. If the user selects "Yes," the chat partner will switch to the new user.

[1235] Example of input prompt for generative AI model

[1236] An example of a prompt is, "My name is Hanako Yamada and my hobbies are reading and traveling. Please tell me about some recent books you recommend." In response to this prompt, the generative AI model generates a response such as, "Reading is a great hobby! Of the books you've read recently, the one that made the biggest impression on you was 'XXX'. What books have you read recently?"

[1237] The system allows users to have initial chats in a safe and relaxed environment, and efficiently matches them with other users who share common interests.

[1238] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1239] Step 1:

[1240] Users: Enter their profile information (name, age, gender, hobbies) and interests, and upload a profile photo or ask the generative AI to create an avatar for them.

[1241] Input: Information entered by a user into an input form.

[1242] Output: When the input form is completed and the submit button is pressed, the data is sent to the device.

[1243] Specific actions: Enter the required information in the form and press the "Submit" button.

[1244] Step 2:

[1245] Terminal: Sends the information entered by the user to the server.

[1246] Input: Request data, including information entered by the user into an input form.

[1247] Output: Sends a POST request to the server in JSON format.

[1248] Specific operation: Converts form data into JSON format and sends it to the server via an HTTP request.

[1249] Step 3:

[1250] Server: Stores the received user information in a database and creates an avatar using a generation AI.

[1251] Input: Profile and interest information submitted by you.

[1252] Output: The new user information and the generated avatar information are saved in the database.

[1253] Specific operations: Parse and validate the incoming data, save the profile information to the database, call the generation AI if necessary to generate an avatar, and save the data.

[1254] Step 4:

[1255] User: Presses the "Start Chat" button in the application.

[1256] Input: "Start Chat" button click event.

[1257] Output: A chat room creation request is sent to the server.

[1258] Specific actions: Click the "Start Chat" button on the application interface.

[1259] Step 5:

[1260] Server: Create a new chat room, add the generated avatar to the room, and start searching for similar users in the background.

[1261] Input: Chat room creation request, user and avatar information.

[1262] Output: A new chat room is created, with a user and avatar added.

[1263] What it does: Creates a new chat room record in the database and launches a background job to start the matching process.

[1264] Step 6:

[1265] User: Start a chat with an avatar, type a message and send it.

[1266] Input: Messages typed in the chat interface.

[1267] Output: The message is sent from the terminal to the server.

[1268] Specific action: Enter text in the chat interface and press the "Send" button.

[1269] Step 7:

[1270] Terminal: Sends a message to the server.

[1271] Input: Message data entered by the user.

[1272] Output: A request containing the message data is sent to the server.

[1273] Specific behavior: Sends message data in JSON format to the server.

[1274] Step 8:

[1275] Server: Passes the received message to the generative AI model, generates an appropriate response, and returns the generated response message to the user, displaying it on the device.

[1276] Input: The message data sent by the user.

[1277] Output: A response message from the generative AI model is generated and sent back to the user.

[1278] Specific operations: Receive and parse message data, call the generation AI, generate a response message, and return the response to the terminal in JSON format.

[1279] Step 9:

[1280] Server: Runs a background process to find similar users based on chat content, user profile information, and interests.

[1281] Input: User profile information, interests, chat content.

[1282] Output: User information of potential matches is stored in a queue.

[1283] What it does: Searches the database to evaluate potential matches and adds them to the queue.

[1284] Step 10:

[1285] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1286] Input: User information of potential matches.

[1287] Output: A notification is sent to the user.

[1288] What it does: Sends notifications via email and in-app notifications.

[1289] Step 11:

[1290] On your device: Show a notification and offer a "yes" or "no" option.

[1291] Input: Notification data from the server.

[1292] Output: A "Yes / No" option is displayed in the user interface.

[1293] Specific behavior: Display a notification message on the interface and display a selection button.

[1294] Step 12:

[1295] User: If you approve the match, press the "Yes" button.

[1296] Input: "Yes" button click event.

[1297] Output: The authorization information is sent to the server.

[1298] Specific action: Click the "Yes" button.

[1299] Step 13:

[1300] Server: If approved, terminate the current chat session and connect the new user. Create a new chat room and continue the existing chat history or start a new session.

[1301] Input: Authorization information, new chat partner's user information, existing chat history.

[1302] Output: A new chat room is created and the user is connected to the new chat partner.

[1303] Specific actions: Create a new chat room, update the database, carry over chat history or start a new session.

[1304] (Application example 1)

[1305] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1306] In modern society, people are increasingly enjoying interactions in virtual environments. However, many users feel uneasy about communicating directly with others they meet for the first time, making it difficult to establish initial connections. Furthermore, in commercial virtual stores, users are not effectively matched, resulting in inactive interactions and limiting the full potential of virtual stores.

[1307] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1308] In this invention, the server includes a means for users to input profile information and interest information, a means for collecting and saving the user information, and a means for placing the generated avatar in a chat room and connecting with the user. This allows users to start communication with others for the first time using the indirect means of the avatar, allowing users to engage in conversation with peace of mind. The server also includes a means for matching users within the virtual store, allowing users with common interests to quickly meet and deepen their interactions. This further enhances the experience in the virtual store and is expected to have a commercial impact due to an increase in users.

[1309] A "user" is a person who enters profile information and interest information into the system and interacts within the virtual store.

[1310] "Profile information" is detailed personal information such as a user's name, age, gender, hobbies, interests, etc.

[1311] "Interest information" is information about areas or activities in which a user is particularly interested.

[1312] An "avatar" is a virtual 3D character that represents a user and is created by a generative AI based on the user's profile information.

[1313] A "chat room" is a virtual space where users and avatars can communicate in real time using text and voice.

[1314] A "virtual store" is a virtual commercial facility that users can access in a virtual space, and is a place that offers a variety of products and services.

[1315] "Matching" is the process by which the system identifies other users who share common hobbies and interests with the user and encourages interaction.

[1316] "Generative AI" is an artificial intelligence system that generates avatars based on user information and generates real-time responses to text messages.

[1317] The system for implementing this invention allows users to input profile information and interest information, and then uses a generative AI model to generate an avatar based on that information, enabling users to safely and effectively match and interact with other users within a virtual store.

[1318] The server is built using Node.js and stores user profile information and interest information in a MySQL database. Avatars are generated using OpenAI's GPT-4 model, and the generated avatars are placed in chat rooms on behalf of the users.

[1319] When a user enters a virtual store, they first start chatting with an avatar. The initial communication is conducted in real time using DialogFlow, and the avatar responds to the user's questions and messages.

[1320] Meanwhile, the server searches for similar users in the background based on other users' profile information, interests, and chat content. If the search process is successful and a match is made, the server sends a notification to the user saying, "Similar users have been found. Would you like to interact?" If the user accepts, the chat partner will switch from the avatar to the matched real user.

[1321] A concrete example of this within a virtual store could be the following scenario:

[1322] Examples:

[1323] User Registration:

[1324] Customers enter information such as "Hobbies: music, cooking" and select to have an avatar generated by AI.

[1325] Entering the Virtual Store:

[1326] A customer enters a virtual cafe and begins a conversation with an avatar.

[1327] Chat content:

[1328] What is the most popular menu item at this cafe?

[1329] Matching Notification:

[1330] The server sends a notification saying, "There are users with the same interests. Would you like to interact with them?"

[1331] Chat Toggle:

[1332] Once the customer approves, they can begin chatting with other users and exchange information about the cafe and recommended dishes.

[1333] Prompt for the generative AI model:

[1334] What is the most popular menu item at this cafe?

[1335] "Please tell me some recommended cafes you've visited recently."

[1336] This invention allows users to enjoy initial communication in a safe virtual environment and quickly and effectively match with other users who share common interests, improving the user experience in the virtual store and potentially bringing about commercial benefits.

[1337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1338] Step 1:

[1339] The user enters profile information and interest information.

[1340] Input: The user enters information into the device, including name, age, gender, hobbies, and interests.

[1341] How it works: The terminal parses the entered data and sends it to the server.

[1342] Output: The server stores the received data in a MySQL database.

[1343] Step 2:

[1344] The server generates an avatar using generation AI.

[1345] Input: User profile and interest information stored in a database.

[1346] How it works: The server feeds data into OpenAI's GPT-4 model and sends prompts to generate an avatar.

[1347] Output: The generated avatar information is returned and stored in a database.

[1348] Step 3:

[1349] A user enters a virtual store.

[1350] Input: The user selects a virtual store on the terminal and presses the enter button.

[1351] Operation: The terminal sends a store entry request to the server.

[1352] Output: The server creates a new chat room and places the user and the generated avatar in it.

[1353] Step 4:

[1354] Start an initial chat with your avatar.

[1355] Input: An initial message from the user (e.g., "What's the popular menu item at this cafe?").

[1356] How it works: The device sends a message to the server, which then asks the generating AI for a response via DialogFlow.

[1357] Output: The response from the generative AI (e.g., "This cafe's popular items are cafe lattes and cheesecake") is displayed on the user's device.

[1358] Step 5:

[1359] The server searches in the background for other users similar to the user.

[1360] Input: User profile information, interests, and chat content.

[1361] How it works: The server queries the database to find other users with common interests.

[1362] Output: A list of candidate matches is generated and stored in a queue.

[1363] Step 6:

[1364] If a match is made, a notification is sent to the user.

[1365] Input: A list of possible matches.

[1366] How it works: The server sends a notification to the user saying, "Similar customers found. Would you like to interact?"

[1367] Output: A notification is displayed on the user's device.

[1368] Step 7:

[1369] If the user accepts the notification, the chat partner is switched from the avatar to another user.

[1370] Input: The user's response ("Yes").

[1371] What it does: The server updates the current chat room and sets another real user as the new chat partner.

[1372] Output: The user switches chat partners and starts a new chat.

[1373] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1374] The present invention relates to a system that allows a user to start a chat with a generated avatar, searches for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. Furthermore, this system has an emotion engine built in, which can recognize the user's emotions and use them in chat responses and matching.

[1375] 1. User Registration

[1376] User: Opens the application and navigates to the registration screen. Enters name, profile information (age, gender, hobbies, etc.) and interests. Uploads a profile photo or asks for an avatar to be created by generative AI.

[1377] Terminal: Sends the information entered by the user to the server.

[1378] Server: The received user information is stored in a database, and if necessary, a generation AI is used to create an avatar based on the user information, and the data of the generated avatar is also stored in the database.

[1379] 2. Start a chat room

[1380] User: Presses the "Start Chat" button within the application.

[1381] Server: Create a new chat room and add the user and the generated avatar to the room.

[1382] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1383] 3. Chat Progression

[1384] User: Start a chat with an avatar, type a message and send it.

[1385] Terminal: Sends a message to the server.

[1386] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[1387] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[1388] 4. Proceeding with the Matching Process

[1389] Server: Runs a background process that searches for similar users based on the current chat content, user profile information, interests, and sentiment data. If a match is found, the user's information is added to the matching queue.

[1390] 5. User Notifications and Chat Switching

[1391] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1392] On your device: Display a notification and offer a choice (yes or no).

[1393] User: Check the notification and press the "Yes" button to approve the match.

[1394] Server: If approved, end the current chat session and start a session with the new user.

[1395] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1396] Terminal: Displays new chat room information to the user and starts a new chat.

[1397] Specific examples

[1398] 1. User Registration

[1399] User: For example, a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and then uploads a profile photo or chooses to have an avatar generated by AI.

[1400] Terminal: Sends input information to the server.

[1401] Server: Stores the received information in a database, generates and saves avatars.

[1402] 2. Start a chat room

[1403] User: Presses the "Start Chat" button.

[1404] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[1405] 3. Chat Progression

[1406] User: Sends a message to the avatar saying "Hi, I love movies."

[1407] Terminal: Sends a message to the server.

[1408] Server: Passes received messages to the generation AI and generates a response.

[1409] Server: Analyzes the user's emotions using an emotion engine and sends back a tone-adjusted response such as "The movie is great, what movie have you seen recently?"

[1410] 4. Proceeding with the Matching Process

[1411] Server: Search for other users who are interested in movies or sports based on chat content and user information.

[1412] 5. User Notifications and Chat Switching

[1413] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[1414] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[1415] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[1416] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[1417] The processing flow will be explained below.

[1418] Step 1:

[1419] User: Opens the application and navigates to the registration screen. Enters the required information (name, profile information, interests), and chooses to upload a profile photo or have a generative AI create an avatar for them.

[1420] Step 2:

[1421] Terminal: Sends the information entered by the user to the server.

[1422] Step 3:

[1423] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[1424] Step 4:

[1425] User: Presses the "Start Chat" button within the application.

[1426] Step 5:

[1427] Server: Create a new chat room and add the user and the generated avatar to the room.

[1428] Step 6:

[1429] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1430] Step 7:

[1431] User: Start a chat with an avatar. Type and send a message.

[1432] Step 8:

[1433] Terminal: Sends a message to the server.

[1434] Step 9:

[1435] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[1436] Step 10:

[1437] Terminal: Receives response messages from the server and displays them to the user.

[1438] Step 11:

[1439] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[1440] Step 12:

[1441] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[1442] Step 13:

[1443] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1444] Step 14:

[1445] On your device: Show a notification and offer a "yes" or "no" option.

[1446] Step 15:

[1447] User: Check the notification and press the "Yes" button if you approve the match.

[1448] Step 16:

[1449] Server: If approved, end the current chat session and start a session with the new user.

[1450] Step 17:

[1451] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1452] Step 18:

[1453] Terminal: Displays new chat room information to the user and starts a new chat.

[1454] Step 19:

[1455] User: Continue chatting with a new user.

[1456] Example 2

[1457] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1458] Conventional online chat systems lack a means for users to communicate with other users with confidence and an efficient way to match users with common interests. Furthermore, they face challenges in generating responses that take users' emotions into account and smoothly switching between chat partners. To address these challenges, a system is needed that recognizes users' emotions to provide more natural dialogue and matches users with common interests in the background.

[1459] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1460] In this invention, the server includes means for users to input profile information and interest information, means for collecting and saving user information, means for placing a generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when another user is found, means for switching the chat partner from the avatar to another user if the user approves, means for recognizing emotions from the user's input message, facial expression, and voice data, and means for adjusting the content and tone of the chat message responses generated by the generation AI based on the recognized emotions. This not only allows users to enjoy natural conversations with other users with peace of mind, but also enables effective matching with other users who share common interests.

[1461] "User" refers to an individual or corporation that uses the system and provides profile information and interest information.

[1462] "Profile information" is basic information about a user, including data such as name, age, sex, address, and occupation.

[1463] "Interest information" is information about activities and topics in which a user is interested, including hobbies and favorite things.

[1464] "Generative AI" refers to artificial intelligence that automatically generates content such as text and images using generative models.

[1465] "Avatar" refers to a digital image or character that represents a user in a virtual world.

[1466] "Chat room" refers to a virtual room for exchanging messages online.

[1467] "Server" refers to the central computer of the system, a device that manages data storage, processing, and communication with other devices and terminals.

[1468] "Background" refers to processing that the system executes automatically, independent of user operations.

[1469] An "emotion engine" refers to a program and system that analyzes and recognizes emotions from a user's messages, voice, facial expressions, etc.

[1470] "Notification" refers to the act of the system sending a message or information to the user.

[1471] "Chat message" refers to text or audio information exchanged between users in a chat room.

[1472] "Matching" refers to the process of connecting users with common interests.

[1473] "Response content" refers to the text of the response that the generation AI generates in response to the user's message.

[1474] "Tone" refers to the style and expression of the chat messages created by the generative AI.

[1475] A "chat partner" refers to another user or avatar with whom a user communicates in a chat room.

[1476] This invention is a system in which a user starts a chat with a generated avatar, searches for similar users in the background, and if a match is successful, switches the chat partner from the avatar to a real person. The system incorporates an emotion engine that can recognize the user's emotions and use them in chat responses and matching. A server, a terminal, a generative AI model, and an emotion engine are required to implement this system.

[1477] System Hardware and Software

[1478] The hardware required to implement the system includes smartphones, tablets, and PCs. Python and Node.js are used for server-side programming. Databases such as MySQL and PostgreSQL are used to store data. GPT-3 and other technologies are used for generative AI, and dedicated emotion analysis software is used as the emotion recognition engine.

[1479] Specific explanation of the process

[1480] 1. User Registration

[1481] Users open the application and enter their name, profile information (age, gender, hobbies, etc.) and interests on the registration screen. They can then upload a profile photo or choose to have their avatar created by generative AI.

[1482] The device converts the information entered by the user into JSON format and sends it to the server via an HTTP POST request.

[1483] The server stores the received user information in a database. If there is a request to create an avatar, it sends an API request to the generation AI and stores the generated avatar data in the database.

[1484] 2. Start a chat room

[1485] The user presses the "Start Chat" button within the application.

[1486] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[1487] 3. Chat Progression

[1488] Users can start chatting with their avatar, typing and sending messages, such as "Hi, I love movies."

[1489] The device sends messages in JSON format to the server.

[1490] The server passes the received message to the generation AI, which generates an appropriate response, sends the generated response back to the user, and displays a response such as "That movie is great. What movie have you seen recently?" on the device.

[1491] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[1492] 4. Proceeding with the Matching Process

[1493] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data. If a match is found, the user's information is added to the matching queue.

[1494] 5. User Notifications and Chat Switching

[1495] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[1496] The device will display a notification and offer a choice ("Yes" or "No").

[1497] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[1498] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[1499] The terminal displays the information about the new chat room to the user and starts a new chat.

[1500] Specific examples

[1501] 1. User Registration

[1502] User: Enter "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports" and choose to upload a profile photo or have an avatar generated by AI.

[1503] Terminal: Sends input information to the server.

[1504] Server: Stores the received information in a database, generates and saves avatars.

[1505] 2. Start a chat room

[1506] User: Presses the "Start Chat" button.

[1507] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[1508] 3. Chat Progression

[1509] User: Sends a message saying "Hi, I love movies."

[1510] Terminal: Sends a message to the server.

[1511] Server: Passes the received message to the generation AI and generates a response. It responds with "That movie is great, what movie have you seen recently?"

[1512] 4. Proceeding with the Matching Process

[1513] Server: Searches for other users who are interested in movies or sports based on chat content and user information.

[1514] 5. User Notifications and Chat Switching

[1515] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[1516] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[1517] Server: Create a new chat room and either carry over existing chat history or start a new session.

[1518] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[1519] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1520] Step 1: User Registration

[1521] Users open the application, enter their name, profile information (age, gender, hobbies, etc.), and interests, and choose to upload a profile photo or have a generative AI create an avatar for them.

[1522] Specific actions: Enter your name ("Yamada Taro") and hobbies ("watching movies and sports"), then select and upload a profile picture or tap the "Generate Avatar" button.

[1523] The device converts the entered information into JSON format and sends it to the server via an HTTP POST request.

[1524] Input: Name, Profile Information, Interests, Profile Picture.

[1525] Output: User information in JSON format.

[1526] Specific operation: Enter data into the input form, format it into JSON, and send it to the endpoint " / register".

[1527] The server receives user information in JSON format and stores it in a database. When a request to create an avatar is received, it sends an API request to the AI ​​generator and stores the generated avatar image in the database.

[1528] Input: JSON formatted user information sent from the terminal.

[1529] Output: User information stored in the database, generated avatar.

[1530] Specific operation: The received information is inserted into the database, a request is sent to the generation AI, and the generated avatar data is saved back into the database.

[1531] Step 2: Start a chat room

[1532] The user presses the "Start Chat" button within the application.

[1533] Specific actions: Press the "Start chat" button to begin the process.

[1534] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[1535] Input: User's request to start a chat room.

[1536] Output: New chat room, background search process.

[1537] Specific actions: Generate a new chat room ID, add the user and avatar to the chat room, and start searching for similar users.

[1538] Step 3: Chat Progression

[1539] Users can start chatting with their avatar, typing and sending messages such as "Hi, I love movies."

[1540] Specific actions: Enter information into the text box and click the send button.

[1541] The device sends messages in JSON format to the server.

[1542] Input: The message entered by the user.

[1543] Output: Message data in JSON format.

[1544] Specific operation: Convert the input message into JSON and POST it to the endpoint " / send_message".

[1545] The server passes the received message to the generation AI, which generates an appropriate response, which is then sent back to the user and displayed on the screen.

[1546] Input: Message received from the terminal.

[1547] Output: Response message from the generating AI.

[1548] Specific operation: The received message is passed to the generation AI, which generates a response saying, "The movie is great. What movie have you seen recently?" and sends it back to the device and displays it on the screen.

[1549] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[1550] Input: User message, emotion data.

[1551] Output: The adjusted response message.

[1552] Specific operation: Emotion analysis is performed using the emotion engine, and the generative AI creates a tone-adjusted response.

[1553] Step 4: Proceed with the matching process

[1554] The server runs a background process to search for similar users based on chat content, user profile information, interests, and emotional data.

[1555] Input: User chat content, profile information, interest data, and emotional data.

[1556] Output: Potential matching users.

[1557] What it does: It uses a scoring algorithm to query the database to find similar users and adds potential matches to a queue.

[1558] Step 5: Notify users and switch chat partners

[1559] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[1560] Input: The result of the matching process.

[1561] Output: Informational message.

[1562] Specific behavior: If a new match is found, create a notification message and send it to the user.

[1563] The device will display a notification and offer a choice ("Yes" or "No").

[1564] Input: Notification message from the server.

[1565] Output: The displayed notification and selection options.

[1566] Specific operation: Implement a mechanism to display a pop-up screen and provide the user with options.

[1567] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[1568] Specific action: Tap the "Yes" button to approve.

[1569] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[1570] Input: Acknowledgement from the user.

[1571] Output: New chat session, new chat room.

[1572] Specific actions: Close the old chat room, generate a new chat room ID, and carry over the history if necessary.

[1573] The terminal displays the information about the new chat room to the user and starts a new chat.

[1574] Input: New chat room information.

[1575] Output: Updated chat screen.

[1576] Specific operation: The new chat room information is reflected on the screen and displayed to the user.

[1577] (Application example 2)

[1578] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1579] Conventional chat systems lack the means to safely and effectively match users with other users. They also struggle to provide a natural, emotion-based chat experience, and are unable to recommend products and services that match the user's interests. The objective of this invention is to provide a system that improves the user experience, enables more natural and personal communication, and recommends appropriate products and services.

[1580] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input profile information and interest information; a means for collecting and saving the user information; a means for placing a generated avatar in a chat room and connecting with the user; a means for searching for other users similar to the user in the background; a means for notifying the user when the other user is found; a means for switching the chat partner from the avatar to the other user if the user approves; a means for analyzing the user's emotions and adjusting the tone of chat messages based on the emotion data; and a means for analyzing the user's input text in real time and recommending appropriate products or services. This allows users to be safely and effectively matched with other users, enjoy a natural chat experience based on their emotions, and receive recommendations for products and services that match their interests.

[1581] "User profile" refers to personal information entered by a user, such as name, age, gender, hobbies, and interests.

[1582] "Generated avatar" refers to a virtual character created by the generation AI based on the user's profile information.

[1583] "Chat room" refers to a virtual space where a user, an avatar, or other users can exchange messages online.

[1584] An "emotion engine" refers to software that recognizes emotions from the user's input text, voice, facial expressions, etc., and uses the results.

[1585] "Searching in the background" refers to the process by which the system automatically searches for other similar users without any explicit user interaction.

[1586] "Matching" refers to the process of finding other users with whom you have common ground and compatibility based on your profile information, interests, and emotional data.

[1587] "Notification" refers to a message that the system displays to inform the user.

[1588] "Adjusting the tone of a chat message" refers to changing the tone and expression of a generated message based on the user's emotional data.

[1589] "Recommending products or services" refers to the system suggesting related products or services based on the user's input text and sentiment.

[1590] "Real-time" refers to immediate response or processing of user input.

[1591] The embodiment of the present invention is based on building a system in which a user starts a chat with a generated avatar, while searching for other users similar to the user in the background, and switching the chat partner from the avatar to a real person if a match is successful, according to the following steps:

[1592] 1. User Registration

[1593] The user opens the application, goes to the registration screen, and enters their name, profile information (age, gender, hobbies, etc.), and interests. They can then upload a profile photo or request that the AI ​​create an avatar for them. The device sends the information the user entered to the server, which stores the received user information in a database and, if necessary, uses the AI ​​to create an avatar based on the user information, storing the generated avatar data in the database.

[1594] 2. Start a chat room

[1595] The user presses a "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[1596] 3. Chat Progression

[1597] The user starts a chat with an avatar, inputs a message, and sends it. The device sends the message to the server, which passes the received message to the generation AI and generates an appropriate response. The generated response message is returned to the user and displayed on the device. The server also uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data. Based on the recognized emotions, the server adjusts the content and tone of the generation AI's response to adapt to the user, providing a more natural chat experience.

[1598] 4. Proceeding with the Matching Process

[1599] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[1600] 5. User Notifications and Chat Switching

[1601] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and provides a choice ("Yes" or "No"). The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays the new chat room information to the user and starts a new chat.

[1602] Hardware and software used

[1603] Generative AI models (e.g., GPT-type language models)

[1604] Emotion engine (e.g., emotion analysis AI)

[1605] Server (e.g. cloud server)

[1606] Devices (e.g. smartphones, head-mounted displays)

[1607] Specific examples

[1608] Consider a scenario where a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and either uploads a profile picture or selects to have an avatar generated by a generative AI. In this case, the user presses the "Start Chat" button to start a chat room and sends the message "Hello, I love movies" to the avatar. The server passes the received message to the generative AI, which generates a response. For example, an emotion engine could analyze the user's emotions and return a tone-adjusted response such as "Movies are great. What movies have you seen recently?"

[1609] Prompt Sentence Examples

[1610] Input sentence: "I've been liking movies lately"

[1611] Prompt: "The user is talking about movies. Analyze the given input for emotions and respond in a friendly manner."

[1612] AI Response: "That's great! What movie did you see recently?"

[1613] As described above, this system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar. Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[1614] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1615] Step 1: User Registration

[1616] The user opens the application, enters their name, profile information (age, gender, hobbies, etc.) and interests, and either uploads a profile photo or requests that the AI ​​create an avatar. The device sends the entered information to the server. The server stores the received user information in a database, uses the AI ​​to create an avatar based on the user information, and also stores the data of the generated avatar in the database.

[1617] Input: Name, profile information, interests, profile picture or avatar creation request

[1618] Output: Generated avatar data

[1619] Specific behavior:

[1620] User enters information via application

[1621] Send information from the device to the server

[1622] The server stores the information in a database and generates an avatar using a generative AI.

[1623] Step 2: Start a chat room

[1624] The user presses the "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[1625] Input: Press the "Start Chat" button

[1626] Output: Creates a new chat room and populates it with users and avatars.

[1627] Specific behavior:

[1628] User presses "Start Chat" button

[1629] The server creates a new chat room and places users and avatars in it.

[1630] Start searching for other similar users in the background

[1631] Step 3: Chat Progression

[1632] The user starts a chat with the avatar, inputs a message, and sends it. The device sends the message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device. The server uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data, and adjusts the content and tone of the generation AI's response based on the recognized emotion.

[1633] Input: User input message

[1634] Output: The generated response message

[1635] Specific behavior:

[1636] User types and sends message

[1637] The device sends a message to the server

[1638] The server passes the message to the generation AI and generates a response.

[1639] A response message is returned to the user and displayed on the terminal.

[1640] Emotion engine recognizes user emotions and adjusts response tone

[1641] Step 4: Proceed with the matching process

[1642] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[1643] Input: Current chat content, profile information, interests, emotional data

[1644] Output: User information of match candidates

[1645] Specific behavior:

[1646] The server analyzes chat content and user information in the background

[1647] Find other similar users

[1648] If a match is found, add the information to the match queue

[1649] Step 5: Notify users and switch chat partners

[1650] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and offers the option of "Yes" or "No." The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays information about the new chat room to the user and starts a new chat.

[1651] Input: User information of match candidates, user approval

[1652] Output: Create a new chat room and switch chat partners

[1653] Specific behavior:

[1654] The server notifies the user when a match is successful

[1655] Your device will display a notification and offer you the option to choose "Yes" or "No."

[1656] User selects "Yes"

[1657] The server ends the chat session and starts a new session with the user.

[1658] Create a new chat room and take over the information

[1659] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1660] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1661] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1662] [Fourth embodiment]

[1663] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1664] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1665] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1666] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1667] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1668] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1669] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1670] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1671] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1672] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1673] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1674] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1675] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1676] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently match with other users who share common interests.

[1677] 1. User Registration

[1678] Users: To register with the application, they enter their name, profile information (age, gender, hobbies, etc.) and interests. They can also upload a profile photo or ask the generative AI to create an avatar for them.

[1679] Terminal: Sends the information entered by the user to the server.

[1680] Server: Stores the received user information in a database, and if necessary, uses a generation AI to create an avatar based on the user information. The generated avatar data is also stored in the database.

[1681] 2. Start a chat room

[1682] User: Presses the "Start Chat" button in the application.

[1683] Server: Creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to find other users similar to the user based on relevant information.

[1684] 3. Chat Progression

[1685] User: Start a chat with an avatar, type a message and send it.

[1686] Terminal: Sends a message to the server.

[1687] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[1688] 4. Proceeding with the Matching Process

[1689] Server: Runs a background process to search for similar users based on the current chat, the user's profile information, and interests. If a match is found, stores the user's information in a queue.

[1690] 5. User Notifications and Chat Switching

[1691] Server: If a match is successful, a notification is sent to the user, informing them of the other user's information.

[1692] On your device: Show a notification and offer a "yes" or "no" option.

[1693] User: If you approve the match, press the "Yes" button.

[1694] Server: If approved, terminate the current chat session and connect the new user, create a new chat room and transfer the existing chat history to the new user, or start a new session.

[1695] Specific examples

[1696] 1. User Registration

[1697] User: For example, a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or chooses to have an avatar generated by AI.

[1698] Terminal: Sends input information to the server.

[1699] Server: Stores the received information in a database, generates and saves avatars.

[1700] 2. Start a chat room

[1701] User: Presses the "Start Chat" button.

[1702] Server: Create a new chat room, add Hanako Yamada and the generated avatar to the room, and start the matching process in the background.

[1703] 3. Chat Progression

[1704] User: Sends a message to the avatar saying "Hi, I love reading."

[1705] Terminal: Sends a message to the server.

[1706] Server: Passes the received message to the generation AI, generates a response, and sends back "Reading is great, what book have you read recently?"

[1707] 4. Proceeding with the Matching Process

[1708] Server: Search for other users who are interested in reading or traveling based on chat content and user information.

[1709] 5. User Notifications and Chat Switching

[1710] Server: If the match is successful, notify Hanako Yamada, "Similar users found. Do you want to continue?"

[1711] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[1712] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[1713] This system allows users to first engage in safe and enjoyable communication with their generated avatars, and then effectively match with other real users who share common interests.

[1714] The processing flow will be explained below.

[1715] Step 1:

[1716] User: Opens the application and navigates to the registration screen. Fills in the required information (username, profile, interests, etc.). Uploads a profile photo or selects the option to create an avatar using generative AI.

[1717] Step 2:

[1718] Terminal: Sends the information entered by the user to the server.

[1719] Step 3:

[1720] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[1721] Step 4:

[1722] User: Presses the "Start Chat" button within the application.

[1723] Step 5:

[1724] Server: Create a new chat room, get the user's avatar information from the database, and add the avatar to the chat room.

[1725] Step 6:

[1726] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1727] Step 7:

[1728] User: Start a chat with an avatar. Type your message and press the "Send" button.

[1729] Step 8:

[1730] Terminal: Sends messages typed by the user to the server.

[1731] Step 9:

[1732] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[1733] Step 10:

[1734] Terminal: Receives response messages from the server and displays them to the user.

[1735] Step 11:

[1736] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[1737] Step 12:

[1738] Server: If the match is successful, notify the user. Send a message saying "We've found a user similar to you. Do you want to continue?"

[1739] Step 13:

[1740] On your device: Display a notification and offer a choice (yes or no).

[1741] Step 14:

[1742] User: Check the notification and press the "Yes" button to approve the match.

[1743] Step 15:

[1744] Server: If approved, end the current chat session and start a session with the new user.

[1745] Step 16:

[1746] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1747] Step 17:

[1748] Terminal: Displays new chat room information to the user and starts a new chat.

[1749] Step 18:

[1750] User: Continue chatting with a new user.

[1751] Example 1

[1752] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1753] In conventional online chat systems, users interact with other real users from the beginning, which can cause tension and anxiety when meeting for the first time. Furthermore, there is a problem that the accuracy of matching users with similar interests is low, making efficient matching difficult. This has led to a need for a system that efficiently matches users with common hobbies and interests so that they can begin communicating with ease.

[1754] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1755] In this invention, the server includes means for a user to input profile information and interest information, means for transmitting the user information to the server, means for storing the received user information in a database and creating an avatar using a generation AI, means for placing the generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when the other user is found, means for switching the chat partner from the avatar to the other user if the user approves, means for generating responses to chat messages in real time using the generated avatar, and means for searching for similar users based on the user's profile information, interest information, and chat content. This allows users to have initial chats in a safe and relaxed state and enables efficient matching with other users who share common interests.

[1756] A "user" is an individual who uses the system to chat and match.

[1757] "Profile information" refers to personal information such as a user's name, age, gender, and hobbies.

[1758] "Interest information" refers to information about activities and topics that interest a user.

[1759] "Server" refers to the computer system that processes information sent by users, stores it in a database, and creates avatars using generative AI.

[1760] "Generative AI" refers to artificial intelligence that automatically generates an avatar based on input information.

[1761] An "avatar" refers to a virtual character that participates in chats on behalf of a user.

[1762] A "chat room" refers to an online space for users and avatars to interact.

[1763] A "background process" refers to work that is carried out in parallel and invisible to the user.

[1764] "Matching" refers to the process of identifying and connecting with other users who share common hobbies and interests.

[1765] "Means for notifying" refers to a function for conveying messages or information to the user.

[1766] "Real-time" refers to a situation where a user sends a message and receives an immediate response.

[1767] "Chat content" refers to messages and conversations exchanged between a user and an avatar or other users.

[1768] The present invention relates to a system that allows a user to start a chat with a generated avatar, while searching for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. This system allows users to feel comfortable in the initial communication and efficiently matches with other users who share common interests.

[1769] User Registration

[1770] The user launches the application and enters their name, profile information (age, gender, hobbies), and interests. They can then upload a profile photo or request the generation AI to create an avatar. The device sends the information entered by the user to the server. The server stores the received user information in a database and, if necessary, uses the generation AI to create an avatar based on the user information. The data of the generated avatar is also stored in the database.

[1771] For example, if a user enters "Name: Yamada Hanako" and "Hobbies: Reading, Traveling" and then uploads a profile picture or asks the AI ​​to create an avatar, the device sends the input information to the server, which then stores the information and generates the avatar.

[1772] Starting a Chat Room

[1773] The user presses the "Start Chat" button in the application. The server creates a new chat room, adds the user and the generated avatar to the room, and starts a background process to search for other users similar to the user based on relevant information.

[1774] Chat progress

[1775] The user starts a chat with the generated avatar, inputs a message, and sends it. The device sends the user's message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device.

[1776] For example, if a user sends a message to an avatar saying, "Hello, I love reading," the device sends the message to the server, which passes it on to the generation AI, which generates a response saying, "Reading is great, what book have you read recently?"

[1777] Progressing through the matching process

[1778] The server runs a background process to search for similar users based on the current chat content, the user's profile information, and their interests. If a match is found, the user's information is stored in a queue.

[1779] User notifications and chat switching

[1780] If the match is successful, the server will notify the user and provide information about the other user. The device will display the notification and provide a "Yes / No" option. The user can press the "Yes" button to approve the match. If approval is granted, the server will end the current chat session and connect with the new user, creating a new chat room and either inheriting the existing chat history or starting a new session.

[1781] For example, after a successful match, the server notifies Hanako Yamada, "Similar users found. Do you want to continue?" and the device displays a "Yes / No" option. If the user selects "Yes," the chat partner will switch to the new user.

[1782] Example of input prompt for generative AI model

[1783] An example of a prompt is, "My name is Hanako Yamada and my hobbies are reading and traveling. Please tell me about some recent books you recommend." In response to this prompt, the generative AI model generates a response such as, "Reading is a great hobby! Of the books you've read recently, the one that made the biggest impression on you was 'XXX'. What books have you read recently?"

[1784] The system allows users to have initial chats in a safe and relaxed environment, and efficiently matches them with other users who share common interests.

[1785] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1786] Step 1:

[1787] Users: Enter their profile information (name, age, gender, hobbies) and interests, and upload a profile photo or ask the generative AI to create an avatar for them.

[1788] Input: Information entered by a user into an input form.

[1789] Output: When the input form is completed and the submit button is pressed, the data is sent to the device.

[1790] Specific actions: Enter the required information in the form and press the "Submit" button.

[1791] Step 2:

[1792] Terminal: Sends the information entered by the user to the server.

[1793] Input: Request data, including information entered by the user into an input form.

[1794] Output: Sends a POST request to the server in JSON format.

[1795] Specific operation: Converts form data into JSON format and sends it to the server via an HTTP request.

[1796] Step 3:

[1797] Server: Stores the received user information in a database and creates an avatar using a generation AI.

[1798] Input: Profile and interest information submitted by you.

[1799] Output: The new user information and the generated avatar information are saved in the database.

[1800] Specific operations: Parse and validate the incoming data, save the profile information to the database, call the generation AI if necessary to generate an avatar, and save the data.

[1801] Step 4:

[1802] User: Presses the "Start Chat" button in the application.

[1803] Input: "Start Chat" button click event.

[1804] Output: A chat room creation request is sent to the server.

[1805] Specific actions: Click the "Start Chat" button on the application interface.

[1806] Step 5:

[1807] Server: Create a new chat room, add the generated avatar to the room, and start searching for similar users in the background.

[1808] Input: Chat room creation request, user and avatar information.

[1809] Output: A new chat room is created, with a user and avatar added.

[1810] What it does: Creates a new chat room record in the database and launches a background job to start the matching process.

[1811] Step 6:

[1812] User: Start a chat with an avatar, type a message and send it.

[1813] Input: Messages typed in the chat interface.

[1814] Output: The message is sent from the terminal to the server.

[1815] Specific action: Enter text in the chat interface and press the "Send" button.

[1816] Step 7:

[1817] Terminal: Sends a message to the server.

[1818] Input: Message data entered by the user.

[1819] Output: A request containing the message data is sent to the server.

[1820] Specific behavior: Sends message data in JSON format to the server.

[1821] Step 8:

[1822] Server: Passes the received message to the generative AI model, generates an appropriate response, and returns the generated response message to the user, displaying it on the device.

[1823] Input: The message data sent by the user.

[1824] Output: A response message from the generative AI model is generated and sent back to the user.

[1825] Specific operations: Receive and parse message data, call the generation AI, generate a response message, and return the response to the terminal in JSON format.

[1826] Step 9:

[1827] Server: Runs a background process to find similar users based on chat content, user profile information, and interests.

[1828] Input: User profile information, interests, chat content.

[1829] Output: User information of potential matches is stored in a queue.

[1830] What it does: Searches the database to evaluate potential matches and adds them to the queue.

[1831] Step 10:

[1832] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1833] Input: User information of potential matches.

[1834] Output: A notification is sent to the user.

[1835] What it does: Sends notifications via email and in-app notifications.

[1836] Step 11:

[1837] On your device: Show a notification and offer a "yes" or "no" option.

[1838] Input: Notification data from the server.

[1839] Output: A "Yes / No" option is displayed in the user interface.

[1840] Specific behavior: Display a notification message on the interface and display a selection button.

[1841] Step 12:

[1842] User: If you approve the match, press the "Yes" button.

[1843] Input: "Yes" button click event.

[1844] Output: The authorization information is sent to the server.

[1845] Specific action: Click the "Yes" button.

[1846] Step 13:

[1847] Server: If approved, terminate the current chat session and connect the new user. Create a new chat room and continue the existing chat history or start a new session.

[1848] Input: Authorization information, new chat partner's user information, existing chat history.

[1849] Output: A new chat room is created and the user is connected to the new chat partner.

[1850] Specific actions: Create a new chat room, update the database, carry over chat history or start a new session.

[1851] (Application example 1)

[1852] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1853] In modern society, people are increasingly enjoying interactions in virtual environments. However, many users feel uneasy about communicating directly with others they meet for the first time, making it difficult to establish initial connections. Furthermore, in commercial virtual stores, users are not effectively matched, resulting in inactive interactions and limiting the full potential of virtual stores.

[1854] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1855] In this invention, the server includes a means for users to input profile information and interest information, a means for collecting and saving the user information, and a means for placing the generated avatar in a chat room and connecting with the user. This allows users to start communication with others for the first time using the indirect means of the avatar, allowing users to engage in conversation with peace of mind. The server also includes a means for matching users within the virtual store, allowing users with common interests to quickly meet and deepen their interactions. This further enhances the experience in the virtual store and is expected to have a commercial impact due to an increase in users.

[1856] A "user" is a person who enters profile information and interest information into the system and interacts within the virtual store.

[1857] "Profile information" is detailed personal information such as a user's name, age, gender, hobbies, interests, etc.

[1858] "Interest information" is information about areas or activities in which a user is particularly interested.

[1859] An "avatar" is a virtual 3D character that represents a user and is created by a generative AI based on the user's profile information.

[1860] A "chat room" is a virtual space where users and avatars can communicate in real time using text and voice.

[1861] A "virtual store" is a virtual commercial facility that users can access in a virtual space, and is a place that offers a variety of products and services.

[1862] "Matching" is the process by which the system identifies other users who share common hobbies and interests with the user and encourages interaction.

[1863] "Generative AI" is an artificial intelligence system that generates avatars based on user information and generates real-time responses to text messages.

[1864] The system for implementing this invention allows users to input profile information and interest information, and then uses a generative AI model to generate an avatar based on that information, enabling users to safely and effectively match and interact with other users within a virtual store.

[1865] The server is built using Node.js and stores user profile information and interest information in a MySQL database. Avatars are generated using OpenAI's GPT-4 model, and the generated avatars are placed in chat rooms on behalf of the users.

[1866] When a user enters a virtual store, they first start chatting with an avatar. The initial communication is conducted in real time using DialogFlow, and the avatar responds to the user's questions and messages.

[1867] Meanwhile, the server searches for similar users in the background based on other users' profile information, interests, and chat content. If the search process is successful and a match is made, the server sends a notification to the user saying, "Similar users have been found. Would you like to interact?" If the user accepts, the chat partner will switch from the avatar to the matched real user.

[1868] A concrete example of this within a virtual store could be the following scenario:

[1869] Examples:

[1870] User Registration:

[1871] Customers enter information such as "Hobbies: music, cooking" and select to have an avatar generated by AI.

[1872] Entering the Virtual Store:

[1873] A customer enters a virtual cafe and begins a conversation with an avatar.

[1874] Chat content:

[1875] What is the most popular menu item at this cafe?

[1876] Matching Notification:

[1877] The server sends a notification saying, "There are users with the same interests. Would you like to interact with them?"

[1878] Chat Toggle:

[1879] Once the customer approves, they can begin chatting with other users and exchange information about the cafe and recommended dishes.

[1880] Prompt for the generative AI model:

[1881] What is the most popular menu item at this cafe?

[1882] "Please tell me some recommended cafes you've visited recently."

[1883] This invention allows users to enjoy initial communication in a safe virtual environment and quickly and effectively match with other users who share common interests, improving the user experience in the virtual store and potentially bringing about commercial benefits.

[1884] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1885] Step 1:

[1886] The user enters profile information and interest information.

[1887] Input: The user enters information into the device, including name, age, gender, hobbies, and interests.

[1888] How it works: The terminal parses the entered data and sends it to the server.

[1889] Output: The server stores the received data in a MySQL database.

[1890] Step 2:

[1891] The server generates an avatar using generation AI.

[1892] Input: User profile and interest information stored in a database.

[1893] How it works: The server feeds data into OpenAI's GPT-4 model and sends prompts to generate an avatar.

[1894] Output: The generated avatar information is returned and stored in a database.

[1895] Step 3:

[1896] A user enters a virtual store.

[1897] Input: The user selects a virtual store on the terminal and presses the enter button.

[1898] Operation: The terminal sends a store entry request to the server.

[1899] Output: The server creates a new chat room and places the user and the generated avatar in it.

[1900] Step 4:

[1901] Start an initial chat with your avatar.

[1902] Input: An initial message from the user (e.g., "What's the popular menu item at this cafe?").

[1903] How it works: The device sends a message to the server, which then asks the generating AI for a response via DialogFlow.

[1904] Output: The response from the generative AI (e.g., "This cafe's popular items are cafe lattes and cheesecake") is displayed on the user's device.

[1905] Step 5:

[1906] The server searches in the background for other users similar to the user.

[1907] Input: User profile information, interests, and chat content.

[1908] How it works: The server queries the database to find other users with common interests.

[1909] Output: A list of candidate matches is generated and stored in a queue.

[1910] Step 6:

[1911] If a match is made, a notification is sent to the user.

[1912] Input: A list of possible matches.

[1913] How it works: The server sends a notification to the user saying, "Similar customers found. Would you like to interact?"

[1914] Output: A notification is displayed on the user's device.

[1915] Step 7:

[1916] If the user accepts the notification, the chat partner is switched from the avatar to another user.

[1917] Input: The user's response ("Yes").

[1918] What it does: The server updates the current chat room and sets another real user as the new chat partner.

[1919] Output: The user switches chat partners and starts a new chat.

[1920] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1921] The present invention relates to a system that allows a user to start a chat with a generated avatar, searches for other users similar to the user in the background, and if a match is successful, switches the chat partner from the avatar to a real person. Furthermore, this system has an emotion engine built in, which can recognize the user's emotions and use them in chat responses and matching.

[1922] 1. User Registration

[1923] User: Opens the application and navigates to the registration screen. Enters name, profile information (age, gender, hobbies, etc.) and interests. Uploads a profile photo or asks for an avatar to be created by generative AI.

[1924] Terminal: Sends the information entered by the user to the server.

[1925] Server: The received user information is stored in a database, and if necessary, a generation AI is used to create an avatar based on the user information, and the data of the generated avatar is also stored in the database.

[1926] 2. Start a chat room

[1927] User: Presses the "Start Chat" button within the application.

[1928] Server: Create a new chat room and add the user and the generated avatar to the room.

[1929] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1930] 3. Chat Progression

[1931] User: Start a chat with an avatar, type a message and send it.

[1932] Terminal: Sends a message to the server.

[1933] Server: Passes the received message to the generation AI, generates an appropriate response, and returns the generated response message to the user, displaying it on the terminal.

[1934] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[1935] 4. Proceeding with the Matching Process

[1936] Server: Runs a background process that searches for similar users based on the current chat content, user profile information, interests, and sentiment data. If a match is found, the user's information is added to the matching queue.

[1937] 5. User Notifications and Chat Switching

[1938] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1939] On your device: Display a notification and offer a choice (yes or no).

[1940] User: Check the notification and press the "Yes" button to approve the match.

[1941] Server: If approved, end the current chat session and start a session with the new user.

[1942] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1943] Terminal: Displays new chat room information to the user and starts a new chat.

[1944] Specific examples

[1945] 1. User Registration

[1946] User: For example, a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and then uploads a profile photo or chooses to have an avatar generated by AI.

[1947] Terminal: Sends input information to the server.

[1948] Server: Stores the received information in a database, generates and saves avatars.

[1949] 2. Start a chat room

[1950] User: Presses the "Start Chat" button.

[1951] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[1952] 3. Chat Progression

[1953] User: Sends a message to the avatar saying "Hi, I love movies."

[1954] Terminal: Sends a message to the server.

[1955] Server: Passes received messages to the generation AI and generates a response.

[1956] Server: Analyzes the user's emotions using an emotion engine and sends back a tone-adjusted response such as "The movie is great, what movie have you seen recently?"

[1957] 4. Proceeding with the Matching Process

[1958] Server: Search for other users who are interested in movies or sports based on chat content and user information.

[1959] 5. User Notifications and Chat Switching

[1960] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[1961] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[1962] Server: Create a new chat room, either inheriting existing chat history or starting a new session.

[1963] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[1964] The processing flow will be explained below.

[1965] Step 1:

[1966] User: Opens the application and navigates to the registration screen. Enters the required information (name, profile information, interests), and chooses to upload a profile photo or have a generative AI create an avatar for them.

[1967] Step 2:

[1968] Terminal: Sends the information entered by the user to the server.

[1969] Step 3:

[1970] Server: Stores the received user information in a database. If a profile photo is provided, it is stored. If avatar generation is selected, it calls the generation AI to generate an avatar. The generated avatar is also stored in a database.

[1971] Step 4:

[1972] User: Presses the "Start Chat" button within the application.

[1973] Step 5:

[1974] Server: Create a new chat room and add the user and the generated avatar to the room.

[1975] Step 6:

[1976] Server: In the background, it starts the process of finding other users similar to the user. It analyzes the user's profile information, interests, and chat content and compares them with other users in its database.

[1977] Step 7:

[1978] User: Start a chat with an avatar. Type and send a message.

[1979] Step 8:

[1980] Terminal: Sends a message to the server.

[1981] Step 9:

[1982] Server: Passes the received message to the generation AI, generates an appropriate response, and sends the generated response message back to the user.

[1983] Step 10:

[1984] Terminal: Receives response messages from the server and displays them to the user.

[1985] Step 11:

[1986] Server: Uses an emotion engine to recognize emotions from user input messages, facial expressions, and voice data. Based on the recognized emotions, the generative AI adjusts the content and tone of responses to adapt to the user, providing a more natural chat experience.

[1987] Step 12:

[1988] Server: Continues the matching process in the background. If other similar users are found, adds them to the matching queue.

[1989] Step 13:

[1990] Server: If a match is successful, the server notifies the user and provides the other user's information.

[1991] Step 14:

[1992] On your device: Show a notification and offer a "yes" or "no" option.

[1993] Step 15:

[1994] User: Check the notification and press the "Yes" button if you approve the match.

[1995] Step 16:

[1996] Server: If approved, end the current chat session and start a session with the new user.

[1997] Step 17:

[1998] Server: Creates a new chat room and optionally transfers existing chat history to the new user.

[1999] Step 18:

[2000] Terminal: Displays new chat room information to the user and starts a new chat.

[2001] Step 19:

[2002] User: Continue chatting with a new user.

[2003] Example 2

[2004] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2005] Conventional online chat systems lack a means for users to communicate with other users with confidence and an efficient way to match users with common interests. Furthermore, they face challenges in generating responses that take users' emotions into account and smoothly switching between chat partners. To address these challenges, a system is needed that recognizes users' emotions to provide more natural dialogue and matches users with common interests in the background.

[2006] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2007] In this invention, the server includes means for users to input profile information and interest information, means for collecting and saving user information, means for placing a generated avatar in a chat room and connecting with the user, means for searching for other users similar to the user in the background, means for notifying the user when another user is found, means for switching the chat partner from the avatar to another user if the user approves, means for recognizing emotions from the user's input message, facial expression, and voice data, and means for adjusting the content and tone of the chat message responses generated by the generation AI based on the recognized emotions. This not only allows users to enjoy natural conversations with other users with peace of mind, but also enables effective matching with other users who share common interests.

[2008] "User" refers to an individual or corporation that uses the system and provides profile information and interest information.

[2009] "Profile information" is basic information about a user, including data such as name, age, sex, address, and occupation.

[2010] "Interest information" is information about activities and topics in which a user is interested, including hobbies and favorite things.

[2011] "Generative AI" refers to artificial intelligence that automatically generates content such as text and images using generative models.

[2012] "Avatar" refers to a digital image or character that represents a user in a virtual world.

[2013] "Chat room" refers to a virtual room for exchanging messages online.

[2014] "Server" refers to the central computer of the system, a device that manages data storage, processing, and communication with other devices and terminals.

[2015] "Background" refers to processing that the system executes automatically, independent of user operations.

[2016] An "emotion engine" refers to a program and system that analyzes and recognizes emotions from a user's messages, voice, facial expressions, etc.

[2017] "Notification" refers to the act of the system sending a message or information to the user.

[2018] "Chat message" refers to text or audio information exchanged between users in a chat room.

[2019] "Matching" refers to the process of connecting users with common interests.

[2020] "Response content" refers to the text of the response that the generation AI generates in response to the user's message.

[2021] "Tone" refers to the style and expression of the chat messages created by the generative AI.

[2022] A "chat partner" refers to another user or avatar with whom a user communicates in a chat room.

[2023] This invention is a system in which a user starts a chat with a generated avatar, searches for similar users in the background, and if a match is successful, switches the chat partner from the avatar to a real person. The system incorporates an emotion engine that can recognize the user's emotions and use them in chat responses and matching. A server, a terminal, a generative AI model, and an emotion engine are required to implement this system.

[2024] System Hardware and Software

[2025] The hardware required to implement the system includes smartphones, tablets, and PCs. Python and Node.js are used for server-side programming. Databases such as MySQL and PostgreSQL are used to store data. GPT-3 and other technologies are used for generative AI, and dedicated emotion analysis software is used as the emotion recognition engine.

[2026] Specific explanation of the process

[2027] 1. User Registration

[2028] Users open the application and enter their name, profile information (age, gender, hobbies, etc.) and interests on the registration screen. They can then upload a profile photo or choose to have their avatar created by generative AI.

[2029] The device converts the information entered by the user into JSON format and sends it to the server via an HTTP POST request.

[2030] The server stores the received user information in a database. If there is a request to create an avatar, it sends an API request to the generation AI and stores the generated avatar data in the database.

[2031] 2. Start a chat room

[2032] The user presses the "Start Chat" button within the application.

[2033] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[2034] 3. Chat Progression

[2035] Users can start chatting with their avatar, typing and sending messages, such as "Hi, I love movies."

[2036] The device sends messages in JSON format to the server.

[2037] The server passes the received message to the generation AI, which generates an appropriate response, sends the generated response back to the user, and displays a response such as "That movie is great. What movie have you seen recently?" on the device.

[2038] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[2039] 4. Proceeding with the Matching Process

[2040] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data. If a match is found, the user's information is added to the matching queue.

[2041] 5. User Notifications and Chat Switching

[2042] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[2043] The device will display a notification and offer a choice ("Yes" or "No").

[2044] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[2045] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[2046] The terminal displays the information about the new chat room to the user and starts a new chat.

[2047] Specific examples

[2048] 1. User Registration

[2049] User: Enter "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports" and choose to upload a profile photo or have an avatar generated by AI.

[2050] Terminal: Sends input information to the server.

[2051] Server: Stores the received information in a database, generates and saves avatars.

[2052] 2. Start a chat room

[2053] User: Presses the "Start Chat" button.

[2054] Server: Create a new chat room, add Yamada Taro and the generated avatar to the room, and start the matching process in the background.

[2055] 3. Chat Progression

[2056] User: Sends a message saying "Hi, I love movies."

[2057] Terminal: Sends a message to the server.

[2058] Server: Passes the received message to the generation AI and generates a response. It responds with "That movie is great, what movie have you seen recently?"

[2059] 4. Proceeding with the Matching Process

[2060] Server: Searches for other users who are interested in movies or sports based on chat content and user information.

[2061] 5. User Notifications and Chat Switching

[2062] Server: If the match is successful, notify Yamada Taro, "A user similar to you has been found. Do you want to continue?"

[2063] Terminal: Show a "Yes / No" option and if the user selects "Yes", switch the chat partner to the new user.

[2064] Server: Create a new chat room and either carry over existing chat history or start a new session.

[2065] This system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar.Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[2066] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2067] Step 1: User Registration

[2068] Users open the application, enter their name, profile information (age, gender, hobbies, etc.), and interests, and choose to upload a profile photo or have a generative AI create an avatar for them.

[2069] Specific actions: Enter your name ("Yamada Taro") and hobbies ("watching movies and sports"), then select and upload a profile picture or tap the "Generate Avatar" button.

[2070] The device converts the entered information into JSON format and sends it to the server via an HTTP POST request.

[2071] Input: Name, Profile Information, Interests, Profile Picture.

[2072] Output: User information in JSON format.

[2073] Specific operation: Enter data into the input form, format it into JSON, and send it to the endpoint " / register".

[2074] The server receives user information in JSON format and stores it in a database. When a request to create an avatar is received, it sends an API request to the AI ​​generator and stores the generated avatar image in the database.

[2075] Input: JSON formatted user information sent from the terminal.

[2076] Output: User information stored in the database, generated avatar.

[2077] Specific operation: The received information is inserted into the database, a request is sent to the generation AI, and the generated avatar data is saved back into the database.

[2078] Step 2: Start a chat room

[2079] The user presses the "Start Chat" button within the application.

[2080] Specific actions: Press the "Start chat" button to begin the process.

[2081] The server creates a new chat room, adds the user and the generated avatar to the room, and also starts a background process to search for other users similar to the user.

[2082] Input: User's request to start a chat room.

[2083] Output: New chat room, background search process.

[2084] Specific actions: Generate a new chat room ID, add the user and avatar to the chat room, and start searching for similar users.

[2085] Step 3: Chat Progression

[2086] Users can start chatting with their avatar, typing and sending messages such as "Hi, I love movies."

[2087] Specific actions: Enter information into the text box and click the send button.

[2088] The device sends messages in JSON format to the server.

[2089] Input: The message entered by the user.

[2090] Output: Message data in JSON format.

[2091] Specific operation: Convert the input message into JSON and POST it to the endpoint " / send_message".

[2092] The server passes the received message to the generation AI, which generates an appropriate response, which is then sent back to the user and displayed on the screen.

[2093] Input: Message received from the terminal.

[2094] Output: Response message from the generating AI.

[2095] Specific operation: The received message is passed to the generation AI, which generates a response saying, "The movie is great. What movie have you seen recently?" and sends it back to the device and displays it on the screen.

[2096] The server uses an emotion engine to recognize emotions from the user's input messages, facial expressions, and voice data, and adjusts the content and tone of the generated AI's responses based on the recognized emotions.

[2097] Input: User message, emotion data.

[2098] Output: The adjusted response message.

[2099] Specific operation: Emotion analysis is performed using the emotion engine, and the generative AI creates a tone-adjusted response.

[2100] Step 4: Proceed with the matching process

[2101] The server runs a background process to search for similar users based on chat content, user profile information, interests, and emotional data.

[2102] Input: User chat content, profile information, interest data, and emotional data.

[2103] Output: Potential matching users.

[2104] What it does: It uses a scoring algorithm to query the database to find similar users and adds potential matches to a queue.

[2105] Step 5: Notify users and switch chat partners

[2106] If the match is successful, the server notifies the user, "A user similar to you has been found. Do you want to continue?"

[2107] Input: The result of the matching process.

[2108] Output: Informational message.

[2109] Specific behavior: If a new match is found, create a notification message and send it to the user.

[2110] The device will display a notification and offer a choice ("Yes" or "No").

[2111] Input: Notification message from the server.

[2112] Output: The displayed notification and selection options.

[2113] Specific operation: Implement a mechanism to display a pop-up screen and provide the user with options.

[2114] The user checks the notification and presses the "Yes" button if they wish to approve the match.

[2115] Specific action: Tap the "Yes" button to approve.

[2116] If the server accepts, it ends the current chat session and starts a new session with the new user, creating a new chat room and optionally inheriting the existing chat history.

[2117] Input: Acknowledgement from the user.

[2118] Output: New chat session, new chat room.

[2119] Specific actions: Close the old chat room, generate a new chat room ID, and carry over the history if necessary.

[2120] The terminal displays the information about the new chat room to the user and starts a new chat.

[2121] Input: New chat room information.

[2122] Output: Updated chat screen.

[2123] Specific operation: The new chat room information is reflected on the screen and displayed to the user.

[2124] (Application example 2)

[2125] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2126] Conventional chat systems lack the means to safely and effectively match users with other users. They also struggle to provide a natural, emotion-based chat experience, and are unable to recommend products and services that match the user's interests. The objective of this invention is to provide a system that improves the user experience, enables more natural and personal communication, and recommends appropriate products and services.

[2127] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input profile information and interest information; a means for collecting and saving the user information; a means for placing a generated avatar in a chat room and connecting with the user; a means for searching for other users similar to the user in the background; a means for notifying the user when the other user is found; a means for switching the chat partner from the avatar to the other user if the user approves; a means for analyzing the user's emotions and adjusting the tone of chat messages based on the emotion data; and a means for analyzing the user's input text in real time and recommending appropriate products or services. This allows users to be safely and effectively matched with other users, enjoy a natural chat experience based on their emotions, and receive recommendations for products and services that match their interests.

[2128] "User profile" refers to personal information entered by a user, such as name, age, gender, hobbies, and interests.

[2129] "Generated avatar" refers to a virtual character created by the generation AI based on the user's profile information.

[2130] "Chat room" refers to a virtual space where a user, an avatar, or other users can exchange messages online.

[2131] An "emotion engine" refers to software that recognizes emotions from the user's input text, voice, facial expressions, etc., and uses the results.

[2132] "Searching in the background" refers to the process by which the system automatically searches for other similar users without any explicit user interaction.

[2133] "Matching" refers to the process of finding other users with whom you have common ground and compatibility based on your profile information, interests, and emotional data.

[2134] "Notification" refers to a message that the system displays to inform the user.

[2135] "Adjusting the tone of a chat message" refers to changing the tone and expression of a generated message based on the user's emotional data.

[2136] "Recommending products or services" refers to the system suggesting related products or services based on the user's input text and sentiment.

[2137] "Real-time" refers to immediate response or processing of user input.

[2138] The embodiment of the present invention is based on building a system in which a user starts a chat with a generated avatar, while searching for other users similar to the user in the background, and switching the chat partner from the avatar to a real person if a match is successful, according to the following steps:

[2139] 1. User Registration

[2140] The user opens the application, goes to the registration screen, and enters their name, profile information (age, gender, hobbies, etc.), and interests. They can then upload a profile photo or request that the AI ​​create an avatar for them. The device sends the information the user entered to the server, which stores the received user information in a database and, if necessary, uses the AI ​​to create an avatar based on the user information, storing the generated avatar data in the database.

[2141] 2. Start a chat room

[2142] The user presses a "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[2143] 3. Chat Progression

[2144] The user starts a chat with an avatar, inputs a message, and sends it. The device sends the message to the server, which passes the received message to the generation AI and generates an appropriate response. The generated response message is returned to the user and displayed on the device. The server also uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data. Based on the recognized emotions, the server adjusts the content and tone of the generation AI's response to adapt to the user, providing a more natural chat experience.

[2145] 4. Proceeding with the Matching Process

[2146] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[2147] 5. User Notifications and Chat Switching

[2148] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and provides a choice ("Yes" or "No"). The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays the new chat room information to the user and starts a new chat.

[2149] Hardware and software used

[2150] Generative AI models (e.g., GPT-type language models)

[2151] Emotion engine (e.g., emotion analysis AI)

[2152] Server (e.g. cloud server)

[2153] Devices (e.g. smartphones, head-mounted displays)

[2154] Specific examples

[2155] Consider a scenario where a user enters "Name: Yamada Taro" and "Hobbies: Watching movies, watching sports," and either uploads a profile picture or selects to have an avatar generated by a generative AI. In this case, the user presses the "Start Chat" button to start a chat room and sends the message "Hello, I love movies" to the avatar. The server passes the received message to the generative AI, which generates a response. For example, an emotion engine could analyze the user's emotions and return a tone-adjusted response such as "Movies are great. What movies have you seen recently?"

[2156] Prompt Sentence Examples

[2157] Input sentence: "I've been liking movies lately"

[2158] Prompt: "The user is talking about movies. Analyze the given input for emotions and respond in a friendly manner."

[2159] AI Response: "That's great! What movie did you see recently?"

[2160] As described above, this system allows users to effectively match with other real users who share common interests through safe and enjoyable communication with the generated avatar. Furthermore, the introduction of an emotion engine enables natural conversations that respond to the user's emotions, providing a more satisfying chat experience.

[2161] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2162] Step 1: User Registration

[2163] The user opens the application, enters their name, profile information (age, gender, hobbies, etc.) and interests, and either uploads a profile photo or requests that the AI ​​create an avatar. The device sends the entered information to the server. The server stores the received user information in a database, uses the AI ​​to create an avatar based on the user information, and also stores the data of the generated avatar in the database.

[2164] Input: Name, profile information, interests, profile picture or avatar creation request

[2165] Output: Generated avatar data

[2166] Specific behavior:

[2167] User enters information via application

[2168] Send information from the device to the server

[2169] The server stores the information in a database and generates an avatar using a generative AI.

[2170] Step 2: Start a chat room

[2171] The user presses the "Start Chat" button in the application. The server creates a new chat room and adds the user and their generated avatar to the room. The server also starts a background process to search for other users similar to the user, analyzing the user's profile information, interests, and chat content and comparing them with other users in the database.

[2172] Input: Press the "Start Chat" button

[2173] Output: Creates a new chat room and populates it with users and avatars.

[2174] Specific behavior:

[2175] User presses "Start Chat" button

[2176] The server creates a new chat room and places users and avatars in it.

[2177] Start searching for other similar users in the background

[2178] Step 3: Chat Progression

[2179] The user starts a chat with the avatar, inputs a message, and sends it. The device sends the message to the server. The server passes the received message to the generation AI, which generates an appropriate response. The generated response message is sent back to the user and displayed on the device. The server uses an emotion engine to recognize emotions from the user's input message, facial expressions, and voice data, and adjusts the content and tone of the generation AI's response based on the recognized emotion.

[2180] Input: User input message

[2181] Output: The generated response message

[2182] Specific behavior:

[2183] User types and sends message

[2184] The device sends a message to the server

[2185] The server passes the message to the generation AI and generates a response.

[2186] A response message is returned to the user and displayed on the terminal.

[2187] Emotion engine recognizes user emotions and adjusts response tone

[2188] Step 4: Proceed with the matching process

[2189] The server runs a background process to search for similar users based on the current chat content, the user's profile information, interests, and emotional data, and if a match is found, adds the user's information to a matching queue.

[2190] Input: Current chat content, profile information, interests, emotional data

[2191] Output: User information of match candidates

[2192] Specific behavior:

[2193] The server analyzes chat content and user information in the background

[2194] Find other similar users

[2195] If a match is found, add the information to the match queue

[2196] Step 5: Notify users and switch chat partners

[2197] If the match is successful, the server sends a notification to the user, informing them of the other user's information. The device displays the notification and offers the option of "Yes" or "No." The user checks the notification and presses the "Yes" button to approve the match. If approval is granted, the server ends the current chat session and starts a session with the new user. The server creates a new chat room and, if necessary, transfers the existing chat history to the new user. The device displays information about the new chat room to the user and starts a new chat.

[2198] Input: User information of match candidates, user approval

[2199] Output: Create a new chat room and switch chat partners

[2200] Specific behavior:

[2201] The server notifies the user when a match is successful

[2202] Your device will display a notification and offer you the option to choose "Yes" or "No."

[2203] User selects "Yes"

[2204] The server ends the chat session and starts a new session with the user.

[2205] Create a new chat room and take over the information

[2206] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2207] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2208] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2209] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2210] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2211] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2212] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2213] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2214] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2215] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2216] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2217] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2218] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2219] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2220] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2221] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2222] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2223] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2224] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2225] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2226] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2227] The following is further disclosed regarding the above embodiment.

[2228] (Claim 1)

[2229] a means for a user to input profile information and interest information;

[2230] means for collecting and storing said user information;

[2231] means for placing the generated avatar in a chat room and connecting with the user;

[2232] A means for searching other users similar to the user in the background;

[2233] means for notifying the user when the other user is found;

[2234] The system includes means for switching the chat partner from the avatar to the other user if the user approves.

[2235] (Claim 2)

[2236] 10. The system of claim 1, further comprising means for generating chat message responses in real time by the generated avatar.

[2237] (Claim 3)

[2238] 2. The system according to claim 1, further comprising means for searching for a match with another user based on the user's profile information, interest information, and chat content.

[2239] "Example 1"

[2240] (Claim 1)

[2241] a means for a user to input profile information and interest information;

[2242] means for transmitting the user information to a server;

[2243] A means for storing the received user information in a database and creating an avatar using a generation AI;

[2244] means for placing the generated avatar in a chat room and connecting with the user;

[2245] A means for searching other users similar to the user in the background;

[2246] means for notifying the user when the other user is found;

[2247] The system includes means for switching the chat partner from the avatar to the other user if the user approves.

[2248] (Claim 2)

[2249] 10. The system of claim 1, further comprising means for generating chat message responses in real time by the generated avatar.

[2250] (Claim 3)

[2251] 2. The system according to claim 1, further comprising means for searching for a match with another user based on the user's profile information, interest information, and chat content.

[2252] "Application Example 1"

[2253] (Claim 1)

[2254] a means for a user to input profile information and interest information;

[2255] means for collecting and storing said user information;

[2256] means for placing the generated avatar in a chat room and connecting with the user;

[2257] A means for searching other users similar to the user in the background;

[2258] means for notifying the user when the other user is found;

[2259] means for switching the chat ...

Claims

1. a means for a user to input profile information and interest information; means for collecting and storing said user information; means for placing the generated avatar in a chat room and connecting with the user; A means for searching other users similar to the user in the background; means for notifying the user when the other user is found; The system includes means for switching the chat partner from the avatar to the other user if the user approves.

2. The system of claim 1 , further comprising means for generating chat message responses in real time by the generated avatar.

3. The system according to claim 1, further comprising means for searching for a match with another user based on the user's profile information, interest information, and chat content.

Images