System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a task by a user; means for generating an appropriate virtual personality based on the task input by the user; means for causing the generated virtual personality to participate in an online meeting; and means for performing a dialogue between the user and the virtual personality in real time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's business environment, advice from people with specialized knowledge and skills is essential to solving specific problems. However, it is difficult to quickly identify people with the necessary skills and experience and consult with them at the appropriate time. Furthermore, when multiple people are required, the effort increases even more. There is a need for a new online meeting system that can solve these problems. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a means for a user to input a task, a means for generating an appropriate virtual personality based on the task input by the user, a means for the generated virtual personality to participate in an online meeting, and a means for real-time interaction between the user and the virtual personality. The system also includes a means for analyzing keywords related to the task input by the user and a means for selecting and generating a virtual personality with skills based on the analyzed keywords, thereby quickly and accurately generating an appropriate virtual personality. The present invention enables users to quickly obtain necessary expertise and advice through online meetings, thereby enabling them to solve tasks efficiently.

[0006] An "issue" is a problem or matter of consultation that a user seeks to resolve in an online meeting.

[0007] A "virtual personality" is a character created by generative AI with specific skills and experiences that participates in online meetings and interacts with users.

[0008] An "online meeting" is a real-time conference or consultation held over the Internet.

[0009] A "user" is an individual or organization that participates in an online meeting and seeks to solve a specific problem.

[0010] A "terminal" is a device used by a user, such as a computer or smartphone, that functions as a platform for online meetings.

[0011] A "server" is a central computer system that manages the entire system and processes data.

[0012] "Profile information" is detailed information such as the virtual personality's name, role, and expertise.

[0013] "Analysis" is the process of extracting issues and keywords entered by the user and identifying related information based on them.

[0014] "Generation" is the process by which the AI ​​model creates an appropriate virtual personality based on the tasks entered by the user.

[0015] "Real-time" refers to processing and interaction occurring immediately, with little to no delay. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system that allows users to use an online meeting platform and input a specific topic to generate an appropriate virtual personality to participate in the online meeting, enabling users to obtain expertise and advice effectively and efficiently.

[0038] Overall system overview

[0039] The system consists of the following main components:

[0040] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0041] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0042] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0043] 4. Online meeting platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0044] System operation flow

[0045] 1. A user enters an assignment and starts an online meeting

[0046] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0047] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0048] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0049] 2. The server analyzes the task and generates a virtual personality

[0050] The server extracts the room name "Consultation regarding the introduction of AI technology" from the received meeting start request.

[0051] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0052] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0053] 3. Involve virtual personalities in online meetings

[0054] The server transmits profile information of the generated virtual personality to the user terminal.

[0055] The user terminal receives this information and displays the virtual persona in the online meeting room.

[0056] 4. User and virtual personality interact in real time

[0057] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0058] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0059] Interaction between the user and the virtual persona occurs in real time with little delay.

[0060] Specific examples

[0061] Example: Consultation on an AI technology implementation project

[0062] The user logs into the online meeting platform and selects "New Meeting."

[0063] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0064] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0065] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0066] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0067] The above is a specific embodiment for carrying out the present invention. This system enables users to quickly obtain specialized knowledge to efficiently solve specific problems.

[0068] The processing flow will be explained below.

[0069] Step 1: User logs into online meeting platform

[0070] Users use their own terminals to access the online meeting platform.

[0071] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0072] Step 2: Enter the assignment in the room name and start the meeting

[0073] The user opens the "New Meeting" screen of the online meeting platform.

[0074] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0075] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0076] Step 3: The server receives and analyzes the issue

[0077] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0078] The server extracts the room name from the received request.

[0079] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0080] Step 4: The server generates the virtual personality

[0081] The server calls a virtual personality generation engine based on the identified keywords.

[0082] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[0083] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[0084] Step 5: The server sends the virtual personality information to the device.

[0085] The server transmits the profile information of the generated virtual personality to the user's terminal.

[0086] Step 6: Your device displays your virtual persona in online meetings

[0087] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0088] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0089] Step 7: User interacts with virtual persona in real time

[0090] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[0091] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[0092] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[0093] Step 8: User ends meeting and provides feedback

[0094] After obtaining the necessary information, the user performs an operation to end the online meeting.

[0095] The terminal sends a meeting end request to the server.

[0096] The server acknowledges the end of the meeting and stores a log of the session.

[0097] The server displays a post-meeting feedback form on the user terminal.

[0098] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In recent years, the convenience of online meetings has increased, and remote dialogue and consultations have become increasingly important. However, when discussing issues that require specific expertise, it is not easy to find the right person to consult with. Furthermore, opportunities to interact with experts in real time are limited, making it difficult for users to quickly obtain expert advice. Therefore, there is a need for a system that allows users to easily obtain expert knowledge and advice.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes a means for a user to input a task, a means for generating an appropriate virtual character based on the task input by the user, and a means for allowing the generated virtual character to participate in an online conference, thereby enabling the user to interact with the professional virtual character in real time.

[0104] "User" means an individual or organization that utilizes the online conferencing platform to enter tasks and interact with virtual people.

[0105] "Issues" refer to problems that users want to solve or consultation topics, and are entered into the online conference platform.

[0106] A "virtual person" is a character with specific expertise or skills that is generated based on a task entered by the user.

[0107] An "online conference" is an electronic conference that allows a user and a virtual person to interact in real time over the Internet.

[0108] "Real-time" refers to immediate interaction between the user and the virtual persona with little delay.

[0109] "Context information" refers to keywords and phrases obtained by analyzing the content of the task entered by the user, and is data necessary for generating a virtual person.

[0110] "Technical capabilities" refer to the specialized knowledge and skills that the virtual person possesses, and are used to provide appropriate advice and information to the user regarding their problems.

[0111] "Specification information" indicates detailed data about the virtual person, such as the profile and skill set of the generated virtual person.

[0112] The "central device" refers to a server that receives requests from users and performs virtual person generation and related processing.

[0113] "Terminal" refers to the device (e.g., PC, smartphone, tablet) that a user uses to access an online meeting and interact with a virtual person.

[0114] The present invention provides a system that allows users to use an online conference platform and input a specific topic to generate an appropriate virtual person and have them participate in the online conference, enabling users to obtain expert knowledge and advice effectively and efficiently.

[0115] System Configuration

[0116] The system consists of the following main components:

[0117] 1. User terminal: A device used by users to start online meetings and enter tasks. This includes PCs, smartphones, tablets, etc.

[0118] 2. Server: A central device that receives the tasks entered by users, analyzes them, generates virtual people, and allows them to participate in online meetings.

[0119] 3. Virtual Person Generation Engine: This engine is located on the server and runs an AI model that generates an appropriate virtual person based on the task. It also performs text analysis using an NLP library.

[0120] 4. Online conference platform: Provides an environment for users and generated virtual characters to interact in real time.

[0121] System Operation Overview

[0122] First, the user logs in to the online conference platform using their device. They then open the "New Meeting" screen and enter the problem they want to solve in the room name field. For example, they might enter "Consultation on the introduction of AI technology." Then, they click the "Start" button, and the user's device sends a meeting start request, including the room name, to the server.

[0123] The server extracts the room name from the received meeting start request and analyzes the problem statement. It uses an NLP library to identify important keywords and invokes a virtual person generation engine to generate virtual people with skill sets related to those keywords. For example, for the problem "Consultation on the introduction of AI technology," it generates virtual people of an AI expert and a project manager.

[0124] The server sends the profile information of the generated virtual person to the user's terminal. The user's terminal receives this information and displays the virtual person in an online conference room. The user can then interact with the displayed virtual person and receive expert advice and suggestions. The interaction between the user and the virtual person is real-time, with almost no delay.

[0125] Specific examples

[0126] As a concrete example, a user logs in to an online conference platform and selects "New Meeting." He enters "Consultation on the Implementation of AI Technology" as the room name and sends a request to start a meeting to the server. The server analyzes the room name and uses a virtual person generation engine to generate virtual characters of an AI expert and project manager with the required skills. The profile information of the generated virtual characters is sent to the user's device, which then displays the virtual characters in the online conference room. The user can then converse with the virtual characters in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[0127] Examples of prompt statements

[0128] For example, you can input the following prompts into a generative AI model:

[0129] "I would like to consult with you regarding the implementation of AI technology. Please generate a virtual person with the appropriate expertise."

[0130] Using this prompt, the system can generate a virtual person best suited to the task and have them participate in the online meeting. By using this system, users can quickly acquire specialized knowledge.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] A user logs in to an online meeting platform and opens the "New Meeting" screen. The user enters the topic in the room name field. For example, they enter "Consultation on the introduction of AI technology" and clicks the "Start" button. This creates a meeting start request that includes the room name.

[0134] Input: The user inputs the task and presses the "Start" button. The input data includes the room name "Consultation on the introduction of AI technology."

[0135] Output: A conference start request is generated and sent to the server.

[0136] Step 2:

[0137] The server receives a conference start request from a user terminal. This request includes the room name entered by the user. The server extracts the room name and performs text analysis.

[0138] Input: Meeting start request (room name "Consultation on the introduction of AI technology")

[0139] Output: Keywords used for analysis (e.g., AI technology, introduction)

[0140] Specific operation: The server extracts the room name from the request received in JSON data format and performs text analysis using an NLP library.

[0141] Step 3:

[0142] The server then activates a virtual person generation engine based on the analyzed keywords to generate appropriate virtual people, such as an AI expert and project manager who correspond to the analyzed keywords "AI technology" and "implementation."

[0143] Input: Analyzed keywords (e.g., AI technology, introduction)

[0144] Output: Profile information of the generated virtual person (e.g. details of an AI expert, project manager)

[0145] Specific operation: The server calls the virtual person generation engine API, passes the necessary parameters (keywords) and requests the generation of a virtual person. It then obtains the profile information of the generated virtual person.

[0146] Step 4:

[0147] The server sends the profile information of the generated virtual person to the user terminal as an HTTP response.

[0148] Input: Profile information of the generated virtual persona (e.g. AI expert, project manager details)

[0149] Output: HTTP response containing the virtual person's profile information

[0150] Specific operation: The server constructs profile information in JSON format and sends it to the user device as an HTTP response.

[0151] Step 5:

[0152] The user terminal receives the profile information of the virtual person from the server and displays it in the online conference room. The virtual person's avatar and profile are displayed, and the user is ready to start a conversation.

[0153] Input: Virtual person profile information sent from the server

[0154] Output: Avatar and profile of a virtual person displayed in an online meeting room

[0155] Specific operation: The user device parses the received JSON data and visually displays the virtual person in the browser or application UI.

[0156] Step 6:

[0157] Users can ask questions or ask the virtual person for advice in an online conference room, and the virtual person will respond in real time, providing expert advice and suggestions to the user's questions.

[0158] Input: User's question or inquiry

[0159] Output: Real-time response from the virtual person

[0160] How it works: The user types a question through a chat window or voice input, and the virtual person uses a generative AI model to generate a response in real time and returns the result to the user, with almost no latency.

[0161] (Application example 1)

[0162] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] In the past, when a factory robot broke down, an engineer had to go to the site and troubleshoot manually. However, this method required time and effort, and could prolong the production line downtime. Another problem was that engineers were not familiar with all causes of the failure, making it difficult to quickly obtain appropriate advice. Therefore, the objective of this invention is to provide a system that efficiently diagnoses factory robot failures and provides prompt support to engineers.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0165] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for the generated virtual personality to participate in an online meeting, means for real-time interaction between the user and the virtual personality, means for inputting a fault symptom of a factory robot, and means for generating a virtual engineer based on the fault symptom, thereby enabling the virtual engineer to assist in fault diagnosis of the factory robot in real time.

[0166] The "means for users to input problems" refers to a device or program that provides an interface that allows users such as engineers to input specific failure symptoms and problems.

[0167] The "means for generating an appropriate virtual personality" is a system that creates a virtual expert with a relevant skill set based on an input task.

[0168] The "means for allowing a generated virtual personality to participate in an online meeting" is a technology that allows a virtual personality to appear in an online meeting in real time and interact with a user.

[0169] "Means for real-time dialogue between a user and a virtual personality" refers to an interactive system in which a generated virtual personality can immediately respond to questions entered by a user.

[0170] The "means for inputting the malfunction symptoms of the factory robot" is an input device or software that allows an engineer to input specific malfunctions or problems related to the factory robot.

[0171] The "means for generating a virtual engineer based on a failure symptom" is a system that generates a virtual mechanical engineer with specialized knowledge in accordance with the inputted failure symptom of a robot.

[0172] "Means for analyzing keywords related to the task entered by the user" refers to technology that extracts and recognizes important words and phrases from the text entered by the user.

[0173] The "means for selecting and generating skilled virtual personalities" is a system that identifies and generates virtual experts with specialized abilities that correspond to the analyzed keywords.

[0174] The "means for the server to transmit profile information of the generated virtual personality to the terminal" is a system that transmits detailed information about the virtual expert created by the server to the user's terminal.

[0175] The "means for a terminal to display a virtual personality in an online meeting" is a technique by which a user's device displays a generated virtual expert on an online meeting screen.

[0176] "A means of supporting the fault diagnosis of factory robots in real time" is a technology in which a virtual engineer provides engineers with immediate fault diagnosis and repair advice.

[0177] The system for realizing this invention is composed of a user terminal, a server, a virtual personality generation engine, and an online meeting platform. As a specific system for supporting fault diagnosis of factory robots, the following components are used in combination.

[0178] 1. User device

[0179] The system uses tablets or smart glasses that allow users to input malfunction symptoms and specific problems of factory robots. Users input malfunction symptoms in text format through a dedicated interface.

[0180] 2. Server

[0181] The server receives and analyzes the failure symptoms sent from the user's device. It uses advanced natural language processing technology to extract important keywords from the input text. The hardware used is a high-performance Linux-based server, and the software uses generative AI models such as GPT-4.

[0182] 3. Virtual personality generation engine

[0183] The virtual personality generation engine installed on the server generates appropriate virtual engineers based on keywords extracted by natural language processing. The virtual personality generation engine creates virtual engineers with specialized skill sets, such as mechanical engineers or electrical control experts.

[0184] 4. Online Meeting Platform

[0185] The user and the generated virtual engineer interact in real time through an online meeting platform. The virtual engineer responds immediately to questions entered by the user and provides expert advice and suggestions. WebRTC and Zoom APIs are used.

[0186] Specifically, when a user types "robot arm malfunctions" into a tablet, the server receives and analyzes this text. The server extracts keywords such as "robot arm" and "malfunction" and passes them to the virtual personality generation engine. The virtual personality generation engine generates a virtual engineer specializing in robot arms based on these keywords. The generated virtual engineer participates in an online meeting room and assists the user in fault diagnosis while interacting in real time with the user.

[0187] As an example of a prompt sentence, by entering "Please tell me about the situation when the robot arm has stopped moving," the virtual engineer will immediately provide advice such as "The robot arm is malfunctioning. The symptoms are that it does not move even when instructed, or it moves very slowly. Please tell me the possible causes of the malfunction and how to deal with it."

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1:

[0190] A user accesses the online meeting platform on a user device such as a tablet and opens a screen to start a new meeting. The user enters a specific malfunction symptom, such as "the robot arm is not working properly," into the input field and clicks the "Start" button. Input: Malfunction symptom, Output: Meeting start request.

[0191] Step 2:

[0192] The terminal sends a meeting start request to the server. The request includes the problem (fault symptom) entered by the user. Input: Meeting start request, Output: Request data.

[0193] Step 3:

[0194] The server analyzes the received meeting start request. The server receives the failure symptom text (e.g., "Robot arm malfunction") and uses natural language processing technology to extract important keywords (e.g., "robot arm," "malfunction"). Input: Request data, Output: Extracted keywords.

[0195] Step 4:

[0196] The server passes the extracted keywords to the virtual personality generation engine, which generates a virtual engineer with related skills (e.g., an engineer specializing in robotic arms) based on these keywords. Input: extracted keywords, Output: virtual engineer.

[0197] Step 5:

[0198] The server sends the profile information and setting information of the created virtual engineer to the user terminal. The user terminal receives this information and displays the virtual engineer in the online meeting room. Input: Virtual engineer profile information, Output: Display data.

[0199] Step 6:

[0200] Users can ask specific questions or ask for advice to a virtual engineer displayed in an online meeting room. The virtual engineer responds immediately and provides expert advice and suggestions to the user's questions. Input: User's question, Output: Virtual engineer's response.

[0201] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0202] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system enables users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[0203] Overall system overview

[0204] The system consists of the following main components:

[0205] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0206] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0207] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0208] 4. Emotion Engine: An engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses accordingly.

[0209] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0210] System operation flow

[0211] 1. A user enters an assignment and starts an online meeting

[0212] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0213] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0214] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0215] 2. The server analyzes the task and generates a virtual personality

[0216] The server extracts the room name from the received start meeting request.

[0217] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0218] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0219] 3. The server sends the virtual personality's profile information to the device.

[0220] The server transmits the profile information (e.g., name, role, expertise) of the generated virtual personality to the user terminal.

[0221] 4. The device displays a virtual personality in online meetings

[0222] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0223] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0224] 5. Real-time interaction between the user and the virtual persona

[0225] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0226] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0227] 6. Emotional engine for analyzing user emotions and adjusting responses

[0228] The emotion engine analyzes the user's voice and text data to identify their emotions (e.g., joy, anger, anxiety).

[0229] The emotion engine adjusts the virtual personality's responses based on the analysis results, providing appropriate advice and assistance according to the user's emotions.

[0230] The virtual personality empathizes with the user's feelings and responds accordingly to the situation.

[0231] Specific examples

[0232] Example: Consultation on AI technology implementation projects and emotion recognition

[0233] The user logs into the online meeting platform and selects "New Meeting."

[0234] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0235] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0236] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0237] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0238] The emotion engine analyzes the user's emotions from their voice and text and determines that the emotion is anxiety.

[0239] Based on the analysis results, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're here to support you through this step of the project.").

[0240] In this way, the system of the present invention not only efficiently solves the user's specific problems but also provides appropriate responses according to the user's emotions, allowing the user to have a more effective and satisfying online meeting experience.

[0241] The processing flow will be explained below.

[0242] Step 1: User logs into online meeting platform

[0243] Users use their own terminals to access the online meeting platform.

[0244] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0245] Step 2: Enter the assignment in the room name and start the meeting

[0246] The user opens the "New Meeting" screen of the online meeting platform.

[0247] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0248] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0249] Step 3: The server receives and analyzes the issue

[0250] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0251] The server extracts the room name from the received request.

[0252] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0253] Step 4: The server generates the virtual personality

[0254] The server calls a virtual personality generation engine based on the identified keywords.

[0255] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[0256] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[0257] Step 5: The server sends the virtual personality information to the device.

[0258] The server transmits the profile information of the generated virtual personality to the user's terminal.

[0259] Step 6: Your device displays your virtual persona in online meetings

[0260] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0261] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0262] Step 7: User interacts with virtual persona in real time

[0263] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[0264] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[0265] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[0266] Step 8: The emotion engine analyzes the user's emotions

[0267] The emotion engine (in the server) collects user voice and text data.

[0268] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., joy, anger, anxiety).

[0269] Step 9: The virtual personality adjusts responses based on emotions

[0270] The emotion engine provides information about the analyzed emotional state to the virtual personality.

[0271] The virtual personality adjusts the tone and content of its responses depending on the user's emotional state (e.g., making reassuring statements to an anxious user).

[0272] Step 10: User ends meeting and provides feedback

[0273] After obtaining the necessary information, the user performs an operation to end the online meeting.

[0274] The terminal sends a meeting end request to the server.

[0275] The server acknowledges the end of the meeting and stores a log of the session.

[0276] The server displays a post-meeting feedback form on the user terminal.

[0277] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[0278] Example 2

[0279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] Modern online meeting systems make it difficult for users to obtain effective support for issues that require specialized knowledge. They also fail to provide responses that take users' feelings into consideration, which can lead to communication issues. As a result, users find it difficult to have a satisfying meeting experience.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0282] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for having the generated virtual personality participate in an online meeting, means for analyzing the user's emotions, and means for adjusting the virtual personality's responses based on the emotion analysis. This allows the user to receive effective and efficient support from a virtual personality with specialized knowledge, and further allows the user to receive empathetic responses according to the user's emotions.

[0283] "User" refers to a person who uses the online meeting platform to enter an issue and receive assistance.

[0284] "Issue" refers to a specific problem or issue that a user seeks to resolve.

[0285] A "virtual personality" refers to a virtual character generated using AI technology that has specific skills or expertise.

[0286] "Online meeting" refers to a virtual conference environment in which users and virtual personalities can interact in real time over the Internet.

[0287] An "emotion engine" refers to technology that analyzes a user's voice and text data to identify the user's emotions.

[0288] "Profile information" refers to basic information such as the virtual personality's name, role, and expertise.

[0289] A "server" refers to a remote computer system that receives requests from users and generates virtual personalities and analyzes emotions based on those requests.

[0290] "Terminal" refers to the device (e.g., PC, smartphone) used by a User to access the online meeting platform.

[0291] "Generative AI model" refers to artificial intelligence technology that generates virtual personalities and responses based on input data.

[0292] A "prompt sentence" refers to an instruction sentence input to a generative AI model.

[0293] A "natural language processing engine" refers to a technology that extracts and analyzes important keywords from text entered by a user.

[0294] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system allows users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[0295] The system consists of the following main components:

[0296] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0297] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0298] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model (e.g., GPT-3) that generates an appropriate virtual personality based on the task.

[0299] 4. Emotion engine: An engine that analyzes emotions from the user's voice or text and adjusts the virtual personality's responses accordingly (e.g., IBM Watson Tone Analyzer).

[0300] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0301] Example of operation

[0302] Example: Consultation on AI technology implementation projects and emotion recognition

[0303] A user logs into an online meeting platform and selects "New Meeting." They enter "Consultation on the Implementation of AI Technology" as the room name and send a request to start a meeting to the server. The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities for an AI expert and project manager with the required skills.

[0304] The server then sends the profile information of the virtual personality to the user's device, which then displays the virtual personality in an online meeting room. The user can then converse with the virtual personality in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[0305] The emotion engine analyzes the user's voice and text to determine if the emotion is anxiety. Based on this analysis, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with this step of the project.").

[0306] Prompt Sentence Examples

[0307] Below are some example prompts for the emotion engine to use to analyze user sentiment and tailor responses to the generative AI model:

[0308] "Tell us how a virtual personality responds when it recognizes that the user is anxious."

[0309] "Please demonstrate how the virtual personality can empathize with the user's anger as analyzed by the emotion engine."

[0310] To implement this invention, a user device accesses the online meeting platform using a browser or a dedicated application, and the server receives and analyzes requests from users. The virtual personality generation engine and emotion engine use dedicated AI models (e.g., GPT-3, IBM Watson Tone Analyzer) that run on the server. This allows users to receive emotionally sensitive responses while solving their problems.

[0311] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0312] Step 1:

[0313] A user enters an assignment and starts an online meeting.

[0314] Users access the online meeting platform using a browser or a dedicated application. They enter their username and password on the login screen and click the "Login" button. After logging in, they open the "New Meeting" screen and enter the problem they want to solve in the room name field. The entered room name is sent from the device to the server in JSON format as a meeting start request. The input data is the content of the problem, and the output is a request to the server.

[0315] Step 2:

[0316] The server analyzes the task and generates a virtual personality.

[0317] The server extracts the room name from the received meeting start request. Based on the room name, the server uses a natural language processing engine to analyze the required keywords (e.g., "AI technology," "implementation"). Using the analyzed keywords as input, the server invokes a virtual personality generation engine (e.g., GPT-3) to generate a virtual personality with the associated skill set (e.g., AI expert, project manager). The output is the virtual personality's profile information.

[0318] Step 3:

[0319] The server transmits the profile information of the virtual personality to the terminal.

[0320] The server sends the profile information of the generated virtual personality to the user's device in JSON format. The profile information includes the virtual personality's name, role, and expertise. The input data is the result of the virtual personality generation, and the output is the transmission of the profile information to the user's device.

[0321] Step 4:

[0322] The device displays a virtual persona in online meetings.

[0323] The user device analyzes the received virtual personality profile information and displays it on the screen of the online meeting room. The virtual personality then gives an initial greeting and introduces itself. The input data is the profile information, and the output is the displayed virtual personality.

[0324] Step 5:

[0325] The user and the virtual persona interact in real time.

[0326] The user asks the virtual persona a question through a chat box or voice input (e.g., "What is the first step in implementing AI?"). The virtual persona receives the user's question and generates a response using a generative AI model (e.g., "We recommend starting by defining the technical requirements."). The input data is the user's question, and the output is the virtual persona's response.

[0327] Step 6:

[0328] Emotion engine analyzes user emotions and adjusts responses.

[0329] The emotion engine analyzes the user's voice and text data to identify the user's emotions. For example, if the voice data indicates that the user is feeling anxious, the emotion engine communicates the analysis result to the virtual personality. The virtual personality then provides reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with each step of this project."). The input data is the user's emotional information, and the output is the adjusted response of the virtual personality.

[0330] Through the above steps, the system can provide users with specialized knowledge and respond in a way that takes into consideration the user's feelings.

[0331] (Application example 2)

[0332] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] Traditional customer support in brick-and-mortar stores often lacks staff on-site to address specific issues, making it difficult to respond quickly to customer needs. Furthermore, traditional support systems are unable to respond in a way that takes into account the customer's emotions, making it difficult to improve customer satisfaction. For this reason, there is a demand for personalized experiences that address customer issues and take their emotions into consideration.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for displaying the generated virtual personality in a physical store and providing customer support, means for analyzing the user's emotions and adjusting the virtual personality's response, and means for conducting a real-time dialogue between the user and the virtual personality. This makes it possible to quickly respond to a user's specific task using an appropriate virtual personality even in a physical store, and to provide personalized services according to the user's emotions.

[0335] A "user" is a person who uses the system to input tasks and interact with a virtual personality.

[0336] An "issue" is a specific problem or inquiry that a user wants the system to solve.

[0337] A "virtual personality" is a digital agent generated by AI that responds appropriately to user challenges.

[0338] A "brick and mortar store" is a commercial establishment that is located in a physical location and can be visited by customers in person.

[0339] "Sentiment analysis" is the process of identifying emotions from a user's voice or text data.

[0340] The "generation means" is a technical means for generating an appropriate virtual personality based on the task input by the user.

[0341] "Display means" refers to the technical means for displaying the generated virtual personality in a physical store or on a terminal.

[0342] "Response adjustment means" refers to a technical means for adjusting the response of a virtual personality based on the results of emotion analysis.

[0343] "Real-time interaction means" refers to a technical means for allowing a user and a virtual personality to interact in real time.

[0344] This invention relates to a system that generates an appropriate virtual personality by having the user input a specific problem, and provides customer support in a physical store. Furthermore, it can perform emotion analysis and adjust the virtual personality's response according to the user's emotions. The entire system is composed of the following main components:

[0345] Hardware and Software

[0346] Hardware:

[0347] iOS / Android smartphones

[0348] software:

[0349] iOS / Android applications (Swift / Kotlin)

[0350] Server (AWS EC2, Amazon RDS)

[0351] AI models (TensorFlow, PyTorch)

[0352] Analysis engine (Amazon Comprehend, Google Cloud Natural Language)

[0353] System configuration

[0354] 1. User Device:

[0355] It is a device that allows users to input tasks and interact with virtual personalities.

[0356] A smartphone application is installed and users can enter assignments through this application.

[0357] 2. Server:

[0358] It is a central server that receives and analyzes user-entered tasks to generate and display virtual personalities.

[0359] The server is equipped with an AI model and an emotion analysis engine, enabling advanced data processing.

[0360] 3. Virtual Personality Generation Engine:

[0361] It is an engine that runs an AI model located on a server and generates an appropriate virtual personality based on the user's challenges.

[0362] 4. Sentiment Analysis Engine:

[0363] It is an engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses based on that.

[0364] Application Examples

[0365] Example 1: Fashion coordination consultation

[0366] The user types into the smartphone app, "Please tell me how to coordinate this outfit."

[0367] The server analyzes the input and generates a virtual personality of a fashion expert.

[0368] The virtual expert appears as a virtual display in a physical store and greets the user with, "Hello, I'm a fashion expert. What outfit would you like to wear today?"

[0369] When a user asks, "Which top goes well with this skirt?", the emotion analysis engine analyzes the emotions in the user's voice and text, and if it determines that the user looks anxious, the virtual expert will respond in a gentle tone, "Don't worry, just relax and choose. I recommend this top."

[0370] Example 2: Shopping assistance

[0371] A user types "Tell me about the new smartphone" into a smartphone app.

[0372] The server analyzes the input and generates a virtual personality of a technical expert.

[0373] The virtual expert appears on a display in a physical store and speaks to the user, saying, "Hello, I'm a technology expert. I'd like to tell you about our new smartphone."

[0374] If a user asks, "What's the battery life?" and the sentiment analysis engine determines that the user is excited, the virtual expert will respond enthusiastically, "This phone has excellent battery life, lasting more than a day with normal use."

[0375] As described above, by combining virtual personality generation and emotion analysis, this system can provide customers with fast and personalized support in physical stores, improving customer satisfaction and contributing to sales promotion in physical stores.

[0376] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0377] Step 1:

[0378] A smartphone application is used as a means for users to input tasks. The user launches the smartphone application, enters the specific task they wish to solve (e.g., "Please tell me how to coordinate this outfit") in the text field on the "Task Input" screen, and taps the "Submit" button. This task input data is sent to the server. The input data is the task content in text format.

[0379] Step 2:

[0380] The server receives the assignment input data and begins analysis. The server analyzes the received assignment content and extracts related keywords. For example, keywords such as "clothes" and "coordinate" are extracted. These keywords are analyzed using a text analysis engine (e.g., Amazon Comprehend) to generate an extracted keyword list. The keyword list is the output of the process.

[0381] Step 3:

[0382] The server generates an appropriate virtual personality based on the extracted keyword list. The server invokes a virtual personality generation engine (AI model) to generate a virtual personality (e.g., a fashion expert) with a skill set related to the specified keywords. Profile information (e.g., name, expertise, role) of this virtual personality is generated, and the virtual personality profile information is output.

[0383] Step 4:

[0384] The server transmits the profile information of the generated virtual personality to the user terminal. The user terminal begins displaying the virtual personality based on the received profile information. Specifically, the virtual personality appears as a virtual display within the smartphone application, and an initial greeting is given, such as "Hello, I'm a fashion expert. What kind of outfit would you like to wear today?" The display screen is output.

[0385] Step 5:

[0386] Users interact with the virtual persona in real time by using the chat function or voice input function of the smartphone app to input and send questions (e.g., "Which top goes well with this skirt?"), and the user input data is sent to the system.

[0387] Step 6:

[0388] The server receives new input data from the user and performs emotion analysis. It uses an emotion analysis engine (e.g., Google Cloud Natural Language) to identify emotions (e.g., anxiety, excitement) from the user's voice or text data. The analysis results are output as emotion data.

[0389] Step 7:

[0390] The server generates and adjusts the response content of the virtual personality based on the emotional data. The virtual personality generation engine generates the response content together with the emotional data (e.g., "It's okay, just relax and choose. I recommend this top."), and the adjusted response data is generated. This adjusted response data is the output.

[0391] Step 8:

[0392] The server transmits the adjusted response data to the user terminal, and the user terminal displays or plays the response of the virtual personality in real time based on the received response data, and the user receives the response from the virtual personality and continues the next dialogue.

[0393] Through the above process, even in a physical store, a user can receive a quick and appropriate response from a virtual personality in response to a problem entered by the user, providing personalized service that takes into consideration the user's problem-solving and emotions.

[0394] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0395] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0396] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0397] [Second embodiment]

[0398] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0399] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0400] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0401] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0402] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0403] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0404] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0405] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0406] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0407] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0408] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0409] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0410] The present invention provides a system that allows users to use an online meeting platform and input a specific topic to generate an appropriate virtual personality to participate in the online meeting, enabling users to obtain expertise and advice effectively and efficiently.

[0411] Overall system overview

[0412] The system consists of the following main components:

[0413] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0414] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0415] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0416] 4. Online meeting platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0417] System operation flow

[0418] 1. A user enters an assignment and starts an online meeting

[0419] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0420] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0421] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0422] 2. The server analyzes the task and generates a virtual personality

[0423] The server extracts the room name "Consultation regarding the introduction of AI technology" from the received meeting start request.

[0424] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0425] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0426] 3. Involve virtual personalities in online meetings

[0427] The server transmits profile information of the generated virtual personality to the user terminal.

[0428] The user terminal receives this information and displays the virtual persona in the online meeting room.

[0429] 4. User and virtual personality interact in real time

[0430] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0431] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0432] Interaction between the user and the virtual persona occurs in real time with little delay.

[0433] Specific examples

[0434] Example: Consultation on an AI technology implementation project

[0435] The user logs into the online meeting platform and selects "New Meeting."

[0436] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0437] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0438] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0439] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0440] The above is a specific embodiment for carrying out the present invention. This system enables users to quickly obtain specialized knowledge to efficiently solve specific problems.

[0441] The processing flow will be explained below.

[0442] Step 1: User logs into online meeting platform

[0443] Users use their own terminals to access the online meeting platform.

[0444] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0445] Step 2: Enter the assignment in the room name and start the meeting

[0446] The user opens the "New Meeting" screen of the online meeting platform.

[0447] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0448] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0449] Step 3: The server receives and analyzes the issue

[0450] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0451] The server extracts the room name from the received request.

[0452] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0453] Step 4: The server generates the virtual personality

[0454] The server calls a virtual personality generation engine based on the identified keywords.

[0455] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[0456] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[0457] Step 5: The server sends the virtual personality information to the device.

[0458] The server transmits the profile information of the generated virtual personality to the user's terminal.

[0459] Step 6: Your device displays your virtual persona in online meetings

[0460] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0461] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0462] Step 7: User interacts with virtual persona in real time

[0463] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[0464] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[0465] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[0466] Step 8: User ends meeting and provides feedback

[0467] After obtaining the necessary information, the user performs an operation to end the online meeting.

[0468] The terminal sends a meeting end request to the server.

[0469] The server acknowledges the end of the meeting and stores a log of the session.

[0470] The server displays a post-meeting feedback form on the user terminal.

[0471] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[0472] Example 1

[0473] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0474] In recent years, the convenience of online meetings has increased, and remote dialogue and consultations have become increasingly important. However, when discussing issues that require specific expertise, it is not easy to find the right person to consult with. Furthermore, opportunities to interact with experts in real time are limited, making it difficult for users to quickly obtain expert advice. Therefore, there is a need for a system that allows users to easily obtain expert knowledge and advice.

[0475] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0476] In this invention, the server includes a means for a user to input a task, a means for generating an appropriate virtual character based on the task input by the user, and a means for allowing the generated virtual character to participate in an online conference, thereby enabling the user to interact with the professional virtual character in real time.

[0477] "User" means an individual or organization that utilizes the online conferencing platform to enter tasks and interact with virtual people.

[0478] "Issues" refer to problems that users want to solve or consultation topics, and are entered into the online conference platform.

[0479] A "virtual person" is a character with specific expertise or skills that is generated based on a task entered by the user.

[0480] An "online conference" is an electronic conference that allows a user and a virtual person to interact in real time over the Internet.

[0481] "Real-time" refers to immediate interaction between the user and the virtual persona with little delay.

[0482] "Context information" refers to keywords and phrases obtained by analyzing the content of the task entered by the user, and is data necessary for generating a virtual person.

[0483] "Technical capabilities" refer to the specialized knowledge and skills that the virtual person possesses, and are used to provide appropriate advice and information to the user regarding their problems.

[0484] "Specification information" indicates detailed data about the virtual person, such as the profile and skill set of the generated virtual person.

[0485] The "central device" refers to a server that receives requests from users and performs virtual person generation and related processing.

[0486] "Terminal" refers to the device (e.g., PC, smartphone, tablet) that a user uses to access an online meeting and interact with a virtual person.

[0487] The present invention provides a system that allows users to use an online conference platform and input a specific topic to generate an appropriate virtual person and have them participate in the online conference, enabling users to obtain expert knowledge and advice effectively and efficiently.

[0488] System Configuration

[0489] The system consists of the following main components:

[0490] 1. User terminal: A device used by users to start online meetings and enter tasks. This includes PCs, smartphones, tablets, etc.

[0491] 2. Server: A central device that receives the tasks entered by users, analyzes them, generates virtual people, and allows them to participate in online meetings.

[0492] 3. Virtual Person Generation Engine: This engine is located on the server and runs an AI model that generates an appropriate virtual person based on the task. It also performs text analysis using an NLP library.

[0493] 4. Online conference platform: Provides an environment for users and generated virtual characters to interact in real time.

[0494] System Operation Overview

[0495] First, the user logs in to the online conference platform using their device. They then open the "New Meeting" screen and enter the problem they want to solve in the room name field. For example, they might enter "Consultation on the introduction of AI technology." Then, they click the "Start" button, and the user's device sends a meeting start request, including the room name, to the server.

[0496] The server extracts the room name from the received meeting start request and analyzes the problem statement. It uses an NLP library to identify important keywords and invokes a virtual person generation engine to generate virtual people with skill sets related to those keywords. For example, for the problem "Consultation on the introduction of AI technology," it generates virtual people of an AI expert and a project manager.

[0497] The server sends the profile information of the generated virtual person to the user's terminal. The user's terminal receives this information and displays the virtual person in an online conference room. The user can then interact with the displayed virtual person and receive expert advice and suggestions. The interaction between the user and the virtual person is real-time, with almost no delay.

[0498] Specific examples

[0499] As a concrete example, a user logs in to an online conference platform and selects "New Meeting." He enters "Consultation on the Implementation of AI Technology" as the room name and sends a request to start a meeting to the server. The server analyzes the room name and uses a virtual person generation engine to generate virtual characters of an AI expert and project manager with the required skills. The profile information of the generated virtual characters is sent to the user's device, which then displays the virtual characters in the online conference room. The user can then converse with the virtual characters in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[0500] Examples of prompt statements

[0501] For example, you can input the following prompts into a generative AI model:

[0502] "I would like to consult with you regarding the implementation of AI technology. Please generate a virtual person with the appropriate expertise."

[0503] Using this prompt, the system can generate a virtual person best suited to the task and have them participate in the online meeting. By using this system, users can quickly acquire specialized knowledge.

[0504] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0505] Step 1:

[0506] A user logs in to an online meeting platform and opens the "New Meeting" screen. The user enters the topic in the room name field. For example, they enter "Consultation on the introduction of AI technology" and clicks the "Start" button. This creates a meeting start request that includes the room name.

[0507] Input: The user inputs the task and presses the "Start" button. The input data includes the room name "Consultation on the introduction of AI technology."

[0508] Output: A conference start request is generated and sent to the server.

[0509] Step 2:

[0510] The server receives a conference start request from a user terminal. This request includes the room name entered by the user. The server extracts the room name and performs text analysis.

[0511] Input: Meeting start request (room name "Consultation on the introduction of AI technology")

[0512] Output: Keywords used for analysis (e.g., AI technology, introduction)

[0513] Specific operation: The server extracts the room name from the request received in JSON data format and performs text analysis using an NLP library.

[0514] Step 3:

[0515] The server then activates a virtual person generation engine based on the analyzed keywords to generate appropriate virtual people, such as an AI expert and project manager who correspond to the analyzed keywords "AI technology" and "implementation."

[0516] Input: Analyzed keywords (e.g., AI technology, introduction)

[0517] Output: Profile information of the generated virtual person (e.g. details of an AI expert, project manager)

[0518] Specific operation: The server calls the virtual person generation engine API, passes the necessary parameters (keywords) and requests the generation of a virtual person. It then obtains the profile information of the generated virtual person.

[0519] Step 4:

[0520] The server sends the profile information of the generated virtual person to the user terminal as an HTTP response.

[0521] Input: Profile information of the generated virtual persona (e.g. AI expert, project manager details)

[0522] Output: HTTP response containing the virtual person's profile information

[0523] Specific operation: The server constructs profile information in JSON format and sends it to the user device as an HTTP response.

[0524] Step 5:

[0525] The user terminal receives the profile information of the virtual person from the server and displays it in the online conference room. The virtual person's avatar and profile are displayed, and the user is ready to start a conversation.

[0526] Input: Virtual person profile information sent from the server

[0527] Output: Avatar and profile of a virtual person displayed in an online meeting room

[0528] Specific operation: The user device parses the received JSON data and visually displays the virtual person in the browser or application UI.

[0529] Step 6:

[0530] Users can ask questions or ask the virtual person for advice in an online conference room, and the virtual person will respond in real time, providing expert advice and suggestions to the user's questions.

[0531] Input: User's question or inquiry

[0532] Output: Real-time response from the virtual person

[0533] How it works: The user types a question through a chat window or voice input, and the virtual person uses a generative AI model to generate a response in real time and returns the result to the user, with almost no latency.

[0534] (Application example 1)

[0535] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] In the past, when a factory robot broke down, an engineer had to go to the site and troubleshoot manually. However, this method required time and effort, and could prolong the production line downtime. Another problem was that engineers were not familiar with all causes of the failure, making it difficult to quickly obtain appropriate advice. Therefore, the objective of this invention is to provide a system that efficiently diagnoses factory robot failures and provides prompt support to engineers.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0538] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for the generated virtual personality to participate in an online meeting, means for real-time interaction between the user and the virtual personality, means for inputting a fault symptom of a factory robot, and means for generating a virtual engineer based on the fault symptom, thereby enabling the virtual engineer to assist in fault diagnosis of the factory robot in real time.

[0539] The "means for users to input problems" refers to a device or program that provides an interface that allows users such as engineers to input specific failure symptoms and problems.

[0540] The "means for generating an appropriate virtual personality" is a system that creates a virtual expert with a relevant skill set based on an input task.

[0541] The "means for allowing a generated virtual personality to participate in an online meeting" is a technology that allows a virtual personality to appear in an online meeting in real time and interact with a user.

[0542] "Means for real-time dialogue between a user and a virtual personality" refers to an interactive system in which a generated virtual personality can immediately respond to questions entered by a user.

[0543] The "means for inputting the malfunction symptoms of the factory robot" is an input device or software that allows an engineer to input specific malfunctions or problems related to the factory robot.

[0544] The "means for generating a virtual engineer based on a failure symptom" is a system that generates a virtual mechanical engineer with specialized knowledge in accordance with the inputted failure symptom of a robot.

[0545] "Means for analyzing keywords related to the task entered by the user" refers to technology that extracts and recognizes important words and phrases from the text entered by the user.

[0546] The "means for selecting and generating skilled virtual personalities" is a system that identifies and generates virtual experts with specialized abilities that correspond to the analyzed keywords.

[0547] The "means for the server to transmit profile information of the generated virtual personality to the terminal" is a system that transmits detailed information about the virtual expert created by the server to the user's terminal.

[0548] The "means for a terminal to display a virtual personality in an online meeting" is a technique by which a user's device displays a generated virtual expert on an online meeting screen.

[0549] "A means of supporting the fault diagnosis of factory robots in real time" is a technology in which a virtual engineer provides engineers with immediate fault diagnosis and repair advice.

[0550] The system for realizing this invention is composed of a user terminal, a server, a virtual personality generation engine, and an online meeting platform. As a specific system for supporting fault diagnosis of factory robots, the following components are used in combination.

[0551] 1. User device

[0552] The system uses tablets or smart glasses that allow users to input malfunction symptoms and specific problems of factory robots. Users input malfunction symptoms in text format through a dedicated interface.

[0553] 2. Server

[0554] The server receives and analyzes the failure symptoms sent from the user's device. It uses advanced natural language processing technology to extract important keywords from the input text. The hardware used is a high-performance Linux-based server, and the software uses generative AI models such as GPT-4.

[0555] 3. Virtual personality generation engine

[0556] The virtual personality generation engine installed on the server generates appropriate virtual engineers based on keywords extracted by natural language processing. The virtual personality generation engine creates virtual engineers with specialized skill sets, such as mechanical engineers or electrical control experts.

[0557] 4. Online Meeting Platform

[0558] The user and the generated virtual engineer interact in real time through an online meeting platform. The virtual engineer responds immediately to questions entered by the user and provides expert advice and suggestions. WebRTC and Zoom APIs are used.

[0559] Specifically, when a user types "robot arm malfunctions" into a tablet, the server receives and analyzes this text. The server extracts keywords such as "robot arm" and "malfunction" and passes them to the virtual personality generation engine. The virtual personality generation engine generates a virtual engineer specializing in robot arms based on these keywords. The generated virtual engineer participates in an online meeting room and assists the user in fault diagnosis while interacting in real time with the user.

[0560] As an example of a prompt sentence, by entering "Please tell me about the situation when the robot arm has stopped moving," the virtual engineer will immediately provide advice such as "The robot arm is malfunctioning. The symptoms are that it does not move even when instructed, or it moves very slowly. Please tell me the possible causes of the malfunction and how to deal with it."

[0561] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0562] Step 1:

[0563] A user accesses the online meeting platform on a user device such as a tablet and opens a screen to start a new meeting. The user enters a specific malfunction symptom, such as "the robot arm is not working properly," into the input field and clicks the "Start" button. Input: Malfunction symptom, Output: Meeting start request.

[0564] Step 2:

[0565] The terminal sends a meeting start request to the server. The request includes the problem (fault symptom) entered by the user. Input: Meeting start request, Output: Request data.

[0566] Step 3:

[0567] The server analyzes the received meeting start request. The server receives the failure symptom text (e.g., "Robot arm malfunction") and uses natural language processing technology to extract important keywords (e.g., "robot arm," "malfunction"). Input: Request data, Output: Extracted keywords.

[0568] Step 4:

[0569] The server passes the extracted keywords to the virtual personality generation engine, which generates a virtual engineer with related skills (e.g., an engineer specializing in robotic arms) based on these keywords. Input: extracted keywords, Output: virtual engineer.

[0570] Step 5:

[0571] The server sends the profile information and setting information of the created virtual engineer to the user terminal. The user terminal receives this information and displays the virtual engineer in the online meeting room. Input: Virtual engineer profile information, Output: Display data.

[0572] Step 6:

[0573] Users can ask specific questions or ask for advice to a virtual engineer displayed in an online meeting room. The virtual engineer responds immediately and provides expert advice and suggestions to the user's questions. Input: User's question, Output: Virtual engineer's response.

[0574] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0575] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system enables users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[0576] Overall system overview

[0577] The system consists of the following main components:

[0578] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0579] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0580] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0581] 4. Emotion Engine: An engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses accordingly.

[0582] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0583] System operation flow

[0584] 1. A user enters an assignment and starts an online meeting

[0585] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0586] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0587] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0588] 2. The server analyzes the task and generates a virtual personality

[0589] The server extracts the room name from the received start meeting request.

[0590] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0591] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0592] 3. The server sends the virtual personality's profile information to the device.

[0593] The server transmits the profile information (e.g., name, role, expertise) of the generated virtual personality to the user terminal.

[0594] 4. The device displays a virtual personality in online meetings

[0595] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0596] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0597] 5. Real-time interaction between the user and the virtual persona

[0598] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0599] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0600] 6. Emotional engine for analyzing user emotions and adjusting responses

[0601] The emotion engine analyzes the user's voice and text data to identify their emotions (e.g., joy, anger, anxiety).

[0602] The emotion engine adjusts the virtual personality's responses based on the analysis results, providing appropriate advice and assistance according to the user's emotions.

[0603] The virtual personality empathizes with the user's feelings and responds accordingly to the situation.

[0604] Specific examples

[0605] Example: Consultation on AI technology implementation projects and emotion recognition

[0606] The user logs into the online meeting platform and selects "New Meeting."

[0607] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0608] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0609] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0610] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0611] The emotion engine analyzes the user's emotions from their voice and text and determines that the emotion is anxiety.

[0612] Based on the analysis results, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're here to support you through this step of the project.").

[0613] In this way, the system of the present invention not only efficiently solves the user's specific problems but also provides appropriate responses according to the user's emotions, allowing the user to have a more effective and satisfying online meeting experience.

[0614] The processing flow will be explained below.

[0615] Step 1: User logs into online meeting platform

[0616] Users use their own terminals to access the online meeting platform.

[0617] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0618] Step 2: Enter the assignment in the room name and start the meeting

[0619] The user opens the "New Meeting" screen of the online meeting platform.

[0620] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0621] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0622] Step 3: The server receives and analyzes the issue

[0623] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0624] The server extracts the room name from the received request.

[0625] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0626] Step 4: The server generates the virtual personality

[0627] The server calls a virtual personality generation engine based on the identified keywords.

[0628] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[0629] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[0630] Step 5: The server sends the virtual personality information to the device.

[0631] The server transmits the profile information of the generated virtual personality to the user's terminal.

[0632] Step 6: Your device displays your virtual persona in online meetings

[0633] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0634] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0635] Step 7: User interacts with virtual persona in real time

[0636] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[0637] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[0638] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[0639] Step 8: The emotion engine analyzes the user's emotions

[0640] The emotion engine (in the server) collects user voice and text data.

[0641] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., joy, anger, anxiety).

[0642] Step 9: The virtual personality adjusts responses based on emotions

[0643] The emotion engine provides information about the analyzed emotional state to the virtual personality.

[0644] The virtual personality adjusts the tone and content of its responses depending on the user's emotional state (e.g., making reassuring statements to an anxious user).

[0645] Step 10: User ends meeting and provides feedback

[0646] After obtaining the necessary information, the user performs an operation to end the online meeting.

[0647] The terminal sends a meeting end request to the server.

[0648] The server acknowledges the end of the meeting and stores a log of the session.

[0649] The server displays a post-meeting feedback form on the user terminal.

[0650] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[0651] Example 2

[0652] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0653] Modern online meeting systems make it difficult for users to obtain effective support for issues that require specialized knowledge. They also fail to provide responses that take users' feelings into consideration, which can lead to communication issues. As a result, users find it difficult to have a satisfying meeting experience.

[0654] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0655] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for having the generated virtual personality participate in an online meeting, means for analyzing the user's emotions, and means for adjusting the virtual personality's responses based on the emotion analysis. This allows the user to receive effective and efficient support from a virtual personality with specialized knowledge, and further allows the user to receive empathetic responses according to the user's emotions.

[0656] "User" refers to a person who uses the online meeting platform to enter an issue and receive assistance.

[0657] "Issue" refers to a specific problem or issue that a user seeks to resolve.

[0658] A "virtual personality" refers to a virtual character generated using AI technology that has specific skills or expertise.

[0659] "Online meeting" refers to a virtual conference environment in which users and virtual personalities can interact in real time over the Internet.

[0660] An "emotion engine" refers to technology that analyzes a user's voice and text data to identify the user's emotions.

[0661] "Profile information" refers to basic information such as the virtual personality's name, role, and expertise.

[0662] A "server" refers to a remote computer system that receives requests from users and generates virtual personalities and analyzes emotions based on those requests.

[0663] "Terminal" refers to the device (e.g., PC, smartphone) used by a User to access the online meeting platform.

[0664] "Generative AI model" refers to artificial intelligence technology that generates virtual personalities and responses based on input data.

[0665] A "prompt sentence" refers to an instruction sentence input to a generative AI model.

[0666] A "natural language processing engine" refers to a technology that extracts and analyzes important keywords from text entered by a user.

[0667] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system allows users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[0668] The system consists of the following main components:

[0669] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0670] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0671] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model (e.g., GPT-3) that generates an appropriate virtual personality based on the task.

[0672] 4. Emotion engine: An engine that analyzes emotions from the user's voice or text and adjusts the virtual personality's responses accordingly (e.g., IBM Watson Tone Analyzer).

[0673] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0674] Example of operation

[0675] Example: Consultation on AI technology implementation projects and emotion recognition

[0676] A user logs into an online meeting platform and selects "New Meeting." They enter "Consultation on the Implementation of AI Technology" as the room name and send a request to start a meeting to the server. The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities for an AI expert and project manager with the required skills.

[0677] The server then sends the profile information of the virtual personality to the user's device, which then displays the virtual personality in an online meeting room. The user can then converse with the virtual personality in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[0678] The emotion engine analyzes the user's voice and text to determine if the emotion is anxiety. Based on this analysis, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with this step of the project.").

[0679] Prompt Sentence Examples

[0680] Below are some example prompts for the emotion engine to use to analyze user sentiment and tailor responses to the generative AI model:

[0681] "Tell us how a virtual personality responds when it recognizes that the user is anxious."

[0682] "Please demonstrate how the virtual personality can empathize with the user's anger as analyzed by the emotion engine."

[0683] To implement this invention, a user device accesses the online meeting platform using a browser or a dedicated application, and the server receives and analyzes requests from users. The virtual personality generation engine and emotion engine use dedicated AI models (e.g., GPT-3, IBM Watson Tone Analyzer) that run on the server. This allows users to receive emotionally sensitive responses while solving their problems.

[0684] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0685] Step 1:

[0686] A user enters an assignment and starts an online meeting.

[0687] Users access the online meeting platform using a browser or a dedicated application. They enter their username and password on the login screen and click the "Login" button. After logging in, they open the "New Meeting" screen and enter the problem they want to solve in the room name field. The entered room name is sent from the device to the server in JSON format as a meeting start request. The input data is the content of the problem, and the output is a request to the server.

[0688] Step 2:

[0689] The server analyzes the task and generates a virtual personality.

[0690] The server extracts the room name from the received meeting start request. Based on the room name, the server uses a natural language processing engine to analyze the required keywords (e.g., "AI technology," "implementation"). Using the analyzed keywords as input, the server invokes a virtual personality generation engine (e.g., GPT-3) to generate a virtual personality with the associated skill set (e.g., AI expert, project manager). The output is the virtual personality's profile information.

[0691] Step 3:

[0692] The server transmits the profile information of the virtual personality to the terminal.

[0693] The server sends the profile information of the generated virtual personality to the user's device in JSON format. The profile information includes the virtual personality's name, role, and expertise. The input data is the result of the virtual personality generation, and the output is the transmission of the profile information to the user's device.

[0694] Step 4:

[0695] The device displays a virtual persona in online meetings.

[0696] The user device analyzes the received virtual personality profile information and displays it on the screen of the online meeting room. The virtual personality then gives an initial greeting and introduces itself. The input data is the profile information, and the output is the displayed virtual personality.

[0697] Step 5:

[0698] The user and the virtual persona interact in real time.

[0699] The user asks the virtual persona a question through a chat box or voice input (e.g., "What is the first step in implementing AI?"). The virtual persona receives the user's question and generates a response using a generative AI model (e.g., "We recommend starting by defining the technical requirements."). The input data is the user's question, and the output is the virtual persona's response.

[0700] Step 6:

[0701] Emotion engine analyzes user emotions and adjusts responses.

[0702] The emotion engine analyzes the user's voice and text data to identify the user's emotions. For example, if the voice data indicates that the user is feeling anxious, the emotion engine communicates the analysis result to the virtual personality. The virtual personality then provides reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with each step of this project."). The input data is the user's emotional information, and the output is the adjusted response of the virtual personality.

[0703] Through the above steps, the system can provide users with specialized knowledge and respond in a way that takes into consideration the user's feelings.

[0704] (Application example 2)

[0705] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Traditional customer support in brick-and-mortar stores often lacks staff on-site to address specific issues, making it difficult to respond quickly to customer needs. Furthermore, traditional support systems are unable to respond in a way that takes into account the customer's emotions, making it difficult to improve customer satisfaction. For this reason, there is a demand for personalized experiences that address customer issues and take their emotions into consideration.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for displaying the generated virtual personality in a physical store and providing customer support, means for analyzing the user's emotions and adjusting the virtual personality's response, and means for conducting a real-time dialogue between the user and the virtual personality. This makes it possible to quickly respond to a user's specific task using an appropriate virtual personality even in a physical store, and to provide personalized services according to the user's emotions.

[0708] A "user" is a person who uses the system to input tasks and interact with a virtual personality.

[0709] An "issue" is a specific problem or inquiry that a user wants the system to solve.

[0710] A "virtual personality" is a digital agent generated by AI that responds appropriately to user challenges.

[0711] A "brick and mortar store" is a commercial establishment that is located in a physical location and can be visited by customers in person.

[0712] "Sentiment analysis" is the process of identifying emotions from a user's voice or text data.

[0713] The "generation means" is a technical means for generating an appropriate virtual personality based on the task input by the user.

[0714] "Display means" refers to the technical means for displaying the generated virtual personality in a physical store or on a terminal.

[0715] "Response adjustment means" refers to a technical means for adjusting the response of a virtual personality based on the results of emotion analysis.

[0716] "Real-time interaction means" refers to a technical means for allowing a user and a virtual personality to interact in real time.

[0717] This invention relates to a system that generates an appropriate virtual personality by having the user input a specific problem, and provides customer support in a physical store. Furthermore, it can perform emotion analysis and adjust the virtual personality's response according to the user's emotions. The entire system is composed of the following main components:

[0718] Hardware and Software

[0719] Hardware:

[0720] iOS / Android smartphones

[0721] software:

[0722] iOS / Android applications (Swift / Kotlin)

[0723] Server (AWS EC2, Amazon RDS)

[0724] AI models (TensorFlow, PyTorch)

[0725] Analysis engine (Amazon Comprehend, Google Cloud Natural Language)

[0726] System configuration

[0727] 1. User Device:

[0728] It is a device that allows users to input tasks and interact with virtual personalities.

[0729] A smartphone application is installed and users can enter assignments through this application.

[0730] 2. Server:

[0731] It is a central server that receives and analyzes user-entered tasks to generate and display virtual personalities.

[0732] The server is equipped with an AI model and an emotion analysis engine, enabling advanced data processing.

[0733] 3. Virtual Personality Generation Engine:

[0734] It is an engine that runs an AI model located on a server and generates an appropriate virtual personality based on the user's challenges.

[0735] 4. Sentiment Analysis Engine:

[0736] It is an engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses based on that.

[0737] Application Examples

[0738] Example 1: Fashion coordination consultation

[0739] The user types into the smartphone app, "Please tell me how to coordinate this outfit."

[0740] The server analyzes the input and generates a virtual personality of a fashion expert.

[0741] The virtual expert appears as a virtual display in a physical store and greets the user with, "Hello, I'm a fashion expert. What outfit would you like to wear today?"

[0742] When a user asks, "Which top goes well with this skirt?", the emotion analysis engine analyzes the emotions in the user's voice and text, and if it determines that the user looks anxious, the virtual expert will respond in a gentle tone, "Don't worry, just relax and choose. I recommend this top."

[0743] Example 2: Shopping assistance

[0744] A user types "Tell me about the new smartphone" into a smartphone app.

[0745] The server analyzes the input and generates a virtual personality of a technical expert.

[0746] The virtual expert appears on a display in a physical store and speaks to the user, saying, "Hello, I'm a technology expert. I'd like to tell you about our new smartphone."

[0747] If a user asks, "What's the battery life?" and the sentiment analysis engine determines that the user is excited, the virtual expert will respond enthusiastically, "This phone has excellent battery life, lasting more than a day with normal use."

[0748] As described above, by combining virtual personality generation and emotion analysis, this system can provide customers with fast and personalized support in physical stores, improving customer satisfaction and contributing to sales promotion in physical stores.

[0749] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0750] Step 1:

[0751] A smartphone application is used as a means for users to input tasks. The user launches the smartphone application, enters the specific task they wish to solve (e.g., "Please tell me how to coordinate this outfit") in the text field on the "Task Input" screen, and taps the "Submit" button. This task input data is sent to the server. The input data is the task content in text format.

[0752] Step 2:

[0753] The server receives the assignment input data and begins analysis. The server analyzes the received assignment content and extracts related keywords. For example, keywords such as "clothes" and "coordinate" are extracted. These keywords are analyzed using a text analysis engine (e.g., Amazon Comprehend) to generate an extracted keyword list. The keyword list is the output of the process.

[0754] Step 3:

[0755] The server generates an appropriate virtual personality based on the extracted keyword list. The server invokes a virtual personality generation engine (AI model) to generate a virtual personality (e.g., a fashion expert) with a skill set related to the specified keywords. Profile information (e.g., name, expertise, role) of this virtual personality is generated, and the virtual personality profile information is output.

[0756] Step 4:

[0757] The server transmits the profile information of the generated virtual personality to the user terminal. The user terminal begins displaying the virtual personality based on the received profile information. Specifically, the virtual personality appears as a virtual display within the smartphone application, and an initial greeting is given, such as "Hello, I'm a fashion expert. What kind of outfit would you like to wear today?" The display screen is output.

[0758] Step 5:

[0759] Users interact with the virtual persona in real time by using the chat function or voice input function of the smartphone app to input and send questions (e.g., "Which top goes well with this skirt?"), and the user input data is sent to the system.

[0760] Step 6:

[0761] The server receives new input data from the user and performs emotion analysis. It uses an emotion analysis engine (e.g., Google Cloud Natural Language) to identify emotions (e.g., anxiety, excitement) from the user's voice or text data. The analysis results are output as emotion data.

[0762] Step 7:

[0763] The server generates and adjusts the response content of the virtual personality based on the emotional data. The virtual personality generation engine generates the response content together with the emotional data (e.g., "It's okay, just relax and choose. I recommend this top."), and the adjusted response data is generated. This adjusted response data is the output.

[0764] Step 8:

[0765] The server transmits the adjusted response data to the user terminal, and the user terminal displays or plays the response of the virtual personality in real time based on the received response data, and the user receives the response from the virtual personality and continues the next dialogue.

[0766] Through the above process, even in a physical store, a user can receive a quick and appropriate response from a virtual personality in response to a problem entered by the user, providing personalized service that takes into consideration the user's problem-solving and emotions.

[0767] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0768] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0769] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0770] [Third embodiment]

[0771] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0772] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0773] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0774] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0775] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0776] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0777] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0778] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0779] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0780] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0781] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0782] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0783] The present invention provides a system that allows users to use an online meeting platform and input a specific topic to generate an appropriate virtual personality to participate in the online meeting, enabling users to obtain expertise and advice effectively and efficiently.

[0784] Overall system overview

[0785] The system consists of the following main components:

[0786] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0787] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0788] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0789] 4. Online meeting platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0790] System operation flow

[0791] 1. A user enters an assignment and starts an online meeting

[0792] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0793] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0794] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0795] 2. The server analyzes the task and generates a virtual personality

[0796] The server extracts the room name "Consultation regarding the introduction of AI technology" from the received meeting start request.

[0797] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0798] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0799] 3. Involve virtual personalities in online meetings

[0800] The server transmits profile information of the generated virtual personality to the user terminal.

[0801] The user terminal receives this information and displays the virtual persona in the online meeting room.

[0802] 4. User and virtual personality interact in real time

[0803] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0804] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0805] Interaction between the user and the virtual persona occurs in real time with little delay.

[0806] Specific examples

[0807] Example: Consultation on an AI technology implementation project

[0808] The user logs into the online meeting platform and selects "New Meeting."

[0809] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0810] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0811] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0812] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0813] The above is a specific embodiment for carrying out the present invention. This system enables users to quickly obtain specialized knowledge to efficiently solve specific problems.

[0814] The processing flow will be explained below.

[0815] Step 1: User logs into online meeting platform

[0816] Users use their own terminals to access the online meeting platform.

[0817] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0818] Step 2: Enter the assignment in the room name and start the meeting

[0819] The user opens the "New Meeting" screen of the online meeting platform.

[0820] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0821] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0822] Step 3: The server receives and analyzes the issue

[0823] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0824] The server extracts the room name from the received request.

[0825] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0826] Step 4: The server generates the virtual personality

[0827] The server calls a virtual personality generation engine based on the identified keywords.

[0828] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[0829] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[0830] Step 5: The server sends the virtual personality information to the device.

[0831] The server transmits the profile information of the generated virtual personality to the user's terminal.

[0832] Step 6: Your device displays your virtual persona in online meetings

[0833] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0834] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0835] Step 7: User interacts with virtual persona in real time

[0836] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[0837] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[0838] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[0839] Step 8: User ends meeting and provides feedback

[0840] After obtaining the necessary information, the user performs an operation to end the online meeting.

[0841] The terminal sends a meeting end request to the server.

[0842] The server acknowledges the end of the meeting and stores a log of the session.

[0843] The server displays a post-meeting feedback form on the user terminal.

[0844] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[0845] Example 1

[0846] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0847] In recent years, the convenience of online meetings has increased, and remote dialogue and consultations have become increasingly important. However, when discussing issues that require specific expertise, it is not easy to find the right person to consult with. Furthermore, opportunities to interact with experts in real time are limited, making it difficult for users to quickly obtain expert advice. Therefore, there is a need for a system that allows users to easily obtain expert knowledge and advice.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0849] In this invention, the server includes a means for a user to input a task, a means for generating an appropriate virtual character based on the task input by the user, and a means for allowing the generated virtual character to participate in an online conference, thereby enabling the user to interact with the professional virtual character in real time.

[0850] "User" means an individual or organization that utilizes the online conferencing platform to enter tasks and interact with virtual people.

[0851] "Issues" refer to problems that users want to solve or consultation topics, and are entered into the online conference platform.

[0852] A "virtual person" is a character with specific expertise or skills that is generated based on a task entered by the user.

[0853] An "online conference" is an electronic conference that allows a user and a virtual person to interact in real time over the Internet.

[0854] "Real-time" refers to immediate interaction between the user and the virtual persona with little delay.

[0855] "Context information" refers to keywords and phrases obtained by analyzing the content of the task entered by the user, and is data necessary for generating a virtual person.

[0856] "Technical capabilities" refer to the specialized knowledge and skills that the virtual person possesses, and are used to provide appropriate advice and information to the user regarding their problems.

[0857] "Specification information" indicates detailed data about the virtual person, such as the profile and skill set of the generated virtual person.

[0858] The "central device" refers to a server that receives requests from users and performs virtual person generation and related processing.

[0859] "Terminal" refers to the device (e.g., PC, smartphone, tablet) that a user uses to access an online meeting and interact with a virtual person.

[0860] The present invention provides a system that allows users to use an online conference platform and input a specific topic to generate an appropriate virtual person and have them participate in the online conference, enabling users to obtain expert knowledge and advice effectively and efficiently.

[0861] System Configuration

[0862] The system consists of the following main components:

[0863] 1. User terminal: A device used by users to start online meetings and enter tasks. This includes PCs, smartphones, tablets, etc.

[0864] 2. Server: A central device that receives the tasks entered by users, analyzes them, generates virtual people, and allows them to participate in online meetings.

[0865] 3. Virtual Person Generation Engine: This engine is located on the server and runs an AI model that generates an appropriate virtual person based on the task. It also performs text analysis using an NLP library.

[0866] 4. Online conference platform: Provides an environment for users and generated virtual characters to interact in real time.

[0867] System Operation Overview

[0868] First, the user logs in to the online conference platform using their device. They then open the "New Meeting" screen and enter the problem they want to solve in the room name field. For example, they might enter "Consultation on the introduction of AI technology." Then, they click the "Start" button, and the user's device sends a meeting start request, including the room name, to the server.

[0869] The server extracts the room name from the received meeting start request and analyzes the problem statement. It uses an NLP library to identify important keywords and invokes a virtual person generation engine to generate virtual people with skill sets related to those keywords. For example, for the problem "Consultation on the introduction of AI technology," it generates virtual people of an AI expert and a project manager.

[0870] The server sends the profile information of the generated virtual person to the user's terminal. The user's terminal receives this information and displays the virtual person in an online conference room. The user can then interact with the displayed virtual person and receive expert advice and suggestions. The interaction between the user and the virtual person is real-time, with almost no delay.

[0871] Specific examples

[0872] As a concrete example, a user logs in to an online conference platform and selects "New Meeting." He enters "Consultation on the Implementation of AI Technology" as the room name and sends a request to start a meeting to the server. The server analyzes the room name and uses a virtual person generation engine to generate virtual characters of an AI expert and project manager with the required skills. The profile information of the generated virtual characters is sent to the user's device, which then displays the virtual characters in the online conference room. The user can then converse with the virtual characters in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[0873] Examples of prompt statements

[0874] For example, you can input the following prompts into a generative AI model:

[0875] "I would like to consult with you regarding the implementation of AI technology. Please generate a virtual person with the appropriate expertise."

[0876] Using this prompt, the system can generate a virtual person best suited to the task and have them participate in the online meeting. By using this system, users can quickly acquire specialized knowledge.

[0877] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0878] Step 1:

[0879] A user logs in to an online meeting platform and opens the "New Meeting" screen. The user enters the topic in the room name field. For example, they enter "Consultation on the introduction of AI technology" and clicks the "Start" button. This creates a meeting start request that includes the room name.

[0880] Input: The user inputs the task and presses the "Start" button. The input data includes the room name "Consultation on the introduction of AI technology."

[0881] Output: A conference start request is generated and sent to the server.

[0882] Step 2:

[0883] The server receives a conference start request from a user terminal. This request includes the room name entered by the user. The server extracts the room name and performs text analysis.

[0884] Input: Meeting start request (room name "Consultation on the introduction of AI technology")

[0885] Output: Keywords used for analysis (e.g., AI technology, introduction)

[0886] Specific operation: The server extracts the room name from the request received in JSON data format and performs text analysis using an NLP library.

[0887] Step 3:

[0888] The server then activates a virtual person generation engine based on the analyzed keywords to generate appropriate virtual people, such as an AI expert and project manager who correspond to the analyzed keywords "AI technology" and "implementation."

[0889] Input: Analyzed keywords (e.g., AI technology, introduction)

[0890] Output: Profile information of the generated virtual person (e.g. details of an AI expert, project manager)

[0891] Specific operation: The server calls the virtual person generation engine API, passes the necessary parameters (keywords) and requests the generation of a virtual person. It then obtains the profile information of the generated virtual person.

[0892] Step 4:

[0893] The server sends the profile information of the generated virtual person to the user terminal as an HTTP response.

[0894] Input: Profile information of the generated virtual persona (e.g. AI expert, project manager details)

[0895] Output: HTTP response containing the virtual person's profile information

[0896] Specific operation: The server constructs profile information in JSON format and sends it to the user device as an HTTP response.

[0897] Step 5:

[0898] The user terminal receives the profile information of the virtual person from the server and displays it in the online conference room. The virtual person's avatar and profile are displayed, and the user is ready to start a conversation.

[0899] Input: Virtual person profile information sent from the server

[0900] Output: Avatar and profile of a virtual person displayed in an online meeting room

[0901] Specific operation: The user device parses the received JSON data and visually displays the virtual person in the browser or application UI.

[0902] Step 6:

[0903] Users can ask questions or ask the virtual person for advice in an online conference room, and the virtual person will respond in real time, providing expert advice and suggestions to the user's questions.

[0904] Input: User's question or inquiry

[0905] Output: Real-time response from the virtual person

[0906] How it works: The user types a question through a chat window or voice input, and the virtual person uses a generative AI model to generate a response in real time and returns the result to the user, with almost no latency.

[0907] (Application example 1)

[0908] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0909] In the past, when a factory robot broke down, an engineer had to go to the site and troubleshoot manually. However, this method required time and effort, and could prolong the production line downtime. Another problem was that engineers were not familiar with all causes of the failure, making it difficult to quickly obtain appropriate advice. Therefore, the objective of this invention is to provide a system that efficiently diagnoses factory robot failures and provides prompt support to engineers.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0911] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for the generated virtual personality to participate in an online meeting, means for real-time interaction between the user and the virtual personality, means for inputting a fault symptom of a factory robot, and means for generating a virtual engineer based on the fault symptom, thereby enabling the virtual engineer to assist in fault diagnosis of the factory robot in real time.

[0912] The "means for users to input problems" refers to a device or program that provides an interface that allows users such as engineers to input specific failure symptoms and problems.

[0913] The "means for generating an appropriate virtual personality" is a system that creates a virtual expert with a relevant skill set based on an input task.

[0914] The "means for allowing a generated virtual personality to participate in an online meeting" is a technology that allows a virtual personality to appear in an online meeting in real time and interact with a user.

[0915] "Means for real-time dialogue between a user and a virtual personality" refers to an interactive system in which a generated virtual personality can immediately respond to questions entered by a user.

[0916] The "means for inputting the malfunction symptoms of the factory robot" is an input device or software that allows an engineer to input specific malfunctions or problems related to the factory robot.

[0917] The "means for generating a virtual engineer based on a failure symptom" is a system that generates a virtual mechanical engineer with specialized knowledge in accordance with the inputted failure symptom of a robot.

[0918] "Means for analyzing keywords related to the task entered by the user" refers to technology that extracts and recognizes important words and phrases from the text entered by the user.

[0919] The "means for selecting and generating skilled virtual personalities" is a system that identifies and generates virtual experts with specialized abilities that correspond to the analyzed keywords.

[0920] The "means for the server to transmit profile information of the generated virtual personality to the terminal" is a system that transmits detailed information about the virtual expert created by the server to the user's terminal.

[0921] The "means for a terminal to display a virtual personality in an online meeting" is a technique by which a user's device displays a generated virtual expert on an online meeting screen.

[0922] "A means of supporting the fault diagnosis of factory robots in real time" is a technology in which a virtual engineer provides engineers with immediate fault diagnosis and repair advice.

[0923] The system for realizing this invention is composed of a user terminal, a server, a virtual personality generation engine, and an online meeting platform. As a specific system for supporting fault diagnosis of factory robots, the following components are used in combination.

[0924] 1. User device

[0925] The system uses tablets or smart glasses that allow users to input malfunction symptoms and specific problems of factory robots. Users input malfunction symptoms in text format through a dedicated interface.

[0926] 2. Server

[0927] The server receives and analyzes the failure symptoms sent from the user's device. It uses advanced natural language processing technology to extract important keywords from the input text. The hardware used is a high-performance Linux-based server, and the software uses generative AI models such as GPT-4.

[0928] 3. Virtual personality generation engine

[0929] The virtual personality generation engine installed on the server generates appropriate virtual engineers based on keywords extracted by natural language processing. The virtual personality generation engine creates virtual engineers with specialized skill sets, such as mechanical engineers or electrical control experts.

[0930] 4. Online Meeting Platform

[0931] The user and the generated virtual engineer interact in real time through an online meeting platform. The virtual engineer responds immediately to questions entered by the user and provides expert advice and suggestions. WebRTC and Zoom APIs are used.

[0932] Specifically, when a user types "robot arm malfunctions" into a tablet, the server receives and analyzes this text. The server extracts keywords such as "robot arm" and "malfunction" and passes them to the virtual personality generation engine. The virtual personality generation engine generates a virtual engineer specializing in robot arms based on these keywords. The generated virtual engineer participates in an online meeting room and assists the user in fault diagnosis while interacting in real time with the user.

[0933] As an example of a prompt sentence, by entering "Please tell me about the situation when the robot arm has stopped moving," the virtual engineer will immediately provide advice such as "The robot arm is malfunctioning. The symptoms are that it does not move even when instructed, or it moves very slowly. Please tell me the possible causes of the malfunction and how to deal with it."

[0934] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0935] Step 1:

[0936] A user accesses the online meeting platform on a user device such as a tablet and opens a screen to start a new meeting. The user enters a specific malfunction symptom, such as "the robot arm is not working properly," into the input field and clicks the "Start" button. Input: Malfunction symptom, Output: Meeting start request.

[0937] Step 2:

[0938] The terminal sends a meeting start request to the server. The request includes the problem (fault symptom) entered by the user. Input: Meeting start request, Output: Request data.

[0939] Step 3:

[0940] The server analyzes the received meeting start request. The server receives the failure symptom text (e.g., "Robot arm malfunction") and uses natural language processing technology to extract important keywords (e.g., "robot arm," "malfunction"). Input: Request data, Output: Extracted keywords.

[0941] Step 4:

[0942] The server passes the extracted keywords to the virtual personality generation engine, which generates a virtual engineer with related skills (e.g., an engineer specializing in robotic arms) based on these keywords. Input: extracted keywords, Output: virtual engineer.

[0943] Step 5:

[0944] The server sends the profile information and setting information of the created virtual engineer to the user terminal. The user terminal receives this information and displays the virtual engineer in the online meeting room. Input: Virtual engineer profile information, Output: Display data.

[0945] Step 6:

[0946] Users can ask specific questions or ask for advice to a virtual engineer displayed in an online meeting room. The virtual engineer responds immediately and provides expert advice and suggestions to the user's questions. Input: User's question, Output: Virtual engineer's response.

[0947] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0948] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system enables users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[0949] Overall system overview

[0950] The system consists of the following main components:

[0951] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[0952] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[0953] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[0954] 4. Emotion Engine: An engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses accordingly.

[0955] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[0956] System operation flow

[0957] 1. A user enters an assignment and starts an online meeting

[0958] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[0959] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[0960] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[0961] 2. The server analyzes the task and generates a virtual personality

[0962] The server extracts the room name from the received start meeting request.

[0963] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[0964] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[0965] 3. The server sends the virtual personality's profile information to the device.

[0966] The server transmits the profile information (e.g., name, role, expertise) of the generated virtual personality to the user terminal.

[0967] 4. The device displays a virtual personality in online meetings

[0968] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[0969] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[0970] 5. Real-time interaction between the user and the virtual persona

[0971] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[0972] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[0973] 6. Emotional engine for analyzing user emotions and adjusting responses

[0974] The emotion engine analyzes the user's voice and text data to identify their emotions (e.g., joy, anger, anxiety).

[0975] The emotion engine adjusts the virtual personality's responses based on the analysis results, providing appropriate advice and assistance according to the user's emotions.

[0976] The virtual personality empathizes with the user's feelings and responds accordingly to the situation.

[0977] Specific examples

[0978] Example: Consultation on AI technology implementation projects and emotion recognition

[0979] The user logs into the online meeting platform and selects "New Meeting."

[0980] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[0981] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[0982] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[0983] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[0984] The emotion engine analyzes the user's emotions from their voice and text and determines that the emotion is anxiety.

[0985] Based on the analysis results, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're here to support you through this step of the project.").

[0986] In this way, the system of the present invention not only efficiently solves the user's specific problems but also provides appropriate responses according to the user's emotions, allowing the user to have a more effective and satisfying online meeting experience.

[0987] The processing flow will be explained below.

[0988] Step 1: User logs into online meeting platform

[0989] Users use their own terminals to access the online meeting platform.

[0990] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[0991] Step 2: Enter the assignment in the room name and start the meeting

[0992] The user opens the "New Meeting" screen of the online meeting platform.

[0993] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[0994] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[0995] Step 3: The server receives and analyzes the issue

[0996] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[0997] The server extracts the room name from the received request.

[0998] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[0999] Step 4: The server generates the virtual personality

[1000] The server calls a virtual personality generation engine based on the identified keywords.

[1001] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[1002] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[1003] Step 5: The server sends the virtual personality information to the device.

[1004] The server transmits the profile information of the generated virtual personality to the user's terminal.

[1005] Step 6: Your device displays your virtual persona in online meetings

[1006] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[1007] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[1008] Step 7: User interacts with virtual persona in real time

[1009] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[1010] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[1011] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[1012] Step 8: The emotion engine analyzes the user's emotions

[1013] The emotion engine (in the server) collects user voice and text data.

[1014] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., joy, anger, anxiety).

[1015] Step 9: The virtual personality adjusts responses based on emotions

[1016] The emotion engine provides information about the analyzed emotional state to the virtual personality.

[1017] The virtual personality adjusts the tone and content of its responses depending on the user's emotional state (e.g., making reassuring statements to an anxious user).

[1018] Step 10: User ends meeting and provides feedback

[1019] After obtaining the necessary information, the user performs an operation to end the online meeting.

[1020] The terminal sends a meeting end request to the server.

[1021] The server acknowledges the end of the meeting and stores a log of the session.

[1022] The server displays a post-meeting feedback form on the user terminal.

[1023] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[1024] Example 2

[1025] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1026] Modern online meeting systems make it difficult for users to obtain effective support for issues that require specialized knowledge. They also fail to provide responses that take users' feelings into consideration, which can lead to communication issues. As a result, users find it difficult to have a satisfying meeting experience.

[1027] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1028] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for having the generated virtual personality participate in an online meeting, means for analyzing the user's emotions, and means for adjusting the virtual personality's responses based on the emotion analysis. This allows the user to receive effective and efficient support from a virtual personality with specialized knowledge, and further allows the user to receive empathetic responses according to the user's emotions.

[1029] "User" refers to a person who uses the online meeting platform to enter an issue and receive assistance.

[1030] "Issue" refers to a specific problem or issue that a user seeks to resolve.

[1031] A "virtual personality" refers to a virtual character generated using AI technology that has specific skills or expertise.

[1032] "Online meeting" refers to a virtual conference environment in which users and virtual personalities can interact in real time over the Internet.

[1033] An "emotion engine" refers to technology that analyzes a user's voice and text data to identify the user's emotions.

[1034] "Profile information" refers to basic information such as the virtual personality's name, role, and expertise.

[1035] A "server" refers to a remote computer system that receives requests from users and generates virtual personalities and analyzes emotions based on those requests.

[1036] "Terminal" refers to the device (e.g., PC, smartphone) used by a User to access the online meeting platform.

[1037] "Generative AI model" refers to artificial intelligence technology that generates virtual personalities and responses based on input data.

[1038] A "prompt sentence" refers to an instruction sentence input to a generative AI model.

[1039] A "natural language processing engine" refers to a technology that extracts and analyzes important keywords from text entered by a user.

[1040] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system allows users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[1041] The system consists of the following main components:

[1042] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[1043] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[1044] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model (e.g., GPT-3) that generates an appropriate virtual personality based on the task.

[1045] 4. Emotion engine: An engine that analyzes emotions from the user's voice or text and adjusts the virtual personality's responses accordingly (e.g., IBM Watson Tone Analyzer).

[1046] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[1047] Example of operation

[1048] Example: Consultation on AI technology implementation projects and emotion recognition

[1049] A user logs into an online meeting platform and selects "New Meeting." They enter "Consultation on the Implementation of AI Technology" as the room name and send a request to start a meeting to the server. The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities for an AI expert and project manager with the required skills.

[1050] The server then sends the profile information of the virtual personality to the user's device, which then displays the virtual personality in an online meeting room. The user can then converse with the virtual personality in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[1051] The emotion engine analyzes the user's voice and text to determine if the emotion is anxiety. Based on this analysis, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with this step of the project.").

[1052] Prompt Sentence Examples

[1053] Below are some example prompts for the emotion engine to use to analyze user sentiment and tailor responses to the generative AI model:

[1054] "Tell us how a virtual personality responds when it recognizes that the user is anxious."

[1055] "Please demonstrate how the virtual personality can empathize with the user's anger as analyzed by the emotion engine."

[1056] To implement this invention, a user device accesses the online meeting platform using a browser or a dedicated application, and the server receives and analyzes requests from users. The virtual personality generation engine and emotion engine use dedicated AI models (e.g., GPT-3, IBM Watson Tone Analyzer) that run on the server. This allows users to receive emotionally sensitive responses while solving their problems.

[1057] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1058] Step 1:

[1059] A user enters an assignment and starts an online meeting.

[1060] Users access the online meeting platform using a browser or a dedicated application. They enter their username and password on the login screen and click the "Login" button. After logging in, they open the "New Meeting" screen and enter the problem they want to solve in the room name field. The entered room name is sent from the device to the server in JSON format as a meeting start request. The input data is the content of the problem, and the output is a request to the server.

[1061] Step 2:

[1062] The server analyzes the task and generates a virtual personality.

[1063] The server extracts the room name from the received meeting start request. Based on the room name, the server uses a natural language processing engine to analyze the required keywords (e.g., "AI technology," "implementation"). Using the analyzed keywords as input, the server invokes a virtual personality generation engine (e.g., GPT-3) to generate a virtual personality with the associated skill set (e.g., AI expert, project manager). The output is the virtual personality's profile information.

[1064] Step 3:

[1065] The server transmits the profile information of the virtual personality to the terminal.

[1066] The server sends the profile information of the generated virtual personality to the user's device in JSON format. The profile information includes the virtual personality's name, role, and expertise. The input data is the result of the virtual personality generation, and the output is the transmission of the profile information to the user's device.

[1067] Step 4:

[1068] The device displays a virtual persona in online meetings.

[1069] The user device analyzes the received virtual personality profile information and displays it on the screen of the online meeting room. The virtual personality then gives an initial greeting and introduces itself. The input data is the profile information, and the output is the displayed virtual personality.

[1070] Step 5:

[1071] The user and the virtual persona interact in real time.

[1072] The user asks the virtual persona a question through a chat box or voice input (e.g., "What is the first step in implementing AI?"). The virtual persona receives the user's question and generates a response using a generative AI model (e.g., "We recommend starting by defining the technical requirements."). The input data is the user's question, and the output is the virtual persona's response.

[1073] Step 6:

[1074] Emotion engine analyzes user emotions and adjusts responses.

[1075] The emotion engine analyzes the user's voice and text data to identify the user's emotions. For example, if the voice data indicates that the user is feeling anxious, the emotion engine communicates the analysis result to the virtual personality. The virtual personality then provides reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with each step of this project."). The input data is the user's emotional information, and the output is the adjusted response of the virtual personality.

[1076] Through the above steps, the system can provide users with specialized knowledge and respond in a way that takes into consideration the user's feelings.

[1077] (Application example 2)

[1078] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1079] Traditional customer support in brick-and-mortar stores often lacks staff on-site to address specific issues, making it difficult to respond quickly to customer needs. Furthermore, traditional support systems are unable to respond in a way that takes into account the customer's emotions, making it difficult to improve customer satisfaction. For this reason, there is a demand for personalized experiences that address customer issues and take their emotions into consideration.

[1080] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for displaying the generated virtual personality in a physical store and providing customer support, means for analyzing the user's emotions and adjusting the virtual personality's response, and means for conducting a real-time dialogue between the user and the virtual personality. This makes it possible to quickly respond to a user's specific task using an appropriate virtual personality even in a physical store, and to provide personalized services according to the user's emotions.

[1081] A "user" is a person who uses the system to input tasks and interact with a virtual personality.

[1082] An "issue" is a specific problem or inquiry that a user wants the system to solve.

[1083] A "virtual personality" is a digital agent generated by AI that responds appropriately to user challenges.

[1084] A "brick and mortar store" is a commercial establishment that is located in a physical location and can be visited by customers in person.

[1085] "Sentiment analysis" is the process of identifying emotions from a user's voice or text data.

[1086] The "generation means" is a technical means for generating an appropriate virtual personality based on the task input by the user.

[1087] "Display means" refers to the technical means for displaying the generated virtual personality in a physical store or on a terminal.

[1088] "Response adjustment means" refers to a technical means for adjusting the response of a virtual personality based on the results of emotion analysis.

[1089] "Real-time interaction means" refers to a technical means for allowing a user and a virtual personality to interact in real time.

[1090] This invention relates to a system that generates an appropriate virtual personality by having the user input a specific problem, and provides customer support in a physical store. Furthermore, it can perform emotion analysis and adjust the virtual personality's response according to the user's emotions. The entire system is composed of the following main components:

[1091] Hardware and Software

[1092] Hardware:

[1093] iOS / Android smartphones

[1094] software:

[1095] iOS / Android applications (Swift / Kotlin)

[1096] Server (AWS EC2, Amazon RDS)

[1097] AI models (TensorFlow, PyTorch)

[1098] Analysis engine (Amazon Comprehend, Google Cloud Natural Language)

[1099] System configuration

[1100] 1. User Device:

[1101] It is a device that allows users to input tasks and interact with virtual personalities.

[1102] A smartphone application is installed and users can enter assignments through this application.

[1103] 2. Server:

[1104] It is a central server that receives and analyzes user-entered tasks to generate and display virtual personalities.

[1105] The server is equipped with an AI model and an emotion analysis engine, enabling advanced data processing.

[1106] 3. Virtual Personality Generation Engine:

[1107] It is an engine that runs an AI model located on a server and generates an appropriate virtual personality based on the user's challenges.

[1108] 4. Sentiment Analysis Engine:

[1109] It is an engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses based on that.

[1110] Application Examples

[1111] Example 1: Fashion coordination consultation

[1112] The user types into the smartphone app, "Please tell me how to coordinate this outfit."

[1113] The server analyzes the input and generates a virtual personality of a fashion expert.

[1114] The virtual expert appears as a virtual display in a physical store and greets the user with, "Hello, I'm a fashion expert. What outfit would you like to wear today?"

[1115] When a user asks, "Which top goes well with this skirt?", the emotion analysis engine analyzes the emotions in the user's voice and text, and if it determines that the user looks anxious, the virtual expert will respond in a gentle tone, "Don't worry, just relax and choose. I recommend this top."

[1116] Example 2: Shopping assistance

[1117] A user types "Tell me about the new smartphone" into a smartphone app.

[1118] The server analyzes the input and generates a virtual personality of a technical expert.

[1119] The virtual expert appears on a display in a physical store and speaks to the user, saying, "Hello, I'm a technology expert. I'd like to tell you about our new smartphone."

[1120] If a user asks, "What's the battery life?" and the sentiment analysis engine determines that the user is excited, the virtual expert will respond enthusiastically, "This phone has excellent battery life, lasting more than a day with normal use."

[1121] As described above, by combining virtual personality generation and emotion analysis, this system can provide customers with fast and personalized support in physical stores, improving customer satisfaction and contributing to sales promotion in physical stores.

[1122] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1123] Step 1:

[1124] A smartphone application is used as a means for users to input tasks. The user launches the smartphone application, enters the specific task they wish to solve (e.g., "Please tell me how to coordinate this outfit") in the text field on the "Task Input" screen, and taps the "Submit" button. This task input data is sent to the server. The input data is the task content in text format.

[1125] Step 2:

[1126] The server receives the assignment input data and begins analysis. The server analyzes the received assignment content and extracts related keywords. For example, keywords such as "clothes" and "coordinate" are extracted. These keywords are analyzed using a text analysis engine (e.g., Amazon Comprehend) to generate an extracted keyword list. The keyword list is the output of the process.

[1127] Step 3:

[1128] The server generates an appropriate virtual personality based on the extracted keyword list. The server invokes a virtual personality generation engine (AI model) to generate a virtual personality (e.g., a fashion expert) with a skill set related to the specified keywords. Profile information (e.g., name, expertise, role) of this virtual personality is generated, and the virtual personality profile information is output.

[1129] Step 4:

[1130] The server transmits the profile information of the generated virtual personality to the user terminal. The user terminal begins displaying the virtual personality based on the received profile information. Specifically, the virtual personality appears as a virtual display within the smartphone application, and an initial greeting is given, such as "Hello, I'm a fashion expert. What kind of outfit would you like to wear today?" The display screen is output.

[1131] Step 5:

[1132] Users interact with the virtual persona in real time by using the chat function or voice input function of the smartphone app to input and send questions (e.g., "Which top goes well with this skirt?"), and the user input data is sent to the system.

[1133] Step 6:

[1134] The server receives new input data from the user and performs emotion analysis. It uses an emotion analysis engine (e.g., Google Cloud Natural Language) to identify emotions (e.g., anxiety, excitement) from the user's voice or text data. The analysis results are output as emotion data.

[1135] Step 7:

[1136] The server generates and adjusts the response content of the virtual personality based on the emotional data. The virtual personality generation engine generates the response content together with the emotional data (e.g., "It's okay, just relax and choose. I recommend this top."), and the adjusted response data is generated. This adjusted response data is the output.

[1137] Step 8:

[1138] The server transmits the adjusted response data to the user terminal, and the user terminal displays or plays the response of the virtual personality in real time based on the received response data, and the user receives the response from the virtual personality and continues the next dialogue.

[1139] Through the above process, even in a physical store, a user can receive a quick and appropriate response from a virtual personality in response to a problem entered by the user, providing personalized service that takes into consideration the user's problem-solving and emotions.

[1140] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1141] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1142] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1143] [Fourth embodiment]

[1144] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1145] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1146] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1147] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1148] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1149] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1150] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1151] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1152] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1153] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1155] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1156] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1157] The present invention provides a system that allows users to use an online meeting platform and input a specific topic to generate an appropriate virtual personality to participate in the online meeting, enabling users to obtain expertise and advice effectively and efficiently.

[1158] Overall system overview

[1159] The system consists of the following main components:

[1160] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[1161] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[1162] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[1163] 4. Online meeting platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[1164] System operation flow

[1165] 1. A user enters an assignment and starts an online meeting

[1166] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[1167] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[1168] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[1169] 2. The server analyzes the task and generates a virtual personality

[1170] The server extracts the room name "Consultation regarding the introduction of AI technology" from the received meeting start request.

[1171] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[1172] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[1173] 3. Involve virtual personalities in online meetings

[1174] The server transmits profile information of the generated virtual personality to the user terminal.

[1175] The user terminal receives this information and displays the virtual persona in the online meeting room.

[1176] 4. User and virtual personality interact in real time

[1177] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[1178] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[1179] Interaction between the user and the virtual persona occurs in real time with little delay.

[1180] Specific examples

[1181] Example: Consultation on an AI technology implementation project

[1182] The user logs into the online meeting platform and selects "New Meeting."

[1183] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[1184] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[1185] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[1186] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[1187] The above is a specific embodiment for carrying out the present invention. This system enables users to quickly obtain specialized knowledge to efficiently solve specific problems.

[1188] The processing flow will be explained below.

[1189] Step 1: User logs into online meeting platform

[1190] Users use their own terminals to access the online meeting platform.

[1191] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[1192] Step 2: Enter the assignment in the room name and start the meeting

[1193] The user opens the "New Meeting" screen of the online meeting platform.

[1194] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[1195] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[1196] Step 3: The server receives and analyzes the issue

[1197] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[1198] The server extracts the room name from the received request.

[1199] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[1200] Step 4: The server generates the virtual personality

[1201] The server calls a virtual personality generation engine based on the identified keywords.

[1202] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[1203] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[1204] Step 5: The server sends the virtual personality information to the device.

[1205] The server transmits the profile information of the generated virtual personality to the user's terminal.

[1206] Step 6: Your device displays your virtual persona in online meetings

[1207] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[1208] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[1209] Step 7: User interacts with virtual persona in real time

[1210] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[1211] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[1212] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[1213] Step 8: User ends meeting and provides feedback

[1214] After obtaining the necessary information, the user performs an operation to end the online meeting.

[1215] The terminal sends a meeting end request to the server.

[1216] The server acknowledges the end of the meeting and stores a log of the session.

[1217] The server displays a post-meeting feedback form on the user terminal.

[1218] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[1219] Example 1

[1220] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1221] In recent years, the convenience of online meetings has increased, and remote dialogue and consultations have become increasingly important. However, when discussing issues that require specific expertise, it is not easy to find the right person to consult with. Furthermore, opportunities to interact with experts in real time are limited, making it difficult for users to quickly obtain expert advice. Therefore, there is a need for a system that allows users to easily obtain expert knowledge and advice.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1223] In this invention, the server includes a means for a user to input a task, a means for generating an appropriate virtual character based on the task input by the user, and a means for allowing the generated virtual character to participate in an online conference, thereby enabling the user to interact with the professional virtual character in real time.

[1224] "User" means an individual or organization that utilizes the online conferencing platform to enter tasks and interact with virtual people.

[1225] "Issues" refer to problems that users want to solve or consultation topics, and are entered into the online conference platform.

[1226] A "virtual person" is a character with specific expertise or skills that is generated based on a task entered by the user.

[1227] An "online conference" is an electronic conference that allows a user and a virtual person to interact in real time over the Internet.

[1228] "Real-time" refers to immediate interaction between the user and the virtual persona with little delay.

[1229] "Context information" refers to keywords and phrases obtained by analyzing the content of the task entered by the user, and is data necessary for generating a virtual person.

[1230] "Technical capabilities" refer to the specialized knowledge and skills that the virtual person possesses, and are used to provide appropriate advice and information to the user regarding their problems.

[1231] "Specification information" indicates detailed data about the virtual person, such as the profile and skill set of the generated virtual person.

[1232] The "central device" refers to a server that receives requests from users and performs virtual person generation and related processing.

[1233] "Terminal" refers to the device (e.g., PC, smartphone, tablet) that a user uses to access an online meeting and interact with a virtual person.

[1234] The present invention provides a system that allows users to use an online conference platform and input a specific topic to generate an appropriate virtual person and have them participate in the online conference, enabling users to obtain expert knowledge and advice effectively and efficiently.

[1235] System Configuration

[1236] The system consists of the following main components:

[1237] 1. User terminal: A device used by users to start online meetings and enter tasks. This includes PCs, smartphones, tablets, etc.

[1238] 2. Server: A central device that receives the tasks entered by users, analyzes them, generates virtual people, and allows them to participate in online meetings.

[1239] 3. Virtual Person Generation Engine: This engine is located on the server and runs an AI model that generates an appropriate virtual person based on the task. It also performs text analysis using an NLP library.

[1240] 4. Online conference platform: Provides an environment for users and generated virtual characters to interact in real time.

[1241] System Operation Overview

[1242] First, the user logs in to the online conference platform using their device. They then open the "New Meeting" screen and enter the problem they want to solve in the room name field. For example, they might enter "Consultation on the introduction of AI technology." Then, they click the "Start" button, and the user's device sends a meeting start request, including the room name, to the server.

[1243] The server extracts the room name from the received meeting start request and analyzes the problem statement. It uses an NLP library to identify important keywords and invokes a virtual person generation engine to generate virtual people with skill sets related to those keywords. For example, for the problem "Consultation on the introduction of AI technology," it generates virtual people of an AI expert and a project manager.

[1244] The server sends the profile information of the generated virtual person to the user's terminal. The user's terminal receives this information and displays the virtual person in an online conference room. The user can then interact with the displayed virtual person and receive expert advice and suggestions. The interaction between the user and the virtual person is real-time, with almost no delay.

[1245] Specific examples

[1246] As a concrete example, a user logs in to an online conference platform and selects "New Meeting." He enters "Consultation on the Implementation of AI Technology" as the room name and sends a request to start a meeting to the server. The server analyzes the room name and uses a virtual person generation engine to generate virtual characters of an AI expert and project manager with the required skills. The profile information of the generated virtual characters is sent to the user's device, which then displays the virtual characters in the online conference room. The user can then converse with the virtual characters in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[1247] Examples of prompt statements

[1248] For example, you can input the following prompts into a generative AI model:

[1249] "I would like to consult with you regarding the implementation of AI technology. Please generate a virtual person with the appropriate expertise."

[1250] Using this prompt, the system can generate a virtual person best suited to the task and have them participate in the online meeting. By using this system, users can quickly acquire specialized knowledge.

[1251] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1252] Step 1:

[1253] A user logs in to an online meeting platform and opens the "New Meeting" screen. The user enters the topic in the room name field. For example, they enter "Consultation on the introduction of AI technology" and clicks the "Start" button. This creates a meeting start request that includes the room name.

[1254] Input: The user inputs the task and presses the "Start" button. The input data includes the room name "Consultation on the introduction of AI technology."

[1255] Output: A conference start request is generated and sent to the server.

[1256] Step 2:

[1257] The server receives a conference start request from a user terminal. This request includes the room name entered by the user. The server extracts the room name and performs text analysis.

[1258] Input: Meeting start request (room name "Consultation on the introduction of AI technology")

[1259] Output: Keywords used for analysis (e.g., AI technology, introduction)

[1260] Specific operation: The server extracts the room name from the request received in JSON data format and performs text analysis using an NLP library.

[1261] Step 3:

[1262] The server then activates a virtual person generation engine based on the analyzed keywords to generate appropriate virtual people, such as an AI expert and project manager who correspond to the analyzed keywords "AI technology" and "implementation."

[1263] Input: Analyzed keywords (e.g., AI technology, introduction)

[1264] Output: Profile information of the generated virtual person (e.g. details of an AI expert, project manager)

[1265] Specific operation: The server calls the virtual person generation engine API, passes the necessary parameters (keywords) and requests the generation of a virtual person. It then obtains the profile information of the generated virtual person.

[1266] Step 4:

[1267] The server sends the profile information of the generated virtual person to the user terminal as an HTTP response.

[1268] Input: Profile information of the generated virtual persona (e.g. AI expert, project manager details)

[1269] Output: HTTP response containing the virtual person's profile information

[1270] Specific operation: The server constructs profile information in JSON format and sends it to the user device as an HTTP response.

[1271] Step 5:

[1272] The user terminal receives the profile information of the virtual person from the server and displays it in the online conference room. The virtual person's avatar and profile are displayed, and the user is ready to start a conversation.

[1273] Input: Virtual person profile information sent from the server

[1274] Output: Avatar and profile of a virtual person displayed in an online meeting room

[1275] Specific operation: The user device parses the received JSON data and visually displays the virtual person in the browser or application UI.

[1276] Step 6:

[1277] Users can ask questions or ask the virtual person for advice in an online conference room, and the virtual person will respond in real time, providing expert advice and suggestions to the user's questions.

[1278] Input: User's question or inquiry

[1279] Output: Real-time response from the virtual person

[1280] How it works: The user types a question through a chat window or voice input, and the virtual person uses a generative AI model to generate a response in real time and returns the result to the user, with almost no latency.

[1281] (Application example 1)

[1282] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1283] In the past, when a factory robot broke down, an engineer had to go to the site and troubleshoot manually. However, this method required time and effort, and could prolong the production line downtime. Another problem was that engineers were not familiar with all causes of the failure, making it difficult to quickly obtain appropriate advice. Therefore, the objective of this invention is to provide a system that efficiently diagnoses factory robot failures and provides prompt support to engineers.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1285] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for the generated virtual personality to participate in an online meeting, means for real-time interaction between the user and the virtual personality, means for inputting a fault symptom of a factory robot, and means for generating a virtual engineer based on the fault symptom, thereby enabling the virtual engineer to assist in fault diagnosis of the factory robot in real time.

[1286] The "means for users to input problems" refers to a device or program that provides an interface that allows users such as engineers to input specific failure symptoms and problems.

[1287] The "means for generating an appropriate virtual personality" is a system that creates a virtual expert with a relevant skill set based on an input task.

[1288] The "means for allowing a generated virtual personality to participate in an online meeting" is a technology that allows a virtual personality to appear in an online meeting in real time and interact with a user.

[1289] "Means for real-time dialogue between a user and a virtual personality" refers to an interactive system in which a generated virtual personality can immediately respond to questions entered by a user.

[1290] The "means for inputting the malfunction symptoms of the factory robot" is an input device or software that allows an engineer to input specific malfunctions or problems related to the factory robot.

[1291] The "means for generating a virtual engineer based on a failure symptom" is a system that generates a virtual mechanical engineer with specialized knowledge in accordance with the inputted failure symptom of a robot.

[1292] "Means for analyzing keywords related to the task entered by the user" refers to technology that extracts and recognizes important words and phrases from the text entered by the user.

[1293] The "means for selecting and generating skilled virtual personalities" is a system that identifies and generates virtual experts with specialized abilities that correspond to the analyzed keywords.

[1294] The "means for the server to transmit profile information of the generated virtual personality to the terminal" is a system that transmits detailed information about the virtual expert created by the server to the user's terminal.

[1295] The "means for a terminal to display a virtual personality in an online meeting" is a technique by which a user's device displays a generated virtual expert on an online meeting screen.

[1296] "A means of supporting the fault diagnosis of factory robots in real time" is a technology in which a virtual engineer provides engineers with immediate fault diagnosis and repair advice.

[1297] The system for realizing this invention is composed of a user terminal, a server, a virtual personality generation engine, and an online meeting platform. As a specific system for supporting fault diagnosis of factory robots, the following components are used in combination.

[1298] 1. User device

[1299] The system uses tablets or smart glasses that allow users to input malfunction symptoms and specific problems of factory robots. Users input malfunction symptoms in text format through a dedicated interface.

[1300] 2. Server

[1301] The server receives and analyzes the failure symptoms sent from the user's device. It uses advanced natural language processing technology to extract important keywords from the input text. The hardware used is a high-performance Linux-based server, and the software uses generative AI models such as GPT-4.

[1302] 3. Virtual personality generation engine

[1303] The virtual personality generation engine installed on the server generates appropriate virtual engineers based on keywords extracted by natural language processing. The virtual personality generation engine creates virtual engineers with specialized skill sets, such as mechanical engineers or electrical control experts.

[1304] 4. Online Meeting Platform

[1305] The user and the generated virtual engineer interact in real time through an online meeting platform. The virtual engineer responds immediately to questions entered by the user and provides expert advice and suggestions. WebRTC and Zoom APIs are used.

[1306] Specifically, when a user types "robot arm malfunctions" into a tablet, the server receives and analyzes this text. The server extracts keywords such as "robot arm" and "malfunction" and passes them to the virtual personality generation engine. The virtual personality generation engine generates a virtual engineer specializing in robot arms based on these keywords. The generated virtual engineer participates in an online meeting room and assists the user in fault diagnosis while interacting in real time with the user.

[1307] As an example of a prompt sentence, by entering "Please tell me about the situation when the robot arm has stopped moving," the virtual engineer will immediately provide advice such as "The robot arm is malfunctioning. The symptoms are that it does not move even when instructed, or it moves very slowly. Please tell me the possible causes of the malfunction and how to deal with it."

[1308] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1309] Step 1:

[1310] A user accesses the online meeting platform on a user device such as a tablet and opens a screen to start a new meeting. The user enters a specific malfunction symptom, such as "the robot arm is not working properly," into the input field and clicks the "Start" button. Input: Malfunction symptom, Output: Meeting start request.

[1311] Step 2:

[1312] The terminal sends a meeting start request to the server. The request includes the problem (fault symptom) entered by the user. Input: Meeting start request, Output: Request data.

[1313] Step 3:

[1314] The server analyzes the received meeting start request. The server receives the failure symptom text (e.g., "Robot arm malfunction") and uses natural language processing technology to extract important keywords (e.g., "robot arm," "malfunction"). Input: Request data, Output: Extracted keywords.

[1315] Step 4:

[1316] The server passes the extracted keywords to the virtual personality generation engine, which generates a virtual engineer with related skills (e.g., an engineer specializing in robotic arms) based on these keywords. Input: extracted keywords, Output: virtual engineer.

[1317] Step 5:

[1318] The server sends the profile information and setting information of the created virtual engineer to the user terminal. The user terminal receives this information and displays the virtual engineer in the online meeting room. Input: Virtual engineer profile information, Output: Display data.

[1319] Step 6:

[1320] Users can ask specific questions or ask for advice to a virtual engineer displayed in an online meeting room. The virtual engineer responds immediately and provides expert advice and suggestions to the user's questions. Input: User's question, Output: Virtual engineer's response.

[1321] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1322] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system enables users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[1323] Overall system overview

[1324] The system consists of the following main components:

[1325] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[1326] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[1327] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model that generates an appropriate virtual personality based on the task.

[1328] 4. Emotion Engine: An engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses accordingly.

[1329] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[1330] System operation flow

[1331] 1. A user enters an assignment and starts an online meeting

[1332] The user logs in to the online meeting platform and opens the "New Meeting" screen.

[1333] In the room name field, enter the issue you want to solve (e.g., "Consultation regarding the introduction of AI technology").

[1334] When the "Start" button is clicked, the user terminal sends a meeting start request, including the room name, to the server.

[1335] 2. The server analyzes the task and generates a virtual personality

[1336] The server extracts the room name from the received start meeting request.

[1337] Based on the room name, the necessary keywords (e.g., "AI technology" and "introduction") are analyzed and identified.

[1338] The server invokes a virtual personality generation engine to generate a virtual personality (e.g., AI expert, project manager) with a skill set related to the keyword.

[1339] 3. The server sends the virtual personality's profile information to the device.

[1340] The server transmits the profile information (e.g., name, role, expertise) of the generated virtual personality to the user terminal.

[1341] 4. The device displays a virtual personality in online meetings

[1342] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[1343] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[1344] 5. Real-time interaction between the user and the virtual persona

[1345] Users can ask questions and ask for advice to the virtual persona in an online meeting room.

[1346] The virtual personality responds instantly and offers expert advice and suggestions to the user's questions.

[1347] 6. Emotional engine for analyzing user emotions and adjusting responses

[1348] The emotion engine analyzes the user's voice and text data to identify their emotions (e.g., joy, anger, anxiety).

[1349] The emotion engine adjusts the virtual personality's responses based on the analysis results, providing appropriate advice and assistance according to the user's emotions.

[1350] The virtual personality empathizes with the user's feelings and responds accordingly to the situation.

[1351] Specific examples

[1352] Example: Consultation on AI technology implementation projects and emotion recognition

[1353] The user logs into the online meeting platform and selects "New Meeting."

[1354] Enter "Consultation on the introduction of AI technology" as the room name and send a request to start a meeting to the server.

[1355] The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities of AI experts and project managers with the required skills.

[1356] The server transmits profile information of the generated virtual personality to the user terminal, and the user terminal displays the virtual personality in the online meeting room.

[1357] Users can talk to the virtual persona in real time and receive specific advice on the steps to implement AI technology and how to manage projects.

[1358] The emotion engine analyzes the user's emotions from their voice and text and determines that the emotion is anxiety.

[1359] Based on the analysis results, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're here to support you through this step of the project.").

[1360] In this way, the system of the present invention not only efficiently solves the user's specific problems but also provides appropriate responses according to the user's emotions, allowing the user to have a more effective and satisfying online meeting experience.

[1361] The processing flow will be explained below.

[1362] Step 1: User logs into online meeting platform

[1363] Users use their own terminals to access the online meeting platform.

[1364] The user enters the necessary login information (e.g., user ID, password) and logs in to the platform.

[1365] Step 2: Enter the assignment in the room name and start the meeting

[1366] The user opens the "New Meeting" screen of the online meeting platform.

[1367] Users enter the "problem they want to solve" (e.g., "Consultation regarding the introduction of AI technology") in the room name field.

[1368] The user clicks the "Start" button to send a meeting start request from the terminal to the server.

[1369] Step 3: The server receives and analyzes the issue

[1370] The device sends a meeting start request to the server, including the room name "Consultation on the introduction of AI technology."

[1371] The server extracts the room name from the received request.

[1372] The server analyzes the room name and identifies keywords related to the topic (e.g., "AI technology" or "introduction").

[1373] Step 4: The server generates the virtual personality

[1374] The server calls a virtual personality generation engine based on the identified keywords.

[1375] The virtual personality generation engine generates virtual personalities (e.g., AI experts, project managers) with skill sets related to the keywords.

[1376] The server maintains the profile information (e.g., name, role, expertise) of the generated virtual persona.

[1377] Step 5: The server sends the virtual personality information to the device.

[1378] The server transmits the profile information of the generated virtual personality to the user's terminal.

[1379] Step 6: Your device displays your virtual persona in online meetings

[1380] The terminal displays the virtual personality in the online meeting room based on the received profile information of the virtual personality.

[1381] The virtual persona joins the online meeting room and gives an initial greeting and self-introduction to the user.

[1382] Step 7: User interacts with virtual persona in real time

[1383] Users can ask questions or seek advice from virtual personalities in online meeting rooms (e.g., "What are the steps to implement AI technology in our company?").

[1384] The virtual personality (AI expert) provides technical advice in response to questions (e.g., "It's important to start with data collection and preprocessing.").

[1385] The virtual persona (project manager) provides advice on project management (e.g., "It is important to create a project schedule and manage risks.").

[1386] Step 8: The emotion engine analyzes the user's emotions

[1387] The emotion engine (in the server) collects user voice and text data.

[1388] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., joy, anger, anxiety).

[1389] Step 9: The virtual personality adjusts responses based on emotions

[1390] The emotion engine provides information about the analyzed emotional state to the virtual personality.

[1391] The virtual personality adjusts the tone and content of its responses depending on the user's emotional state (e.g., making reassuring statements to an anxious user).

[1392] Step 10: User ends meeting and provides feedback

[1393] After obtaining the necessary information, the user performs an operation to end the online meeting.

[1394] The terminal sends a meeting end request to the server.

[1395] The server acknowledges the end of the meeting and stores a log of the session.

[1396] The server displays a post-meeting feedback form on the user terminal.

[1397] Users enter their satisfaction with the problem-solving process, suggestions, etc. into the feedback form and submit it.

[1398] Example 2

[1399] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1400] Modern online meeting systems make it difficult for users to obtain effective support for issues that require specialized knowledge. They also fail to provide responses that take users' feelings into consideration, which can lead to communication issues. As a result, users find it difficult to have a satisfying meeting experience.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1402] In this invention, the server includes means for a user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for having the generated virtual personality participate in an online meeting, means for analyzing the user's emotions, and means for adjusting the virtual personality's responses based on the emotion analysis. This allows the user to receive effective and efficient support from a virtual personality with specialized knowledge, and further allows the user to receive empathetic responses according to the user's emotions.

[1403] "User" refers to a person who uses the online meeting platform to enter an issue and receive assistance.

[1404] "Issue" refers to a specific problem or issue that a user seeks to resolve.

[1405] A "virtual personality" refers to a virtual character generated using AI technology that has specific skills or expertise.

[1406] "Online meeting" refers to a virtual conference environment in which users and virtual personalities can interact in real time over the Internet.

[1407] An "emotion engine" refers to technology that analyzes a user's voice and text data to identify the user's emotions.

[1408] "Profile information" refers to basic information such as the virtual personality's name, role, and expertise.

[1409] A "server" refers to a remote computer system that receives requests from users and generates virtual personalities and analyzes emotions based on those requests.

[1410] "Terminal" refers to the device (e.g., PC, smartphone) used by a User to access the online meeting platform.

[1411] "Generative AI model" refers to artificial intelligence technology that generates virtual personalities and responses based on input data.

[1412] A "prompt sentence" refers to an instruction sentence input to a generative AI model.

[1413] A "natural language processing engine" refers to a technology that extracts and analyzes important keywords from text entered by a user.

[1414] The present invention combines a system in which a user uses an online meeting platform, inputs a specific topic, generates an appropriate virtual personality, and allows the virtual personality to participate in the online meeting, with an emotion engine that recognizes the user's emotions. This system allows users to effectively and efficiently obtain expert knowledge and advice, and further analyzes the user's emotions to adjust the virtual personality's responses, providing a more appropriate and human-like response.

[1415] The system consists of the following main components:

[1416] 1. User terminal: A device that allows users to start online meetings and enter assignments.

[1417] 2. Server: A central server that receives the tasks entered by users, analyzes them, generates virtual personalities, and participates in online meetings.

[1418] 3. Virtual personality generation engine: Located within the server, this engine runs an AI model (e.g., GPT-3) that generates an appropriate virtual personality based on the task.

[1419] 4. Emotion engine: An engine that analyzes emotions from the user's voice or text and adjusts the virtual personality's responses accordingly (e.g., IBM Watson Tone Analyzer).

[1420] 5. Online Meeting Platform: Provides an environment for real-time interaction between users and generated virtual personalities.

[1421] Example of operation

[1422] Example: Consultation on AI technology implementation projects and emotion recognition

[1423] A user logs into an online meeting platform and selects "New Meeting." They enter "Consultation on the Implementation of AI Technology" as the room name and send a request to start a meeting to the server. The server analyzes the room name and uses a virtual personality generation engine to generate virtual personalities for an AI expert and project manager with the required skills.

[1424] The server then sends the profile information of the virtual personality to the user's device, which then displays the virtual personality in an online meeting room. The user can then converse with the virtual personality in real time and receive specific advice on the steps to implement AI technology and how to manage the project.

[1425] The emotion engine analyzes the user's voice and text to determine if the emotion is anxiety. Based on this analysis, the virtual personality provides the user with reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with this step of the project.").

[1426] Prompt Sentence Examples

[1427] Below are some example prompts for the emotion engine to use to analyze user sentiment and tailor responses to the generative AI model:

[1428] "Tell us how a virtual personality responds when it recognizes that the user is anxious."

[1429] "Please demonstrate how the virtual personality can empathize with the user's anger as analyzed by the emotion engine."

[1430] To implement this invention, a user device accesses the online meeting platform using a browser or a dedicated application, and the server receives and analyzes requests from users. The virtual personality generation engine and emotion engine use dedicated AI models (e.g., GPT-3, IBM Watson Tone Analyzer) that run on the server. This allows users to receive emotionally sensitive responses while solving their problems.

[1431] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1432] Step 1:

[1433] A user enters an assignment and starts an online meeting.

[1434] Users access the online meeting platform using a browser or a dedicated application. They enter their username and password on the login screen and click the "Login" button. After logging in, they open the "New Meeting" screen and enter the problem they want to solve in the room name field. The entered room name is sent from the device to the server in JSON format as a meeting start request. The input data is the content of the problem, and the output is a request to the server.

[1435] Step 2:

[1436] The server analyzes the task and generates a virtual personality.

[1437] The server extracts the room name from the received meeting start request. Based on the room name, the server uses a natural language processing engine to analyze the required keywords (e.g., "AI technology," "implementation"). Using the analyzed keywords as input, the server invokes a virtual personality generation engine (e.g., GPT-3) to generate a virtual personality with the associated skill set (e.g., AI expert, project manager). The output is the virtual personality's profile information.

[1438] Step 3:

[1439] The server transmits the profile information of the virtual personality to the terminal.

[1440] The server sends the profile information of the generated virtual personality to the user's device in JSON format. The profile information includes the virtual personality's name, role, and expertise. The input data is the result of the virtual personality generation, and the output is the transmission of the profile information to the user's device.

[1441] Step 4:

[1442] The device displays a virtual persona in online meetings.

[1443] The user device analyzes the received virtual personality profile information and displays it on the screen of the online meeting room. The virtual personality then gives an initial greeting and introduces itself. The input data is the profile information, and the output is the displayed virtual personality.

[1444] Step 5:

[1445] The user and the virtual persona interact in real time.

[1446] The user asks the virtual persona a question through a chat box or voice input (e.g., "What is the first step in implementing AI?"). The virtual persona receives the user's question and generates a response using a generative AI model (e.g., "We recommend starting by defining the technical requirements."). The input data is the user's question, and the output is the virtual persona's response.

[1447] Step 6:

[1448] Emotion engine analyzes user emotions and adjusts responses.

[1449] The emotion engine analyzes the user's voice and text data to identify the user's emotions. For example, if the voice data indicates that the user is feeling anxious, the emotion engine communicates the analysis result to the virtual personality. The virtual personality then provides reassuring advice in a gentle tone (e.g., "Don't worry, we're fully supporting you with each step of this project."). The input data is the user's emotional information, and the output is the adjusted response of the virtual personality.

[1450] Through the above steps, the system can provide users with specialized knowledge and respond in a way that takes into consideration the user's feelings.

[1451] (Application example 2)

[1452] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1453] Traditional customer support in brick-and-mortar stores often lacks staff on-site to address specific issues, making it difficult to respond quickly to customer needs. Furthermore, traditional support systems are unable to respond in a way that takes into account the customer's emotions, making it difficult to improve customer satisfaction. For this reason, there is a demand for personalized experiences that address customer issues and take their emotions into consideration.

[1454] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input a task, means for generating an appropriate virtual personality based on the task input by the user, means for displaying the generated virtual personality in a physical store and providing customer support, means for analyzing the user's emotions and adjusting the virtual personality's response, and means for conducting a real-time dialogue between the user and the virtual personality. This makes it possible to quickly respond to a user's specific task using an appropriate virtual personality even in a physical store, and to provide personalized services according to the user's emotions.

[1455] A "user" is a person who uses the system to input tasks and interact with a virtual personality.

[1456] An "issue" is a specific problem or inquiry that a user wants the system to solve.

[1457] A "virtual personality" is a digital agent generated by AI that responds appropriately to user challenges.

[1458] A "brick and mortar store" is a commercial establishment that is located in a physical location and can be visited by customers in person.

[1459] "Sentiment analysis" is the process of identifying emotions from a user's voice or text data.

[1460] The "generation means" is a technical means for generating an appropriate virtual personality based on the task input by the user.

[1461] "Display means" refers to the technical means for displaying the generated virtual personality in a physical store or on a terminal.

[1462] "Response adjustment means" refers to a technical means for adjusting the response of a virtual personality based on the results of emotion analysis.

[1463] "Real-time interaction means" refers to a technical means for allowing a user and a virtual personality to interact in real time.

[1464] This invention relates to a system that generates an appropriate virtual personality by having the user input a specific problem, and provides customer support in a physical store. Furthermore, it can perform emotion analysis and adjust the virtual personality's response according to the user's emotions. The entire system is composed of the following main components:

[1465] Hardware and Software

[1466] Hardware:

[1467] iOS / Android smartphones

[1468] software:

[1469] iOS / Android applications (Swift / Kotlin)

[1470] Server (AWS EC2, Amazon RDS)

[1471] AI models (TensorFlow, PyTorch)

[1472] Analysis engine (Amazon Comprehend, Google Cloud Natural Language)

[1473] System configuration

[1474] 1. User Device:

[1475] It is a device that allows users to input tasks and interact with virtual personalities.

[1476] A smartphone application is installed and users can enter assignments through this application.

[1477] 2. Server:

[1478] It is a central server that receives and analyzes user-entered tasks to generate and display virtual personalities.

[1479] The server is equipped with an AI model and an emotion analysis engine, enabling advanced data processing.

[1480] 3. Virtual Personality Generation Engine:

[1481] It is an engine that runs an AI model located on a server and generates an appropriate virtual personality based on the user's challenges.

[1482] 4. Sentiment Analysis Engine:

[1483] It is an engine that analyzes emotions from the user's voice and text and adjusts the virtual personality's responses based on that.

[1484] Application Examples

[1485] Example 1: Fashion coordination consultation

[1486] The user types into the smartphone app, "Please tell me how to coordinate this outfit."

[1487] The server analyzes the input and generates a virtual personality of a fashion expert.

[1488] The virtual expert appears as a virtual display in a physical store and greets the user with, "Hello, I'm a fashion expert. What outfit would you like to wear today?"

[1489] When a user asks, "Which top goes well with this skirt?", the emotion analysis engine analyzes the emotions in the user's voice and text, and if it determines that the user looks anxious, the virtual expert will respond in a gentle tone, "Don't worry, just relax and choose. I recommend this top."

[1490] Example 2: Shopping assistance

[1491] A user types "Tell me about the new smartphone" into a smartphone app.

[1492] The server analyzes the input and generates a virtual personality of a technical expert.

[1493] The virtual expert appears on a display in a physical store and speaks to the user, saying, "Hello, I'm a technology expert. I'd like to tell you about our new smartphone."

[1494] If a user asks, "What's the battery life?" and the sentiment analysis engine determines that the user is excited, the virtual expert will respond enthusiastically, "This phone has excellent battery life, lasting more than a day with normal use."

[1495] As described above, by combining virtual personality generation and emotion analysis, this system can provide customers with fast and personalized support in physical stores, improving customer satisfaction and contributing to sales promotion in physical stores.

[1496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1497] Step 1:

[1498] A smartphone application is used as a means for users to input tasks. The user launches the smartphone application, enters the specific task they wish to solve (e.g., "Please tell me how to coordinate this outfit") in the text field on the "Task Input" screen, and taps the "Submit" button. This task input data is sent to the server. The input data is the task content in text format.

[1499] Step 2:

[1500] The server receives the assignment input data and begins analysis. The server analyzes the received assignment content and extracts related keywords. For example, keywords such as "clothes" and "coordinate" are extracted. These keywords are analyzed using a text analysis engine (e.g., Amazon Comprehend) to generate an extracted keyword list. The keyword list is the output of the process.

[1501] Step 3:

[1502] The server generates an appropriate virtual personality based on the extracted keyword list. The server invokes a virtual personality generation engine (AI model) to generate a virtual personality (e.g., a fashion expert) with a skill set related to the specified keywords. Profile information (e.g., name, expertise, role) of this virtual personality is generated, and the virtual personality profile information is output.

[1503] Step 4:

[1504] The server transmits the profile information of the generated virtual personality to the user terminal. The user terminal begins displaying the virtual personality based on the received profile information. Specifically, the virtual personality appears as a virtual display within the smartphone application, and an initial greeting is given, such as "Hello, I'm a fashion expert. What kind of outfit would you like to wear today?" The display screen is output.

[1505] Step 5:

[1506] Users interact with the virtual persona in real time by using the chat function or voice input function of the smartphone app to input and send questions (e.g., "Which top goes well with this skirt?"), and the user input data is sent to the system.

[1507] Step 6:

[1508] The server receives new input data from the user and performs emotion analysis. It uses an emotion analysis engine (e.g., Google Cloud Natural Language) to identify emotions (e.g., anxiety, excitement) from the user's voice or text data. The analysis results are output as emotion data.

[1509] Step 7:

[1510] The server generates and adjusts the response content of the virtual personality based on the emotional data. The virtual personality generation engine generates the response content together with the emotional data (e.g., "It's okay, just relax and choose. I recommend this top."), and the adjusted response data is generated. This adjusted response data is the output.

[1511] Step 8:

[1512] The server transmits the adjusted response data to the user terminal, and the user terminal displays or plays the response of the virtual personality in real time based on the received response data, and the user receives the response from the virtual personality and continues the next dialogue.

[1513] Through the above process, even in a physical store, a user can receive a quick and appropriate response from a virtual personality in response to a problem entered by the user, providing personalized service that takes into consideration the user's problem-solving and emotions.

[1514] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1515] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1516] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1517] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1518] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1519] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1520] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1521] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1522] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1523] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1524] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1525] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1526] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1527] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1528] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1529] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1530] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1531] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1532] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1533] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1534] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1535] The following is further disclosed regarding the above embodiment.

[1536] (Claim 1)

[1537] a means for a user to input a task;

[1538] A means for generating an appropriate virtual personality based on a task input by a user;

[1539] A means for allowing the generated virtual personality to participate in an online meeting;

[1540] means for real-time interaction between the user and the virtual persona;

[1541] A system including:

[1542] (Claim 2)

[1543] A means for analyzing keywords related to the task input by the user;

[1544] A means for selecting and generating a virtual personality with skills based on the analyzed keywords;

[1545] 10. The system of claim 1, further comprising:

[1546] (Claim 3)

[1547] A means for the server to transmit profile information of the generated virtual personality to the terminal;

[1548] A means for the terminal to display the virtual persona in the online meeting;

[1549] 10. The system of claim 1, further comprising:

[1550] "Example 1"

[1551] (Claim 1)

[1552] a means for a user to input a task;

[1553] A means for generating an appropriate virtual person based on a task input by a user;

[1554] A means for allowing the generated virtual person to participate in an online conference;

[1555] means for real-time interaction between a user and a virtual person;

[1556] A system including:

[1557] (Claim 2)

[1558] means for analyzing contextual information associated with a user-entered task;

[1559] A means for selecting and generating a virtual person with technical capabilities based on the analyzed context information;

[1560] 10. The system of claim 1, further comprising:

[1561] (Claim 3)

[1562] a means for transmitting specification information of the generated virtual person from a central device to a terminal;

[1563] A means for the terminal to display a virtual person in an online conference;

[1564] 10. The system of claim 1, further comprising:

[1565] "Application Example 1"

[1566] (Claim 1)

[1567] a means for a user to input a task;

[1568] A means for generating an appropriate virtual personality based on a task input by a user;

[1569] A means for allowing the generated virtual personality to participate in an online meeting;

[1570] means for real-time interaction between the user and the virtual persona;

[1571] A means for inputting the malfunction symptoms of the factory robot;

[1572] means for generating a virtual engineer based on the fault symptoms;

[1573] A system including:

[1574] (Claim 2)

[1575] A means for analyzing keywords related to the task input by the user;

[1576] A means for selecting and generating a virtual personality with skills based on the analyzed keywords;

[1577] means for analyzing information related to a failure symptom;

[1578] 10. The system of claim 1, further comprising:

[1579] (Claim 3)

[1580] A means for the server to transmit profile information of the generated virtual personality to the terminal;

[1581] A means for the terminal to display the virtual persona in the online meeting;

[1582] A means of supporting the fault diagnosis of factory robots in real time,

[1583] 10. The system of claim 1, further comprising:

[1584] "Example 2: Combining Emotion Engines"

[1585] (Claim 1)

[1586] a means for a user to input a task;

[1587] A means for generating an appropriate virtual personality based on a task input by a user;

[1588] A means for allowing the generated virtual personality to participate in an online meeting;

[1589] means for real-time interaction between the user and the virtual persona;

[1590] means for analyzing user sentiment;

[1591] means for adjusting the responses of the virtual personality based on emotion analysis;

[1592] A system including:

[1593] (Claim 2)

[1594] A means for analyzing keywords related to the task input by the user;

[1595] A means for selecting and generating a virtual personality with skills based on the analyzed keywords;

[1596] 10. The system of claim 1, further comprising:

[1597] (Claim 3)

[1598] A means for the server to transmit profile information of the generated virtual personality to the terminal;

[1599] A means for the terminal to display the virtual persona in the online meeting;

[1600] 10. The system of claim 1, further comprising:

[1601] "Application example 2 when combining emotion engines"

[1602] (Claim 1)

[1603] a means for a user to input a task;

[1604] A means for generating an appropriate virtual personality based on a task input by a user;

[1605] A means for displaying the generated virtual personality in a physical store and providing customer support;

[1606] means for analyzing the user's emotions and adjusting the response of the virtual personality;

[1607] means for real-time interaction between the user and the virtual persona;

[1608] A system including:

[1609] (Claim 2)

[1610] means for analyzing language data related to a task input by a user;

[1611] A means for selecting and generating a virtual personality with skills based on the analyzed language data;

[1612] 10. The system of claim 1, further comprising:

[1613] (Claim 3)

[1614] A means for the server to transmit profile information of the generated virtual personality to the terminal;

[1615] A means for the terminal to display the virtual personality in a physical store;

[1616] 10. The system of claim 1, further comprising: [Explanation of symbols]

[1617] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input a task; A means for generating an appropriate virtual personality based on a task input by a user; A means for allowing the generated virtual personality to participate in an online meeting; means for real-time interaction between the user and the virtual persona; A system including:

2. A means for analyzing keywords related to the task input by the user; A means for selecting and generating a virtual personality with skills based on the analyzed keywords; The system of claim 1 further comprising:

3. A means for the server to transmit profile information of the generated virtual personality to the terminal; A means for the terminal to display the virtual persona in the online meeting; The system of claim 1 further comprising:

Images