System

Abstract

A system is provided.SOLUTION: A system comprising: means for creating a virtual lecturer specialized in a specific field using generative artificial intelligence; means for storing the virtual lecturer created by the generative artificial intelligence in a database; means for displaying a list of the virtual lecturers to allow a user to select one; means for reserving a session with the virtual lecturer at a date and time desired by the user; means for performing a real-time session with the virtual lecturer based on the reservation; means for storing data of the real-time session; and means for distributing earnings to creators of the virtual lecturer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Students who want to take English conversation lessons face the problem of not being able to find a teacher's schedule that suits their needs, or being unable to find topics to talk about. Furthermore, it is difficult to easily find a teacher who specializes in a specific field. This limits students' opportunities to take English conversation lessons tailored to their interests and needs. There is a need to solve these problems and provide an environment where students can take English conversation lessons at their own time and on their own topics. [Means for solving the problem]

[0005] The present invention relates to a system that uses generative artificial intelligence to create virtual tutors specialized in a specific field. The system provides a means for a user (creator) to input prompts related to a specific field and store the generated virtual tutor in a database. It also includes a means for students to select a desired tutor from a list of virtual tutors and make a reservation for a desired date and time. By providing a means for conducting real-time sessions based on reservations and a means for storing session data, the system creates an environment in which students can freely take English conversation lessons. Furthermore, by providing a means for virtual tutor creators to share revenue, a means for sending reminder notifications, and a dashboard for checking revenue status, the system enhances convenience for creators and students, and improves mutual satisfaction.

[0006] "Generative artificial intelligence" is a software system that generates a subject-specific virtual tutor based on user-specified prompts.

[0007] A "virtual teacher" is a virtual teacher generated by generative artificial intelligence who provides online English conversation lessons.

[0008] A "database" is an electronic storage device for systematically storing and managing information such as virtual instructors, reservation information, and session data.

[0009] A "list" displays a list of multiple virtual instructors so that the user can select one.

[0010] "Teacher" means an individual or corporation who wishes to take, book, and attend English conversation lessons with a virtual teacher.

[0011] "Reservation" refers to the act of a student specifying the date and time of a session with a desired virtual instructor.

[0012] A "real-time session" is an English conversation lesson format in which a student and a virtual instructor interact in real time at a reserved date and time.

[0013] The "Revenue Sharing Method" is part of a system that appropriately distributes revenue from English conversation lessons to the creators of virtual tutors.

[0014] "Reminder notification" is a function that notifies participants when the date and time of a reserved session is approaching.

[0015] "Dashboard" means the user interface through which Virtual Teacher Creators can view and manage their earnings and other management information.

[0016] "Session Data" means recorded data, including conversational content and other related information, generated during a real-time session. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[0039] Overall system overview

[0040] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[0041] Program processing flow

[0042] Registration and prompt entry

[0043] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0044] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[0045] The terminal checks these inputs and sends them to the server.

[0046] Generate and save AI instructors

[0047] The server passes the received prompts to a generation artificial intelligence to generate a virtual instructor specialized in a specific field.

[0048] The data of the generated virtual instructor (instructor ID, creator information, generation date and time, prompt content, etc.) is saved in the database.

[0049] Students can select and book AI instructors

[0050] The device displays a list of available virtual instructors to the student, including instructor profiles and past evaluations.

[0051] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form.

[0052] The terminal transmits this reservation information to the server.

[0053] Lesson implementation and log saving

[0054] The server prepares the real-time session based on the reserved date and time.

[0055] The device receives a reminder notification and notifies the user (student) before the lesson begins.

[0056] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[0057] The AI ​​instructor answers students' questions in real time, and the exchange is saved as a log in a database.

[0058] Revenue sharing

[0059] The server updates the revenue information after the lesson ends and distributes the revenue to the virtual instructor creator. The revenue calculation is based on the lesson fee and the student's evaluation.

[0060] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[0061] Specific examples

[0062] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music":

[0063] The creator fills in the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[0064] The generative artificial intelligence generates a virtual instructor based on this prompt and stores the data in a database.

[0065] Students select this instructor from the list and book a lesson on the desired date and time.

[0066] The session begins at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[0067] This system allows users (students) to take English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0071] Step 2:

[0072] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[0073] Step 3:

[0074] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Input checks include checking whether all fields are filled in and whether the email address format is correct.

[0075] Step 4:

[0076] The server accepts the received data and passes prompts to the generative artificial intelligence to generate a virtual tutor specialized in a specific field, which analyzes the prompts and selects the required model.

[0077] Step 5:

[0078] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[0079] Step 6:

[0080] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[0081] Step 7:

[0082] The user (student) selects the desired instructor from a list of virtual instructors and enters the desired date and time in the lesson reservation form, which often uses a calendar-style UI.

[0083] Step 8:

[0084] The terminal sends the student's reservation information to the server, which includes the student ID, the selected virtual instructor ID, the desired date and time, etc.

[0085] Step 9:

[0086] The server receives the reservation information and stores it in a database, which confirms the reservation and allows a reservation confirmation notice to be sent to the user.

[0087] Step 10:

[0088] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[0089] Step 11:

[0090] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[0091] Step 12:

[0092] The user (student) clicks the session start button at the start time of the lesson, which starts a real-time session with the virtual instructor.

[0093] Step 13:

[0094] The device initiates a real-time session with the virtual instructor via a real-time chat system or voice call system. The student's questions and comments are sent to the virtual instructor, and answers are generated and displayed in real time.

[0095] Step 14:

[0096] The server stores conversation data from real-time sessions, including the questions asked by the student, the responses given by the virtual instructor, and session timestamps.

[0097] Step 15:

[0098] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[0099] Step 16:

[0100] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] In today's education market, instructors with specialized knowledge in specific fields are scarce and in high demand. Furthermore, there is a need for a system that allows students to conveniently take lessons according to their own schedule. There is also a need for a system that provides fair revenue distribution to virtual instructor creators. A new system is needed to solve these issues.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using a generating AI, means for saving the virtual tutors created by the generating AI in a database, means for displaying a list of the virtual tutors to allow users to select one, means for reserving a session with the virtual tutor at a date and time desired by the user, means for conducting a real-time session with the virtual tutor based on the reservation, means for enabling a chat screen and voice call function during the real-time session, means for saving data on the real-time session, and means for distributing revenue to the creators of the virtual tutors. This makes it possible to efficiently meet the demand for tutors specialized in a specific field, enable students to take lessons conveniently, and provide appropriate revenue distribution to creators.

[0106] "Generative AI" is AI that uses natural language processing techniques to generate text or scripts based on specific input data or prompts.

[0107] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, who has knowledge of a specific field and provides education and guidance through dialogue with students.

[0108] A "database" is an electronic system or structure for efficiently storing, managing, and retrieving information.

[0109] "User" refers to an individual or organization that uses this system to create a virtual teacher or take English conversation lessons.

[0110] A "list" is a format that displays information about virtual instructors in a centralized manner, allowing users to select the instructors according to their purpose.

[0111] "Reservation" means the process by which a User reserves a session with a Virtual Instructor for a specific date and time.

[0112] A "real-time session" is a session in which a user and a virtual instructor interact in real time at a reserved date and time.

[0113] The "chat screen" is an interface that allows users to have text-based conversations with virtual instructors.

[0114] The "voice call function" is a function that allows users and virtual instructors to communicate via voice during a real-time session.

[0115] "Data" means records and information relating to your interactions and sessions with a Virtual Instructor.

[0116] "Revenue" means profit calculated based on lesson fees paid by users and other income sources.

[0117] "Distribution" means the act of appropriately sharing revenue with the creator of the virtual instructor and other parties.

[0118] A "reminder" is an electronic notification that alerts a user before a session begins.

[0119] A "dashboard" is an interface for visually displaying revenue status and other management information.

[0120] This invention is a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[0121] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[0122] The system is implemented using the following hardware and software:

[0123] Hardware: personal computer, smartphone, server (can be cloud-based)

[0124] Software: Registration form (web application), generative AI (e.g., OpenAI's GPT-4), database (MySQL, PostgreSQL, etc.)

[0125] The user (creator) initiates the process of generating a virtual tutor using a domain-specific prompt (e.g., "A tutor who has knowledge of 80's music and can answer related questions").

[0126] The generated virtual instructors are stored on the server side, and the user (student) selects the desired virtual instructor from this list to reserve a lesson. Based on the reserved date and time, the server prepares a real-time session and sends a notification to the terminal. The user (student) starts the real-time session by clicking the session start button, and the interaction is stored in the database.

[0127] For example:

[0128] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music," the process would be as follows:

[0129] 1. The creator fills the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[0130] 2. Generative AI generates a virtual instructor based on this prompt and stores the data in a database.

[0131] 3. The student selects this instructor from the list and reserves the lesson on the desired date and time.

[0132] 4. The session will begin at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[0133] This system allows users (students) to receive English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] The device displays a registration form for the creator. The form contains input fields for name, email address, and a prompt for the virtual instructor. The input data includes name, email address, and prompt, and is collected using HTML and JavaScript. Based on this data input, appropriate validation is performed to ensure all required fields are filled in. The input data is converted to JSON format and sent to the server.

[0137] Step 2:

[0138] The server passes the received prompt to a generative artificial intelligence model (generative AI model), which generates a virtual tutor specialized in a specific field. The prompt and related metadata are used as input data. The generative AI model generates text data and scripts for the virtual tutor based on the input prompt, and outputs the results in JSON format. The output data includes the tutor ID, creator information, generation date and time, prompt content, etc. This data is returned to the server and stored in a database.

[0139] Step 3:

[0140] The device displays a list of available virtual instructors to the student. This list includes instructor profiles and past evaluations, and the latest instructor information is obtained from the server using an AJAX request. The displayed list is dynamically generated using HTML and CSS, and the user (student) can select the virtual instructor they are interested in. The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. The entered reservation information (student ID, instructor ID, desired date and time) is converted to JSON format and sent to the server.

[0141] Step 4:

[0142] The server prepares the real-time session based on the reserved date and time. Session details are set based on the reservation information, and a reminder notification is sent to the device. This notification is sent using an API such as Firebase Cloud Messaging. The user (student) receives the reminder notification and is given a warning before the session begins. The real-time session begins when the user (student) clicks the session start button. During the real-time session, the chat screen and voice call functions are enabled, and WebRTC technology is used for this.

[0143] Step 5:

[0144] The server records interactions during the real-time session, and the data (text messages, audio data) is stored in a database. This allows the history of the session content to be managed. After the session ends, the server updates the lesson revenue information and distributes revenue to the virtual instructor creator. Revenue is calculated based on the lesson fee and student ratings. Finally, the device updates the creator's dashboard, which displays revenue status and other management information in real time. This dashboard uses UI components such as graphs and tables to present information to the user in an easy-to-understand manner.

[0145] (Application example 1)

[0146] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0147] Traditional online lesson systems have had issues with users finding instructors who specialize in specific fields, and they are unable to answer questions or receive product explanations in real time. Another factor is the lack of virtual assistants who can provide detailed product explanations and support needed when shopping. This often leads to a poor user experience and delays in making purchasing decisions.

[0148] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0149] In this invention, the server includes a means for creating a virtual assistant specialized in a specific field using a generative artificial intelligence, a means for storing the virtual assistant created by the generative artificial intelligence in a database, and a means for the virtual assistant to answer questions about products from the user in real time, thereby enabling the user to obtain detailed product descriptions and immediate answers to individual questions.

[0150] "Generative artificial intelligence" is a technology that generates specialized virtual assistants based on prompts entered by the user.

[0151] A "virtual assistant" is a digital agent created by generative artificial intelligence that provides real-time information and assistance in a specific field.

[0152] The "database" is a digital system for systematically storing and managing the created virtual assistant data and session log information.

[0153] A "real-time session" is a means of communication without time delay for a user to interact directly with a virtual assistant.

[0154] "Reminder notification" is a feature that sends advance notifications to users so that they do not forget the time of their reserved session.

[0155] The "dashboard" is an interface that allows the creator of a virtual assistant to view session revenue and management information in real time.

[0156] "Revenue sharing" is a system that distributes revenue to virtual assistant creators based on sessions with users.

[0157] This invention relates to a system that uses generative artificial intelligence to generate a virtual assistant specialized in a specific field, and provides product explanations and purchasing support through the assistant. Specific embodiments are described below.

[0158] Overall system overview

[0159] First, the user (creator) provides input prompts to the generation AI to create a virtual assistant specialized in a specific field. Based on these prompts, the generation AI generates a virtual assistant, and the data is stored in a database. Next, the user (consumer) can select the desired virtual assistant from this list and reserve a session at the desired date and time. At the reserved date and time, a real-time session is conducted, the session data is saved, and revenue is distributed to the virtual assistant creator.

[0160] Program processing flow

[0161] User (creator) registration and prompt input

[0162] The device displays the creator's registration form, which includes input fields for name, email address, and a prompt for a virtual assistant.

[0163] The user (creator) enters this information and fills in the prompts for a detailed description of the particular product.

[0164] The terminal checks the input and sends it to the server.

[0165] Creating and saving an AI assistant

[0166] The server passes the received prompts to a generative AI model to generate a virtual assistant specialized for a specific product (e.g., using GPT-4 as the generative AI model).

[0167] The data of the generated assistant (assistant ID, creator information, generation date and time, prompt content, etc.) is saved in a database (MySQL is used as the database).

[0168] User selection and reservation of AI assistants

[0169] The device displays a list of available virtual assistants, including each assistant's profile and the product categories they support.

[0170] The user selects the desired assistant and enters the desired date and time in the session reservation form.

[0171] The terminal transmits this reservation information to the server.

[0172] Conducting product explanation sessions

[0173] The server prepares the real-time session based on the reserved date and time.

[0174] The device receives a reminder notification and notifies the user before the session begins.

[0175] The user clicks the session start button to start a real-time session. During this time, the chat screen and voice call functions are enabled.

[0176] The virtual assistant answers users' product questions in real time, and the interactions are stored in a database.

[0177] Revenue sharing

[0178] The server updates the revenue information after the session ends and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee, user ratings, etc.

[0179] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[0180] Specific examples

[0181] For example, if a user (creator) wants to create a "virtual assistant that is knowledgeable about the latest smartwatches":

[0182] The creator fills in the form with the prompt, "A virtual assistant who can provide detailed instructions on the features and usage of the latest smartwatch."

[0183] The generative artificial intelligence generates a virtual assistant based on these prompts and stores the data in a database.

[0184] The user selects this assistant from the list and makes a reservation for the desired date and time.

[0185] The session will begin at the scheduled time and date, and the user can ask questions about their smartwatch, with the virtual assistant providing real-time answers.

[0186] Prompt Sentence Examples

[0187] "Generate a virtual shopping assistant that can provide detailed instructions on the features and usage of the latest smartwatch."

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1:

[0190] To create a specialized virtual assistant, a user (creator) uses a device to enter information into a registration form. Input items include a name, email address, and a prompt for the virtual assistant. The device sends this input data to the server, which receives the input data as a text prompt (e.g., "Please generate a virtual shopping assistant that can provide detailed explanations about the features and usage of the latest smartwatches.").

[0191] Step 2:

[0192] The server passes the received prompt to a generative AI model (e.g., GPT-4) to generate a virtual assistant. The generative AI model analyzes the prompt and generates data (text, voice data, etc.) for an assistant specialized in a specific field. The server stores this generated virtual assistant data in a database (e.g., a MySQL database). The stored data includes the assistant ID, creator information, generation date and time, prompt content, etc.

[0193] Step 3:

[0194] The terminal displays a list of available virtual assistants to the user. This list is obtained from the server and includes each assistant's profile and the product categories they can handle. The user selects the desired virtual assistant from this list.

[0195] Step 4:

[0196] The user selects the desired assistant and enters the desired date and time in the session reservation form via the terminal. The terminal sends this reservation information to the server. The sent reservation information includes the user's ID, the selected assistant ID, and the desired date and time.

[0197] Step 5:

[0198] The server prepares the real-time session based on the reservation information. Before the session starts, the server sends a reminder notification to the terminal to notify the user that the session is about to begin.

[0199] Step 6:

[0200] When the user clicks the session start button, the device starts a real-time session. During the session, the chat screen and voice call function are enabled, and the user can ask questions to the virtual assistant in real time.

[0201] Step 7:

[0202] The virtual assistant answers users' questions in real time, and the server logs these interactions in a database. The stored data includes the session timestamp, the user's question, and the assistant's response.

[0203] Step 8:

[0204] After the session ends, the server updates the revenue information and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee and the user's rating. The device updates the creator's dashboard based on this information and displays the revenue status in real time.

[0205] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0206] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. This system can provide more interactive and adaptive lessons by combining it with an emotion engine that recognizes the user's emotions.

[0207] Overall system overview

[0208] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session begins at the reserved date and time, and not only is the session data saved, but the emotion engine recognizes the user's emotions and adjusts the response. This data is also saved as session data. In addition, revenue is distributed to the virtual tutor creator.

[0209] Program processing flow

[0210] Registration and prompt entry

[0211] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0212] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[0213] The terminal checks these inputs and sends them to the server.

[0214] Generate and save AI instructors

[0215] The server passes the received prompts to the artificial intelligence generator, which then generates a virtual instructor specialized in a specific field. At this time, the prompts are analyzed and the necessary model is selected.

[0216] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[0217] Students can select and book AI instructors

[0218] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[0219] The user (student) selects the desired virtual instructor and enters the desired date and time into the lesson reservation form, which often uses a calendar-style UI.

[0220] The terminal transmits this reservation information to the server.

[0221] Lesson implementation and emotion recognition

[0222] The server prepares the real-time session based on the reserved date and time.

[0223] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[0224] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[0225] The emotion engine built into the device recognizes emotions from the student's facial expressions and voice. For example, it uses a camera and microphone to collect emotional information from facial expressions and voice tone.

[0226] The emotion data recognized by the emotion engine is sent to the server in real time, and the virtual instructor's response is adjusted accordingly. For example, if the student is confused, the virtual instructor will provide a simpler explanation.

[0227] The server stores conversation data and emotion data during the real-time session.

[0228] Revenue sharing

[0229] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[0230] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[0231] Specific examples

[0232] For example, if a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson:

[0233] Based on the generated information, the virtual instructor begins a real-time conversation with the student.

[0234] If a student asks, "I'd like to know more about '80s rock music," the emotion engine analyzes the student's facial expressions and tone of voice and recognizes that they are excited.

[0235] The virtual instructor responds by offering further details or amusing anecdotes, and if the student looks confused, they simplify the explanation or provide additional information.

[0236] After the lesson is over, the conversation content and the student's emotional data are saved, and revenue is distributed based on that.

[0237] This system allows students to receive lessons that best suit their emotions, and allows virtual instructor creators to earn higher revenue.

[0238] The processing flow will be explained below.

[0239] Program processing flow

[0240] Registration and prompt entry

[0241] Step 1:

[0242] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and virtual instructor prompts.

[0243] Step 2:

[0244] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[0245] Step 3:

[0246] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Basic input checks are performed (e.g., ensuring all required fields are filled in).

[0247] Generate and save AI instructors

[0248] Step 4:

[0249] The server accepts the received data and passes prompts to the generative AI to generate a virtual tutor specialized in a specific field. The prompts are analyzed and the necessary AI model is selected.

[0250] Step 5:

[0251] The server stores the generated virtual instructor data (e.g. instructor ID, creator ID, creation date and time, prompt content) in a database for later retrieval and reference.

[0252] Students can select and book AI instructors

[0253] Step 6:

[0254] The terminal retrieves a list of available virtual instructors from the database and displays it to the student. This list includes information such as the profile, evaluation, and areas of expertise of each virtual instructor.

[0255] Step 7:

[0256] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. A calendar UI allows for visual reservations.

[0257] Step 8:

[0258] The terminal sends the student's reservation information (e.g., student ID, selected virtual instructor ID, desired date and time) to the server.

[0259] Step 9:

[0260] The server receives the reservation information and stores it in a database. Once the reservation is confirmed, a reservation confirmation notification can be sent to the student.

[0261] Lesson implementation and emotion recognition

[0262] Step 10:

[0263] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[0264] Step 11:

[0265] The device receives a reminder notification and notifies the user (student) before the lesson begins. Notifications can be sent via browser or in-app notifications.

[0266] Step 12:

[0267] The user (student) clicks the Start Session button at the start of the lesson, which starts a real-time session with the virtual instructor.

[0268] Step 13:

[0269] The terminal starts a real-time session with the virtual instructor via a real-time chat system or a voice call system, and the student's questions and comments are sent to the virtual instructor, who then responds in real time.

[0270] Step 14:

[0271] The emotion engine built into the device uses a camera and microphone to recognize emotions from the students' facial expressions and voices, and this emotional data is analyzed in real time.

[0272] Step 15:

[0273] The server receives the student's emotional data in real time and adjusts the virtual instructor's response accordingly. For example, if the student is confused, the virtual instructor updates the response to provide a simpler explanation.

[0274] Step 16:

[0275] The server stores conversation and emotion data from the real-time session in a database, which can be used for later review and improvement.

[0276] Revenue sharing

[0277] Step 17:

[0278] After the lesson is completed, the server updates the revenue information and distributes the revenue to the virtual instructor creator based on the lesson fee, rating, and student usage.

[0279] Step 18:

[0280] The device updates the creator's dashboard with new revenue and other management information in real time, allowing creators to monitor their virtual tutor's performance and revenue.

[0281] Specific examples

[0282] For example, if a user (creator) wants to create a "virtual instructor who is knowledgeable about 80s music," they would enter "an instructor who has knowledge about 80s music and can answer questions" into the prompt. The generative AI generates a virtual instructor based on this prompt and stores it in a database. When a student selects this instructor, books a lesson, and starts a real-time session, the emotion engine analyzes the student's facial expressions and tone of voice to recognize emotions and adjust the virtual instructor's responses based on that information.

[0283] Example 2

[0284] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Traditional online education systems face challenges in quickly and effectively providing instructors specializing in specific fields. Furthermore, they are unable to recognize students' emotions and adjust responses on the spot, meaning the quality of lessons is heavily influenced by the students' emotional state, making it difficult to provide an optimal learning environment. Furthermore, creators lack a way to monitor the performance and revenue status of virtual instructors in real time.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0287] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using generative artificial intelligence, means for storing the created virtual tutor in a database, and means for recognizing the user's emotions during a real-time session and adjusting responses. This allows students to receive the most appropriate lesson according to their emotions, and the creator can also check and manage the virtual tutor's performance and revenue status in real time.

[0288] "Generative AI" is an AI system that has the ability to generate new information or functions from data using specific algorithms or models.

[0289] A "virtual instructor" is a type of generated digital personality that has specialized knowledge in a particular field and provides educational services through dialogue.

[0290] A "database" is a system for organizing, storing, and retrieving a collection of data in a certain structure.

[0291] A "list" refers to a set of items or entries organized and displayed according to certain criteria.

[0292] "User" refers to an end user who uses the system to receive the services of a virtual instructor.

[0293] "Real-time session" refers to an interactive session in which the virtual instructor and the user interact directly and respond instantly.

[0294] "Means for recognizing emotions and adjusting responses" refers to technology that detects the user's emotional state in real time and adjusts the content and tone of the virtual instructor's responses based on that information.

[0295] "Revenue Sharing Mechanism" means the mechanism by which revenues earned based on the provision of Virtual Instructor Services are distributed to Instructor Creators.

[0296] A "dashboard" refers to an interface that allows users and creators to visually check the system's operational status and performance data.

[0297] This invention is a system that uses generative artificial intelligence to create a virtual instructor specialized in a specific field and then uses the virtual instructor to provide an interactive educational session. The system consists of the following main components:

[0298] Registration and prompt entry

[0299] First, the device displays a registration form for the creator, where the creator enters their name, email address, and a subject-specific prompt (e.g., "Tennis Technique Instructor"). The input is format-checked by a client-side script, such as JavaScript, and if valid, is sent to the server via an HTTP POST request.

[0300] Generate and save AI instructors

[0301] The server passes the received prompts to a generative AI to generate a virtual tutor specialized in a specific field, using a generative AI model such as OpenAI's GPT-4. The generated virtual tutor data is stored in a relational database such as MySQL or PostgreSQL along with the tutor ID, creator ID, generation date and time, and prompt content.

[0302] Students can select and book AI instructors

[0303] Next, the device displays a list of available virtual instructors. The list includes each instructor's profile, ratings, and areas of expertise. The user (student) selects a virtual instructor that best suits their needs and schedules a lesson using a calendar UI. The reservation information is checked again and sent to the server.

[0304] Lesson implementation and emotion recognition

[0305] The server prepares the real-time session based on the reserved date and time. When the lesson starts, the device receives a reminder notification and notifies the user. When the user clicks a link or button to start the session, a voice or video call begins.

[0306] During this time, an emotion engine built into the device recognizes the student's facial expressions and tone of voice in real time. The emotion engine typically uses emotion recognition APIs such as Google Cloud Vision or IBM Watson. The recognized emotion data is sent to a server, and the virtual instructor's responses are adjusted in real time. For example, if a student is confused, the virtual instructor can simplify their explanation.

[0307] Revenue sharing

[0308] After the lesson is over, the server updates the session revenue information and distributes the revenue to the virtual instructor creator. Payments are made via payment processing systems such as PayPal or Stripe. The device also updates the creator's dashboard with the new revenue information and instructor performance data.

[0309] Specific examples

[0310] For example, consider a case where a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson. The virtual teacher will start a real-time conversation based on the generated information. If the student asks, "I want to know more about 80s rock music," the emotion engine will analyze the student's facial expressions and tone of voice and recognize that the student is excited. In response, the virtual teacher will provide more detailed information or an interesting anecdote. If the student looks confused, the virtual teacher will simplify the explanation or provide additional information. After the lesson, the student's emotional data will be saved along with the conversation content, and revenue will be distributed based on that.

[0311] In this way, the system provides highly interactive and adaptive educational sessions, enhancing the learning experience for the student and benefiting the instructor creator.

[0312] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0313] Step 1:

[0314] The device displays the creator's registration form. The creator enters their name, email address, and a prompt (e.g., "Tennis Technique Instructor"). The input is done through an HTML form, and JavaScript is used to check the data for formatting. The entered data is temporarily stored on the client side.

[0315] Example input:

[0316] Name: Taro Yamada

[0317] Email address: yamada@example.com

[0318] Prompt: "An instructor who is knowledgeable about tennis techniques."

[0319] Example output:

[0320] {

[0321] "Name": "Yamada Taro",

[0322] "Email address": "yamada@example.com",

[0323] "prompt": "Tennis technique instructor"

[0324] }

[0325] Step 2:

[0326] The device checks the input and sends it to the server. The check checks whether required fields have been entered and validates the email address format. Once the check is complete, an HTTP POST request is sent to the server using a JavaScript AJAX request. The server receives this data and temporarily stores it in a database.

[0327] Example input:

[0328] {

[0329] "Name": "Yamada Taro",

[0330] "Email address": "yamada@example.com",

[0331] "prompt": "Tennis technique instructor"

[0332] }

[0333] Example output:

[0334] A new record is created in the database.

[0335] Step 3:

[0336] The server passes the received prompt to a generative AI to generate a virtual tutor. The prompt sentence is analyzed using a generative AI model (e.g., GPT-4). A script and dialogue model for the generated virtual tutor are created and returned as the generated result.

[0337] Example input:

[0338] Prompt: "An instructor who is knowledgeable about tennis techniques."

[0339] Example output:

[0340] Virtual instructor model (script, dialogue model)

[0341] Step 4:

[0342] The server saves the generated virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content. The data is inserted into a relational database, such as MySQL or PostgreSQL.

[0343] Example input:

[0344] Virtual instructor model (script, dialogue model), prompt sentences

[0345] Example output:

[0346] {

[0347] "Teacher ID": 12345,

[0348] "Creator ID": "yamada@example.com",

[0349] "Generation date and time": "2023-10-10T10:00:00Z",

[0350] "prompt": "Tennis technique instructor",

[0351] "ModelData": "{...}" / / Virtual instructor details

[0352] }

[0353] Step 5:

[0354] The device displays a list of available virtual instructors. The data to be displayed is retrieved from a database and displayed in the browser in a table or card format using HTML and JavaScript.

[0355] Example input:

[0356] A list of instructors retrieved by a database query

[0357] Example output:

[0358] Display instructor list in HTML format

[0359] Step 6:

[0360] The user (student) selects the desired virtual instructor and books a lesson. The student selects the desired date and time using the calendar UI and enters the information into the booking form. This data is sent to the server using AJAX.

[0361] Example input:

[0362] Instructor ID: 12345, Reservation date and time: "2023-10-15T15:00:00Z"

[0363] Example output:

[0364] {

[0365] "Reservation ID": 67890,

[0366] "Teacher ID": 12345,

[0367] "Student ID": "suzuki@example.com",

[0368] "Reservation date and time": "2023-10-15T15:00:00Z"

[0369] }

[0370] Step 7:

[0371] The server prepares the session. When the reserved date and time arrives, it generates resources for the session (e.g., a room ID for a video conference) and prepares data to be sent as a reminder to the device.

[0372] Example input:

[0373] Reservation Information

[0374] Example output:

[0375] {

[0376] "Session ID": "ROOM12345",

[0377] "Reminder": "Notification 10 minutes before lesson starts"

[0378] }

[0379] Step 8:

[0380] The device receives the reminder notification and notifies the user. The device displays a reminder to the user to start the lesson using a browser notification or a mobile app notification.

[0381] Example input:

[0382] Reminder notification data

[0383] Example output:

[0384] Pop-up notifications

[0385] Step 9:

[0386] A user (student) starts a session. A voice or video call is started by clicking a link or button. A chat screen or voice call function is enabled within the session.

[0387] Example input:

[0388] Signaling the start of a session

[0389] Example output:

[0390] Open the video call screen

[0391] Step 10:

[0392] The device's built-in emotion engine recognizes the student's facial expressions and tone of voice. The emotion engine collects data using a camera and microphone, analyzes the emotion data in real time using Google Cloud Vision and IBM Watson APIs, and sends it to a server.

[0393] Example input:

[0394] Facial expression data, voice data

[0395] Example output:

[0396] Real-time sentiment analysis data

[0397] Step 11:

[0398] The server stores data during real-time sessions, records conversation data and emotion data in a database, and saves the session details.

[0399] Example input:

[0400] Conversation data, emotion data

[0401] Example output:

[0402] Database Update

[0403] Step 12:

[0404] The server updates the revenue information and distributes the revenue to the creator. After the session ends, the revenue is calculated and paid to the creator via, for example, PayPal or Stripe.

[0405] Example input:

[0406] Session data, revenue sharing algorithms

[0407] Example output:

[0408] Payment Transactions

[0409] Step 13:

[0410] The device updates the creator's dashboard and UI to display new revenue information and instructor performance data in real time.

[0411] Example input:

[0412] Updated earnings and performance data

[0413] Example output:

[0414] Updated Dashboard View

[0415] (Application example 2)

[0416] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0417] Conventional virtual instructor systems were capable of providing users with interactive lessons specialized in specific fields, but lacked the ability to recognize the user's emotions and adjust responses in real time. As a result, there were problems with user satisfaction and learning effectiveness. It was also difficult to provide the interactivity that users experience in real-life interpersonal communication. Furthermore, the same problem existed in shopping, where users were unable to receive product guidance that responded to their emotions in real time. There is a need to solve these problems.

[0418] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0419] In this invention, the server includes means for creating a virtual instructor specialized in a specific field using a generative artificial intelligence, means for saving the virtual instructor created by the generative artificial intelligence in a database, means for displaying a list of the virtual instructors to allow a user to select one, means for reserving a session with the virtual instructor at a date and time desired by the user, means for conducting a real-time session with the virtual instructor based on the reservation, means for saving data of the real-time session, means for distributing revenue to the creator of the virtual instructor, means for analyzing the user's emotions using emotion recognition technology and adjusting the virtual instructor's responses, and means for saving the emotion recognition data in a database. This enables optimal responses according to the user's emotions and makes it possible to provide real-time interactive lessons and shopping assistant experiences.

[0420] "Generative AI" is an AI technology that generates domain-specific responses and information based on input prompts.

[0421] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, with knowledge and skills in a specific field, who provides interactive lessons to users.

[0422] The "database" is an information system for storing and managing information about the generated virtual instructor, session data, emotion recognition data, etc.

[0423] "Emotion recognition technology" is a technology that analyzes emotions from a user's facial expressions, tone of voice, etc., and adjusts the system's response based on the results.

[0424] A "real-time session" is an online session in which a virtual instructor and a user interact in real time to provide interactive feedback.

[0425] "Revenue sharing" is a system in which the creator of a virtual instructor is given a share of the profits and revenues earned from students.

[0426] A "server" is a computer system that processes user requests, generates virtual instructors, stores data, manages real-time sessions, and so on.

[0427] Overall system overview

[0428] This invention provides a system for creating and using virtual instructors specialized in specific fields. This system includes functions for user registration, virtual instructor creation, selection and reservation, real-time sessions, emotion recognition, data storage, and revenue sharing. This system is primarily implemented by a server, terminals (smartphones, PCs, etc.), and software using emotion recognition technology.

[0429] Hardware and software used

[0430] Hardware: Camera, microphone, smartphone, PC, server

[0431] Software: Generative artificial intelligence models (e.g., OpenAI GPT-4), emotion recognition engines, database management systems (e.g., MySQL), real-time communication tools (e.g., WebRTC)

[0432] A natural language explanation of the program's processing overview

[0433] User Registration

[0434] The device displays a user registration form, where the user enters their name, email address, specific areas of interest, etc. This information is sent to the server, which stores the user data in a database.

[0435] Virtual Instructor Creation

[0436] The server uses a generative artificial intelligence model to generate a virtual tutor based on prompts received from the user, such as "a tutor who is knowledgeable about fashion trends for Spring 2023." The generated virtual tutor is then stored in a database by the server.

[0437] Select and book a virtual teacher

[0438] The terminal retrieves a list of virtual instructors from the database and displays it to the user. The user selects the desired virtual instructor and date and time, and sends the reservation information to the server. The server saves the reservation data.

[0439] Conducting real-time sessions

[0440] At the scheduled time, the device will send a reminder to the user and start the real-time session. During the session, emotion recognition technology will analyze the user's facial expressions and tone of voice and transmit the results in real time to the server. The server will then adjust the virtual instructor's responses based on this emotion data.

[0441] Data storage

[0442] Once the session ends, the server stores the conversation data and emotion recognition data in a database.

[0443] Revenue sharing

[0444] After the session ends, the server updates the revenue data and distributes the revenue to the virtual instructor creator, who can check the revenue status through their device.

[0445] Specific examples

[0446] The user selects "A virtual assistant knowledgeable about fashionable Spring 2023 fashion" and books a shopping session. The real-time session begins, and the user turns on their camera to interact with the assistant. When the user asks, "What spring outfits do you recommend?", the emotion recognition engine analyzes the user's facial expression and recognizes that they are excited. The assistant then presents detailed and glamorous products.

[0447] Prompt Sentence Examples

[0448] Create a fashion-savvy virtual assistant that can provide information on fashion trends, especially for Spring 2023, and guide you through the shopping process in a fun and interactive way.

[0449] The above is a specific embodiment for carrying out the invention.

[0450] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0451] Step 1: User Registration

[0452] Input: The user enters their name, email address, and specific areas of interest (e.g., fashion, home appliances) into the device.

[0453] Processing: The terminal checks these input data and sends them to the server.

[0454] Data processing and calculation: The server stores the received data in a database.

[0455] Output: A new user record is created in the database and the user registration is complete.

[0456] Step 2: Create a virtual instructor

[0457] Input: A subject-specific prompt from the user (e.g., "Teacher who is knowledgeable about fashion trends for Spring 2023").

[0458] Processing: The server passes the prompts to a generative artificial intelligence model to generate a virtual instructor specialized in the specific subject area.

[0459] Data processing and calculation: A generative artificial intelligence model analyzes the prompts and generates data for a virtual instructor with domain-specific knowledge.

[0460] Output: The generated virtual instructor data is saved in a database via the server.

[0461] Step 3: Select and book a virtual teacher

[0462] Input: The user browses the list of virtual instructors through the terminal and selects the desired instructor and date and time.

[0463] Processing: The device sends information about the selected instructor and reservation date and time to the server.

[0464] Data processing and calculation: The server verifies the reservation data and stores it in the database.

[0465] Output: The virtual instructor and date / time are booked and a confirmation is sent to the user.

[0466] Step 4: Reminders

[0467] Enter your reservation information shortly before your reservation date and time.

[0468] Processing: The server sends a reminder notification to the device based on the reservation information.

[0469] Data processing and calculation: The server calculates the reservation date and time and sends a notification at the appropriate time.

[0470] Output: A reminder notification is displayed to the user on their device.

[0471] Step 5: Conduct a real-time session

[0472] Input: The user clicks the session start button on the terminal at the scheduled date and time.

[0473] Processing: The device initiates a real-time session and interacts with the server to stream session data.

[0474] Data processing and calculation: The server processes the session data and passes it to the emotion recognition engine in real time.

[0475] Output: A real-time interactive session is initiated and continues.

[0476] Step 6: Emotion recognition and response adjustment

[0477] Input: User's facial expression and voice tone data are sent from the device.

[0478] Processing: The emotion recognition engine analyzes the input data and identifies the user's emotion.

[0479] Data processing and calculation: The server adjusts the virtual instructor's responses based on the emotion recognition results.

[0480] Output: Tailored responses are provided to the user in real time.

[0481] Step 7: Save your data

[0482] Input: Conversation data and emotion recognition data from a real-time session.

[0483] Processing: The server stores these data in a database.

[0484] Data processing and calculations: After the session, the data is organized and stored.

[0485] Output: All session and sentiment data is stored securely in a database.

[0486] Step 8: Revenue sharing

[0487] Input: Revenue data after the session ends.

[0488] Processing: The server calculates the revenue data and distributes it to the creators of the virtual instructors.

[0489] Data processing and calculation: The server calculates the revenue and distributes it appropriately.

[0490] Output: Creator dashboard with revenue information.

[0491] The above are the specific processing steps of the system based on the present invention.

[0492] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0494] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0495] [Second embodiment]

[0496] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0497] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0498] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0499] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0500] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0501] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0502] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0503] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0504] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0505] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0506] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0507] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0508] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[0509] Overall system overview

[0510] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[0511] Program processing flow

[0512] Registration and prompt entry

[0513] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0514] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[0515] The terminal checks these inputs and sends them to the server.

[0516] Generate and save AI instructors

[0517] The server passes the received prompts to a generation artificial intelligence to generate a virtual instructor specialized in a specific field.

[0518] The data of the generated virtual instructor (instructor ID, creator information, generation date and time, prompt content, etc.) is saved in the database.

[0519] Students can select and book AI instructors

[0520] The device displays a list of available virtual instructors to the student, including instructor profiles and past evaluations.

[0521] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form.

[0522] The terminal transmits this reservation information to the server.

[0523] Lesson implementation and log saving

[0524] The server prepares the real-time session based on the reserved date and time.

[0525] The device receives a reminder notification and notifies the user (student) before the lesson begins.

[0526] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[0527] The AI ​​instructor answers students' questions in real time, and the exchange is saved as a log in a database.

[0528] Revenue sharing

[0529] The server updates the revenue information after the lesson ends and distributes the revenue to the virtual instructor creator. The revenue calculation is based on the lesson fee and the student's evaluation.

[0530] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[0531] Specific examples

[0532] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music":

[0533] The creator fills in the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[0534] The generative artificial intelligence generates a virtual instructor based on this prompt and stores the data in a database.

[0535] Students select this instructor from the list and book a lesson on the desired date and time.

[0536] The session begins at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[0537] This system allows users (students) to take English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[0538] The processing flow will be explained below.

[0539] Step 1:

[0540] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0541] Step 2:

[0542] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[0543] Step 3:

[0544] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Input checks include checking whether all fields are filled in and whether the email address format is correct.

[0545] Step 4:

[0546] The server accepts the received data and passes prompts to the generative artificial intelligence to generate a virtual tutor specialized in a specific field, which analyzes the prompts and selects the required model.

[0547] Step 5:

[0548] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[0549] Step 6:

[0550] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[0551] Step 7:

[0552] The user (student) selects the desired instructor from a list of virtual instructors and enters the desired date and time in the lesson reservation form, which often uses a calendar-style UI.

[0553] Step 8:

[0554] The terminal sends the student's reservation information to the server, which includes the student ID, the selected virtual instructor ID, the desired date and time, etc.

[0555] Step 9:

[0556] The server receives the reservation information and stores it in a database, which confirms the reservation and allows a reservation confirmation notice to be sent to the user.

[0557] Step 10:

[0558] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[0559] Step 11:

[0560] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[0561] Step 12:

[0562] The user (student) clicks the session start button at the start time of the lesson, which starts a real-time session with the virtual instructor.

[0563] Step 13:

[0564] The device initiates a real-time session with the virtual instructor via a real-time chat system or voice call system. The student's questions and comments are sent to the virtual instructor, and answers are generated and displayed in real time.

[0565] Step 14:

[0566] The server stores conversation data from real-time sessions, including the questions asked by the student, the responses given by the virtual instructor, and session timestamps.

[0567] Step 15:

[0568] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[0569] Step 16:

[0570] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[0571] Example 1

[0572] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0573] In today's education market, instructors with specialized knowledge in specific fields are scarce and in high demand. Furthermore, there is a need for a system that allows students to conveniently take lessons according to their own schedule. There is also a need for a system that provides fair revenue distribution to virtual instructor creators. A new system is needed to solve these issues.

[0574] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0575] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using a generating AI, means for saving the virtual tutors created by the generating AI in a database, means for displaying a list of the virtual tutors to allow users to select one, means for reserving a session with the virtual tutor at a date and time desired by the user, means for conducting a real-time session with the virtual tutor based on the reservation, means for enabling a chat screen and voice call function during the real-time session, means for saving data on the real-time session, and means for distributing revenue to the creators of the virtual tutors. This makes it possible to efficiently meet the demand for tutors specialized in a specific field, enable students to take lessons conveniently, and provide appropriate revenue distribution to creators.

[0576] "Generative AI" is AI that uses natural language processing techniques to generate text or scripts based on specific input data or prompts.

[0577] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, who has knowledge of a specific field and provides education and guidance through dialogue with students.

[0578] A "database" is an electronic system or structure for efficiently storing, managing, and retrieving information.

[0579] "User" refers to an individual or organization that uses this system to create a virtual teacher or take English conversation lessons.

[0580] A "list" is a format that displays information about virtual instructors in a centralized manner, allowing users to select the instructors according to their purpose.

[0581] "Reservation" means the process by which a User reserves a session with a Virtual Instructor for a specific date and time.

[0582] A "real-time session" is a session in which a user and a virtual instructor interact in real time at a reserved date and time.

[0583] The "chat screen" is an interface that allows users to have text-based conversations with virtual instructors.

[0584] The "voice call function" is a function that allows users and virtual instructors to communicate via voice during a real-time session.

[0585] "Data" means records and information relating to your interactions and sessions with a Virtual Instructor.

[0586] "Revenue" means profit calculated based on lesson fees paid by users and other income sources.

[0587] "Distribution" means the act of appropriately sharing revenue with the creator of the virtual instructor and other parties.

[0588] A "reminder" is an electronic notification that alerts a user before a session begins.

[0589] A "dashboard" is an interface for visually displaying revenue status and other management information.

[0590] This invention is a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[0591] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[0592] The system is implemented using the following hardware and software:

[0593] Hardware: personal computer, smartphone, server (can be cloud-based)

[0594] Software: Registration form (web application), generative AI (e.g., OpenAI's GPT-4), database (MySQL, PostgreSQL, etc.)

[0595] The user (creator) initiates the process of generating a virtual tutor using a domain-specific prompt (e.g., "A tutor who has knowledge of 80's music and can answer related questions").

[0596] The generated virtual instructors are stored on the server side, and the user (student) selects the desired virtual instructor from this list to reserve a lesson. Based on the reserved date and time, the server prepares a real-time session and sends a notification to the terminal. The user (student) starts the real-time session by clicking the session start button, and the interaction is stored in the database.

[0597] For example:

[0598] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music," the process would be as follows:

[0599] 1. The creator fills the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[0600] 2. Generative AI generates a virtual instructor based on this prompt and stores the data in a database.

[0601] 3. The student selects this instructor from the list and reserves the lesson on the desired date and time.

[0602] 4. The session will begin at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[0603] This system allows users (students) to receive English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[0604] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0605] Step 1:

[0606] The device displays a registration form for the creator. The form contains input fields for name, email address, and a prompt for the virtual instructor. The input data includes name, email address, and prompt, and is collected using HTML and JavaScript. Based on this data input, appropriate validation is performed to ensure all required fields are filled in. The input data is converted to JSON format and sent to the server.

[0607] Step 2:

[0608] The server passes the received prompt to a generative artificial intelligence model (generative AI model), which generates a virtual tutor specialized in a specific field. The prompt and related metadata are used as input data. The generative AI model generates text data and scripts for the virtual tutor based on the input prompt, and outputs the results in JSON format. The output data includes the tutor ID, creator information, generation date and time, prompt content, etc. This data is returned to the server and stored in a database.

[0609] Step 3:

[0610] The device displays a list of available virtual instructors to the student. This list includes instructor profiles and past evaluations, and the latest instructor information is obtained from the server using an AJAX request. The displayed list is dynamically generated using HTML and CSS, and the user (student) can select the virtual instructor they are interested in. The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. The entered reservation information (student ID, instructor ID, desired date and time) is converted to JSON format and sent to the server.

[0611] Step 4:

[0612] The server prepares the real-time session based on the reserved date and time. Session details are set based on the reservation information, and a reminder notification is sent to the device. This notification is sent using an API such as Firebase Cloud Messaging. The user (student) receives the reminder notification and is given a warning before the session begins. The real-time session begins when the user (student) clicks the session start button. During the real-time session, the chat screen and voice call functions are enabled, and WebRTC technology is used for this.

[0613] Step 5:

[0614] The server records interactions during the real-time session, and the data (text messages, audio data) is stored in a database. This allows the history of the session content to be managed. After the session ends, the server updates the lesson revenue information and distributes revenue to the virtual instructor creator. Revenue is calculated based on the lesson fee and student ratings. Finally, the device updates the creator's dashboard, which displays revenue status and other management information in real time. This dashboard uses UI components such as graphs and tables to present information to the user in an easy-to-understand manner.

[0615] (Application example 1)

[0616] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0617] Traditional online lesson systems have had issues with users finding instructors who specialize in specific fields, and they are unable to answer questions or receive product explanations in real time. Another factor is the lack of virtual assistants who can provide detailed product explanations and support needed when shopping. This often leads to a poor user experience and delays in making purchasing decisions.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0619] In this invention, the server includes a means for creating a virtual assistant specialized in a specific field using a generative artificial intelligence, a means for storing the virtual assistant created by the generative artificial intelligence in a database, and a means for the virtual assistant to answer questions about products from the user in real time, thereby enabling the user to obtain detailed product descriptions and immediate answers to individual questions.

[0620] "Generative artificial intelligence" is a technology that generates specialized virtual assistants based on prompts entered by the user.

[0621] A "virtual assistant" is a digital agent created by generative artificial intelligence that provides real-time information and assistance in a specific field.

[0622] The "database" is a digital system for systematically storing and managing the created virtual assistant data and session log information.

[0623] A "real-time session" is a means of communication without time delay for a user to interact directly with a virtual assistant.

[0624] "Reminder notification" is a feature that sends advance notifications to users so that they do not forget the time of their reserved session.

[0625] The "dashboard" is an interface that allows the creator of a virtual assistant to view session revenue and management information in real time.

[0626] "Revenue sharing" is a system that distributes revenue to virtual assistant creators based on sessions with users.

[0627] This invention relates to a system that uses generative artificial intelligence to generate a virtual assistant specialized in a specific field, and provides product explanations and purchasing support through the assistant. Specific embodiments are described below.

[0628] Overall system overview

[0629] First, the user (creator) provides input prompts to the generation AI to create a virtual assistant specialized in a specific field. Based on these prompts, the generation AI generates a virtual assistant, and the data is stored in a database. Next, the user (consumer) can select the desired virtual assistant from this list and reserve a session at the desired date and time. At the reserved date and time, a real-time session is conducted, the session data is saved, and revenue is distributed to the virtual assistant creator.

[0630] Program processing flow

[0631] User (creator) registration and prompt input

[0632] The device displays the creator's registration form, which includes input fields for name, email address, and a prompt for a virtual assistant.

[0633] The user (creator) enters this information and fills in the prompts for a detailed description of the particular product.

[0634] The terminal checks the input and sends it to the server.

[0635] Creating and saving an AI assistant

[0636] The server passes the received prompts to a generative AI model to generate a virtual assistant specialized for a specific product (e.g., using GPT-4 as the generative AI model).

[0637] The data of the generated assistant (assistant ID, creator information, generation date and time, prompt content, etc.) is saved in a database (MySQL is used as the database).

[0638] User selection and reservation of AI assistants

[0639] The device displays a list of available virtual assistants, including each assistant's profile and the product categories they support.

[0640] The user selects the desired assistant and enters the desired date and time in the session reservation form.

[0641] The terminal transmits this reservation information to the server.

[0642] Conducting product explanation sessions

[0643] The server prepares the real-time session based on the reserved date and time.

[0644] The device receives a reminder notification and notifies the user before the session begins.

[0645] The user clicks the session start button to start a real-time session. During this time, the chat screen and voice call functions are enabled.

[0646] The virtual assistant answers users' product questions in real time, and the interactions are stored in a database.

[0647] Revenue sharing

[0648] The server updates the revenue information after the session ends and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee, user ratings, etc.

[0649] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[0650] Specific examples

[0651] For example, if a user (creator) wants to create a "virtual assistant that is knowledgeable about the latest smartwatches":

[0652] The creator fills in the form with the prompt, "A virtual assistant who can provide detailed instructions on the features and usage of the latest smartwatch."

[0653] The generative artificial intelligence generates a virtual assistant based on these prompts and stores the data in a database.

[0654] The user selects this assistant from the list and makes a reservation for the desired date and time.

[0655] The session will begin at the scheduled time and date, and the user can ask questions about their smartwatch, with the virtual assistant providing real-time answers.

[0656] Prompt Sentence Examples

[0657] "Generate a virtual shopping assistant that can provide detailed instructions on the features and usage of the latest smartwatch."

[0658] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0659] Step 1:

[0660] To create a specialized virtual assistant, a user (creator) uses a device to enter information into a registration form. Input items include a name, email address, and a prompt for the virtual assistant. The device sends this input data to the server, which receives the input data as a text prompt (e.g., "Please generate a virtual shopping assistant that can provide detailed explanations about the features and usage of the latest smartwatches.").

[0661] Step 2:

[0662] The server passes the received prompt to a generative AI model (e.g., GPT-4) to generate a virtual assistant. The generative AI model analyzes the prompt and generates data (text, voice data, etc.) for an assistant specialized in a specific field. The server stores this generated virtual assistant data in a database (e.g., a MySQL database). The stored data includes the assistant ID, creator information, generation date and time, prompt content, etc.

[0663] Step 3:

[0664] The terminal displays a list of available virtual assistants to the user. This list is obtained from the server and includes each assistant's profile and the product categories they can handle. The user selects the desired virtual assistant from this list.

[0665] Step 4:

[0666] The user selects the desired assistant and enters the desired date and time in the session reservation form via the terminal. The terminal sends this reservation information to the server. The sent reservation information includes the user's ID, the selected assistant ID, and the desired date and time.

[0667] Step 5:

[0668] The server prepares the real-time session based on the reservation information. Before the session starts, the server sends a reminder notification to the terminal to notify the user that the session is about to begin.

[0669] Step 6:

[0670] When the user clicks the session start button, the device starts a real-time session. During the session, the chat screen and voice call function are enabled, and the user can ask questions to the virtual assistant in real time.

[0671] Step 7:

[0672] The virtual assistant answers users' questions in real time, and the server logs these interactions in a database. The stored data includes the session timestamp, the user's question, and the assistant's response.

[0673] Step 8:

[0674] After the session ends, the server updates the revenue information and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee and the user's rating. The device updates the creator's dashboard based on this information and displays the revenue status in real time.

[0675] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0676] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. This system can provide more interactive and adaptive lessons by combining it with an emotion engine that recognizes the user's emotions.

[0677] Overall system overview

[0678] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session begins at the reserved date and time, and not only is the session data saved, but the emotion engine recognizes the user's emotions and adjusts the response. This data is also saved as session data. In addition, revenue is distributed to the virtual tutor creator.

[0679] Program processing flow

[0680] Registration and prompt entry

[0681] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0682] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[0683] The terminal checks these inputs and sends them to the server.

[0684] Generate and save AI instructors

[0685] The server passes the received prompts to the artificial intelligence generator, which then generates a virtual instructor specialized in a specific field. At this time, the prompts are analyzed and the necessary model is selected.

[0686] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[0687] Students can select and book AI instructors

[0688] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[0689] The user (student) selects the desired virtual instructor and enters the desired date and time into the lesson reservation form, which often uses a calendar-style UI.

[0690] The terminal transmits this reservation information to the server.

[0691] Lesson implementation and emotion recognition

[0692] The server prepares the real-time session based on the reserved date and time.

[0693] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[0694] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[0695] The emotion engine built into the device recognizes emotions from the student's facial expressions and voice. For example, it uses a camera and microphone to collect emotional information from facial expressions and voice tone.

[0696] The emotion data recognized by the emotion engine is sent to the server in real time, and the virtual instructor's response is adjusted accordingly. For example, if the student is confused, the virtual instructor will provide a simpler explanation.

[0697] The server stores conversation data and emotion data during the real-time session.

[0698] Revenue sharing

[0699] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[0700] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[0701] Specific examples

[0702] For example, if a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson:

[0703] Based on the generated information, the virtual instructor begins a real-time conversation with the student.

[0704] If a student asks, "I'd like to know more about '80s rock music," the emotion engine analyzes the student's facial expressions and tone of voice and recognizes that they are excited.

[0705] The virtual instructor responds by offering further details or amusing anecdotes, and if the student looks confused, they simplify the explanation or provide additional information.

[0706] After the lesson is over, the conversation content and the student's emotional data are saved, and revenue is distributed based on that.

[0707] This system allows students to receive lessons that best suit their emotions, and allows virtual instructor creators to earn higher revenue.

[0708] The processing flow will be explained below.

[0709] Program processing flow

[0710] Registration and prompt entry

[0711] Step 1:

[0712] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and virtual instructor prompts.

[0713] Step 2:

[0714] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[0715] Step 3:

[0716] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Basic input checks are performed (e.g., ensuring all required fields are filled in).

[0717] Generate and save AI instructors

[0718] Step 4:

[0719] The server accepts the received data and passes prompts to the generative AI to generate a virtual tutor specialized in a specific field. The prompts are analyzed and the necessary AI model is selected.

[0720] Step 5:

[0721] The server stores the generated virtual instructor data (e.g. instructor ID, creator ID, creation date and time, prompt content) in a database for later retrieval and reference.

[0722] Students can select and book AI instructors

[0723] Step 6:

[0724] The terminal retrieves a list of available virtual instructors from the database and displays it to the student. This list includes information such as the profile, evaluation, and areas of expertise of each virtual instructor.

[0725] Step 7:

[0726] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. A calendar UI allows for visual reservations.

[0727] Step 8:

[0728] The terminal sends the student's reservation information (e.g., student ID, selected virtual instructor ID, desired date and time) to the server.

[0729] Step 9:

[0730] The server receives the reservation information and stores it in a database. Once the reservation is confirmed, a reservation confirmation notification can be sent to the student.

[0731] Lesson implementation and emotion recognition

[0732] Step 10:

[0733] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[0734] Step 11:

[0735] The device receives a reminder notification and notifies the user (student) before the lesson begins. Notifications can be sent via browser or in-app notifications.

[0736] Step 12:

[0737] The user (student) clicks the Start Session button at the start of the lesson, which starts a real-time session with the virtual instructor.

[0738] Step 13:

[0739] The terminal starts a real-time session with the virtual instructor via a real-time chat system or a voice call system, and the student's questions and comments are sent to the virtual instructor, who then responds in real time.

[0740] Step 14:

[0741] The emotion engine built into the device uses a camera and microphone to recognize emotions from the students' facial expressions and voices, and this emotional data is analyzed in real time.

[0742] Step 15:

[0743] The server receives the student's emotional data in real time and adjusts the virtual instructor's response accordingly. For example, if the student is confused, the virtual instructor updates the response to provide a simpler explanation.

[0744] Step 16:

[0745] The server stores conversation and emotion data from the real-time session in a database, which can be used for later review and improvement.

[0746] Revenue sharing

[0747] Step 17:

[0748] After the lesson is completed, the server updates the revenue information and distributes the revenue to the virtual instructor creator based on the lesson fee, rating, and student usage.

[0749] Step 18:

[0750] The device updates the creator's dashboard with new revenue and other management information in real time, allowing creators to monitor their virtual tutor's performance and revenue.

[0751] Specific examples

[0752] For example, if a user (creator) wants to create a "virtual instructor who is knowledgeable about 80s music," they would enter "an instructor who has knowledge about 80s music and can answer questions" into the prompt. The generative AI generates a virtual instructor based on this prompt and stores it in a database. When a student selects this instructor, books a lesson, and starts a real-time session, the emotion engine analyzes the student's facial expressions and tone of voice to recognize emotions and adjust the virtual instructor's responses based on that information.

[0753] Example 2

[0754] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0755] Traditional online education systems face challenges in quickly and effectively providing instructors specializing in specific fields. Furthermore, they are unable to recognize students' emotions and adjust responses on the spot, meaning the quality of lessons is heavily influenced by the students' emotional state, making it difficult to provide an optimal learning environment. Furthermore, creators lack a way to monitor the performance and revenue status of virtual instructors in real time.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0757] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using generative artificial intelligence, means for storing the created virtual tutor in a database, and means for recognizing the user's emotions during a real-time session and adjusting responses. This allows students to receive the most appropriate lesson according to their emotions, and the creator can also check and manage the virtual tutor's performance and revenue status in real time.

[0758] "Generative AI" is an AI system that has the ability to generate new information or functions from data using specific algorithms or models.

[0759] A "virtual instructor" is a type of generated digital personality that has specialized knowledge in a particular field and provides educational services through dialogue.

[0760] A "database" is a system for organizing, storing, and retrieving a collection of data in a certain structure.

[0761] A "list" refers to a set of items or entries organized and displayed according to certain criteria.

[0762] "User" refers to an end user who uses the system to receive the services of a virtual instructor.

[0763] "Real-time session" refers to an interactive session in which the virtual instructor and the user interact directly and respond instantly.

[0764] "Means for recognizing emotions and adjusting responses" refers to technology that detects the user's emotional state in real time and adjusts the content and tone of the virtual instructor's responses based on that information.

[0765] "Revenue Sharing Mechanism" means the mechanism by which revenues earned based on the provision of Virtual Instructor Services are distributed to Instructor Creators.

[0766] A "dashboard" refers to an interface that allows users and creators to visually check the system's operational status and performance data.

[0767] This invention is a system that uses generative artificial intelligence to create a virtual instructor specialized in a specific field and then uses the virtual instructor to provide an interactive educational session. The system consists of the following main components:

[0768] Registration and prompt entry

[0769] First, the device displays a registration form for the creator, where the creator enters their name, email address, and a subject-specific prompt (e.g., "Tennis Technique Instructor"). The input is format-checked by a client-side script, such as JavaScript, and if valid, is sent to the server via an HTTP POST request.

[0770] Generate and save AI instructors

[0771] The server passes the received prompts to a generative AI to generate a virtual tutor specialized in a specific field, using a generative AI model such as OpenAI's GPT-4. The generated virtual tutor data is stored in a relational database such as MySQL or PostgreSQL along with the tutor ID, creator ID, generation date and time, and prompt content.

[0772] Students can select and book AI instructors

[0773] Next, the device displays a list of available virtual instructors. The list includes each instructor's profile, ratings, and areas of expertise. The user (student) selects a virtual instructor that best suits their needs and schedules a lesson using a calendar UI. The reservation information is checked again and sent to the server.

[0774] Lesson implementation and emotion recognition

[0775] The server prepares the real-time session based on the reserved date and time. When the lesson starts, the device receives a reminder notification and notifies the user. When the user clicks a link or button to start the session, a voice or video call begins.

[0776] During this time, an emotion engine built into the device recognizes the student's facial expressions and tone of voice in real time. The emotion engine typically uses emotion recognition APIs such as Google Cloud Vision or IBM Watson. The recognized emotion data is sent to a server, and the virtual instructor's responses are adjusted in real time. For example, if a student is confused, the virtual instructor can simplify their explanation.

[0777] Revenue sharing

[0778] After the lesson is over, the server updates the session revenue information and distributes the revenue to the virtual instructor creator. Payments are made via payment processing systems such as PayPal or Stripe. The device also updates the creator's dashboard with the new revenue information and instructor performance data.

[0779] Specific examples

[0780] For example, consider a case where a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson. The virtual teacher will start a real-time conversation based on the generated information. If the student asks, "I want to know more about 80s rock music," the emotion engine will analyze the student's facial expressions and tone of voice and recognize that the student is excited. In response, the virtual teacher will provide more detailed information or an interesting anecdote. If the student looks confused, the virtual teacher will simplify the explanation or provide additional information. After the lesson, the student's emotional data will be saved along with the conversation content, and revenue will be distributed based on that.

[0781] In this way, the system provides highly interactive and adaptive educational sessions, enhancing the learning experience for the student and benefiting the instructor creator.

[0782] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0783] Step 1:

[0784] The device displays the creator's registration form. The creator enters their name, email address, and a prompt (e.g., "Tennis Technique Instructor"). The input is done through an HTML form, and JavaScript is used to check the data for formatting. The entered data is temporarily stored on the client side.

[0785] Example input:

[0786] Name: Taro Yamada

[0787] Email address: yamada@example.com

[0788] Prompt: "An instructor who is knowledgeable about tennis techniques."

[0789] Example output:

[0790] {

[0791] "Name": "Yamada Taro",

[0792] "Email address": "yamada@example.com",

[0793] "prompt": "Tennis technique instructor"

[0794] }

[0795] Step 2:

[0796] The device checks the input and sends it to the server. The check checks whether required fields have been entered and validates the email address format. Once the check is complete, an HTTP POST request is sent to the server using a JavaScript AJAX request. The server receives this data and temporarily stores it in a database.

[0797] Example input:

[0798] {

[0799] "Name": "Yamada Taro",

[0800] "Email address": "yamada@example.com",

[0801] "prompt": "Tennis technique instructor"

[0802] }

[0803] Example output:

[0804] A new record is created in the database.

[0805] Step 3:

[0806] The server passes the received prompt to a generative AI to generate a virtual tutor. The prompt sentence is analyzed using a generative AI model (e.g., GPT-4). A script and dialogue model for the generated virtual tutor are created and returned as the generated result.

[0807] Example input:

[0808] Prompt: "An instructor who is knowledgeable about tennis techniques."

[0809] Example output:

[0810] Virtual instructor model (script, dialogue model)

[0811] Step 4:

[0812] The server saves the generated virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content. The data is inserted into a relational database, such as MySQL or PostgreSQL.

[0813] Example input:

[0814] Virtual instructor model (script, dialogue model), prompt sentences

[0815] Example output:

[0816] {

[0817] "Teacher ID": 12345,

[0818] "Creator ID": "yamada@example.com",

[0819] "Generation date and time": "2023-10-10T10:00:00Z",

[0820] "prompt": "Tennis technique instructor",

[0821] "ModelData": "{...}" / / Virtual instructor details

[0822] }

[0823] Step 5:

[0824] The device displays a list of available virtual instructors. The data to be displayed is retrieved from a database and displayed in the browser in a table or card format using HTML and JavaScript.

[0825] Example input:

[0826] A list of instructors retrieved by a database query

[0827] Example output:

[0828] Display instructor list in HTML format

[0829] Step 6:

[0830] The user (student) selects the desired virtual instructor and books a lesson. The student selects the desired date and time using the calendar UI and enters the information into the booking form. This data is sent to the server using AJAX.

[0831] Example input:

[0832] Instructor ID: 12345, Reservation date and time: "2023-10-15T15:00:00Z"

[0833] Example output:

[0834] {

[0835] "Reservation ID": 67890,

[0836] "Teacher ID": 12345,

[0837] "Student ID": "suzuki@example.com",

[0838] "Reservation date and time": "2023-10-15T15:00:00Z"

[0839] }

[0840] Step 7:

[0841] The server prepares the session. When the reserved date and time arrives, it generates resources for the session (e.g., a room ID for a video conference) and prepares data to be sent as a reminder to the device.

[0842] Example input:

[0843] Reservation Information

[0844] Example output:

[0845] {

[0846] "Session ID": "ROOM12345",

[0847] "Reminder": "Notification 10 minutes before lesson starts"

[0848] }

[0849] Step 8:

[0850] The device receives the reminder notification and notifies the user. The device displays a reminder to the user to start the lesson using a browser notification or a mobile app notification.

[0851] Example input:

[0852] Reminder notification data

[0853] Example output:

[0854] Pop-up notifications

[0855] Step 9:

[0856] A user (student) starts a session. A voice or video call is started by clicking a link or button. A chat screen or voice call function is enabled within the session.

[0857] Example input:

[0858] Signaling the start of a session

[0859] Example output:

[0860] Open the video call screen

[0861] Step 10:

[0862] The device's built-in emotion engine recognizes the student's facial expressions and tone of voice. The emotion engine collects data using a camera and microphone, analyzes the emotion data in real time using Google Cloud Vision and IBM Watson APIs, and sends it to a server.

[0863] Example input:

[0864] Facial expression data, voice data

[0865] Example output:

[0866] Real-time sentiment analysis data

[0867] Step 11:

[0868] The server stores data during real-time sessions, records conversation data and emotion data in a database, and saves the session details.

[0869] Example input:

[0870] Conversation data, emotion data

[0871] Example output:

[0872] Database Update

[0873] Step 12:

[0874] The server updates the revenue information and distributes the revenue to the creator. After the session ends, the revenue is calculated and paid to the creator via, for example, PayPal or Stripe.

[0875] Example input:

[0876] Session data, revenue sharing algorithms

[0877] Example output:

[0878] Payment Transactions

[0879] Step 13:

[0880] The device updates the creator's dashboard and UI to display new revenue information and instructor performance data in real time.

[0881] Example input:

[0882] Updated earnings and performance data

[0883] Example output:

[0884] Updated Dashboard View

[0885] (Application example 2)

[0886] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0887] Conventional virtual instructor systems were capable of providing users with interactive lessons specialized in specific fields, but lacked the ability to recognize the user's emotions and adjust responses in real time. As a result, there were problems with user satisfaction and learning effectiveness. It was also difficult to provide the interactivity that users experience in real-life interpersonal communication. Furthermore, the same problem existed in shopping, where users were unable to receive product guidance that responded to their emotions in real time. There is a need to solve these problems.

[0888] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0889] In this invention, the server includes means for creating a virtual instructor specialized in a specific field using a generative artificial intelligence, means for saving the virtual instructor created by the generative artificial intelligence in a database, means for displaying a list of the virtual instructors to allow a user to select one, means for reserving a session with the virtual instructor at a date and time desired by the user, means for conducting a real-time session with the virtual instructor based on the reservation, means for saving data of the real-time session, means for distributing revenue to the creator of the virtual instructor, means for analyzing the user's emotions using emotion recognition technology and adjusting the virtual instructor's responses, and means for saving the emotion recognition data in a database. This enables optimal responses according to the user's emotions and makes it possible to provide real-time interactive lessons and shopping assistant experiences.

[0890] "Generative AI" is an AI technology that generates domain-specific responses and information based on input prompts.

[0891] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, with knowledge and skills in a specific field, who provides interactive lessons to users.

[0892] The "database" is an information system for storing and managing information about the generated virtual instructor, session data, emotion recognition data, etc.

[0893] "Emotion recognition technology" is a technology that analyzes emotions from a user's facial expressions, tone of voice, etc., and adjusts the system's response based on the results.

[0894] A "real-time session" is an online session in which a virtual instructor and a user interact in real time to provide interactive feedback.

[0895] "Revenue sharing" is a system in which the creator of a virtual instructor is given a share of the profits and revenues earned from students.

[0896] A "server" is a computer system that processes user requests, generates virtual instructors, stores data, manages real-time sessions, and so on.

[0897] Overall system overview

[0898] This invention provides a system for creating and using virtual instructors specialized in specific fields. This system includes functions for user registration, virtual instructor creation, selection and reservation, real-time sessions, emotion recognition, data storage, and revenue sharing. This system is primarily implemented by a server, terminals (smartphones, PCs, etc.), and software using emotion recognition technology.

[0899] Hardware and software used

[0900] Hardware: Camera, microphone, smartphone, PC, server

[0901] Software: Generative artificial intelligence models (e.g., OpenAI GPT-4), emotion recognition engines, database management systems (e.g., MySQL), real-time communication tools (e.g., WebRTC)

[0902] A natural language explanation of the program's processing overview

[0903] User Registration

[0904] The device displays a user registration form, where the user enters their name, email address, specific areas of interest, etc. This information is sent to the server, which stores the user data in a database.

[0905] Virtual Instructor Creation

[0906] The server uses a generative artificial intelligence model to generate a virtual tutor based on prompts received from the user, such as "a tutor who is knowledgeable about fashion trends for Spring 2023." The generated virtual tutor is then stored in a database by the server.

[0907] Select and book a virtual teacher

[0908] The terminal retrieves a list of virtual instructors from the database and displays it to the user. The user selects the desired virtual instructor and date and time, and sends the reservation information to the server. The server saves the reservation data.

[0909] Conducting real-time sessions

[0910] At the scheduled time, the device will send a reminder to the user and start the real-time session. During the session, emotion recognition technology will analyze the user's facial expressions and tone of voice and transmit the results in real time to the server. The server will then adjust the virtual instructor's responses based on this emotion data.

[0911] Data storage

[0912] Once the session ends, the server stores the conversation data and emotion recognition data in a database.

[0913] Revenue sharing

[0914] After the session ends, the server updates the revenue data and distributes the revenue to the virtual instructor creator, who can check the revenue status through their device.

[0915] Specific examples

[0916] The user selects "A virtual assistant knowledgeable about fashionable Spring 2023 fashion" and books a shopping session. The real-time session begins, and the user turns on their camera to interact with the assistant. When the user asks, "What spring outfits do you recommend?", the emotion recognition engine analyzes the user's facial expression and recognizes that they are excited. The assistant then presents detailed and glamorous products.

[0917] Prompt Sentence Examples

[0918] Create a fashion-savvy virtual assistant that can provide information on fashion trends, especially for Spring 2023, and guide you through the shopping process in a fun and interactive way.

[0919] The above is a specific embodiment for carrying out the invention.

[0920] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0921] Step 1: User Registration

[0922] Input: The user enters their name, email address, and specific areas of interest (e.g., fashion, home appliances) into the device.

[0923] Processing: The terminal checks these input data and sends them to the server.

[0924] Data processing and calculation: The server stores the received data in a database.

[0925] Output: A new user record is created in the database and the user registration is complete.

[0926] Step 2: Create a virtual instructor

[0927] Input: A subject-specific prompt from the user (e.g., "Teacher who is knowledgeable about fashion trends for Spring 2023").

[0928] Processing: The server passes the prompts to a generative artificial intelligence model to generate a virtual instructor specialized in the specific subject area.

[0929] Data processing and calculation: A generative artificial intelligence model analyzes the prompts and generates data for a virtual instructor with domain-specific knowledge.

[0930] Output: The generated virtual instructor data is saved in a database via the server.

[0931] Step 3: Select and book a virtual teacher

[0932] Input: The user browses the list of virtual instructors through the terminal and selects the desired instructor and date and time.

[0933] Processing: The device sends information about the selected instructor and reservation date and time to the server.

[0934] Data processing and calculation: The server verifies the reservation data and stores it in the database.

[0935] Output: The virtual instructor and date / time are booked and a confirmation is sent to the user.

[0936] Step 4: Reminders

[0937] Enter your reservation information shortly before your reservation date and time.

[0938] Processing: The server sends a reminder notification to the device based on the reservation information.

[0939] Data processing and calculation: The server calculates the reservation date and time and sends a notification at the appropriate time.

[0940] Output: A reminder notification is displayed to the user on their device.

[0941] Step 5: Conduct a real-time session

[0942] Input: The user clicks the session start button on the terminal at the scheduled date and time.

[0943] Processing: The device initiates a real-time session and interacts with the server to stream session data.

[0944] Data processing and calculation: The server processes the session data and passes it to the emotion recognition engine in real time.

[0945] Output: A real-time interactive session is initiated and continues.

[0946] Step 6: Emotion recognition and response adjustment

[0947] Input: User's facial expression and voice tone data are sent from the device.

[0948] Processing: The emotion recognition engine analyzes the input data and identifies the user's emotion.

[0949] Data processing and calculation: The server adjusts the virtual instructor's responses based on the emotion recognition results.

[0950] Output: Tailored responses are provided to the user in real time.

[0951] Step 7: Save your data

[0952] Input: Conversation data and emotion recognition data from a real-time session.

[0953] Processing: The server stores these data in a database.

[0954] Data processing and calculations: After the session, the data is organized and stored.

[0955] Output: All session and sentiment data is stored securely in a database.

[0956] Step 8: Revenue sharing

[0957] Input: Revenue data after the session ends.

[0958] Processing: The server calculates the revenue data and distributes it to the creators of the virtual instructors.

[0959] Data processing and calculation: The server calculates the revenue and distributes it appropriately.

[0960] Output: Creator dashboard with revenue information.

[0961] The above are the specific processing steps of the system based on the present invention.

[0962] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0963] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0964] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0965] [Third embodiment]

[0966] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0967] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0968] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0969] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0970] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0971] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0972] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0973] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0974] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0975] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0976] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0977] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0978] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[0979] Overall system overview

[0980] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[0981] Program processing flow

[0982] Registration and prompt entry

[0983] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[0984] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[0985] The terminal checks these inputs and sends them to the server.

[0986] Generate and save AI instructors

[0987] The server passes the received prompts to a generation artificial intelligence to generate a virtual instructor specialized in a specific field.

[0988] The data of the generated virtual instructor (instructor ID, creator information, generation date and time, prompt content, etc.) is saved in the database.

[0989] Students can select and book AI instructors

[0990] The device displays a list of available virtual instructors to the student, including instructor profiles and past evaluations.

[0991] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form.

[0992] The terminal transmits this reservation information to the server.

[0993] Lesson implementation and log saving

[0994] The server prepares the real-time session based on the reserved date and time.

[0995] The device receives a reminder notification and notifies the user (student) before the lesson begins.

[0996] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[0997] The AI ​​instructor answers students' questions in real time, and the exchange is saved as a log in a database.

[0998] Revenue sharing

[0999] The server updates the revenue information after the lesson ends and distributes the revenue to the virtual instructor creator. The revenue calculation is based on the lesson fee and the student's evaluation.

[1000] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[1001] Specific examples

[1002] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music":

[1003] The creator fills in the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[1004] The generative artificial intelligence generates a virtual instructor based on this prompt and stores the data in a database.

[1005] Students select this instructor from the list and book a lesson on the desired date and time.

[1006] The session begins at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[1007] This system allows users (students) to take English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[1008] The processing flow will be explained below.

[1009] Step 1:

[1010] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and a prompt for the virtual instructor.

[1011] Step 2:

[1012] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[1013] Step 3:

[1014] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Input checks include checking whether all fields are filled in and whether the email address format is correct.

[1015] Step 4:

[1016] The server accepts the received data and passes prompts to the generative artificial intelligence to generate a virtual tutor specialized in a specific field, which analyzes the prompts and selects the required model.

[1017] Step 5:

[1018] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[1019] Step 6:

[1020] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[1021] Step 7:

[1022] The user (student) selects the desired instructor from a list of virtual instructors and enters the desired date and time in the lesson reservation form, which often uses a calendar-style UI.

[1023] Step 8:

[1024] The terminal sends the student's reservation information to the server, which includes the student ID, the selected virtual instructor ID, the desired date and time, etc.

[1025] Step 9:

[1026] The server receives the reservation information and stores it in a database, which confirms the reservation and allows a reservation confirmation notice to be sent to the user.

[1027] Step 10:

[1028] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[1029] Step 11:

[1030] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[1031] Step 12:

[1032] The user (student) clicks the session start button at the start time of the lesson, which starts a real-time session with the virtual instructor.

[1033] Step 13:

[1034] The device initiates a real-time session with the virtual instructor via a real-time chat system or voice call system. The student's questions and comments are sent to the virtual instructor, and answers are generated and displayed in real time.

[1035] Step 14:

[1036] The server stores conversation data from real-time sessions, including the questions asked by the student, the responses given by the virtual instructor, and session timestamps.

[1037] Step 15:

[1038] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[1039] Step 16:

[1040] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[1041] Example 1

[1042] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1043] In today's education market, instructors with specialized knowledge in specific fields are scarce and in high demand. Furthermore, there is a need for a system that allows students to conveniently take lessons according to their own schedule. There is also a need for a system that provides fair revenue distribution to virtual instructor creators. A new system is needed to solve these issues.

[1044] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1045] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using a generating AI, means for saving the virtual tutors created by the generating AI in a database, means for displaying a list of the virtual tutors to allow users to select one, means for reserving a session with the virtual tutor at a date and time desired by the user, means for conducting a real-time session with the virtual tutor based on the reservation, means for enabling a chat screen and voice call function during the real-time session, means for saving data on the real-time session, and means for distributing revenue to the creators of the virtual tutors. This makes it possible to efficiently meet the demand for tutors specialized in a specific field, enable students to take lessons conveniently, and provide appropriate revenue distribution to creators.

[1046] "Generative AI" is AI that uses natural language processing techniques to generate text or scripts based on specific input data or prompts.

[1047] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, who has knowledge of a specific field and provides education and guidance through dialogue with students.

[1048] A "database" is an electronic system or structure for efficiently storing, managing, and retrieving information.

[1049] "User" refers to an individual or organization that uses this system to create a virtual teacher or take English conversation lessons.

[1050] A "list" is a format that displays information about virtual instructors in a centralized manner, allowing users to select the instructors according to their purpose.

[1051] "Reservation" means the process by which a User reserves a session with a Virtual Instructor for a specific date and time.

[1052] A "real-time session" is a session in which a user and a virtual instructor interact in real time at a reserved date and time.

[1053] The "chat screen" is an interface that allows users to have text-based conversations with virtual instructors.

[1054] The "voice call function" is a function that allows users and virtual instructors to communicate via voice during a real-time session.

[1055] "Data" means records and information relating to your interactions and sessions with a Virtual Instructor.

[1056] "Revenue" means profit calculated based on lesson fees paid by users and other income sources.

[1057] "Distribution" means the act of appropriately sharing revenue with the creator of the virtual instructor and other parties.

[1058] A "reminder" is an electronic notification that alerts a user before a session begins.

[1059] A "dashboard" is an interface for visually displaying revenue status and other management information.

[1060] This invention is a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[1061] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[1062] The system is implemented using the following hardware and software:

[1063] Hardware: personal computer, smartphone, server (can be cloud-based)

[1064] Software: Registration form (web application), generative AI (e.g., OpenAI's GPT-4), database (MySQL, PostgreSQL, etc.)

[1065] The user (creator) initiates the process of generating a virtual tutor using a domain-specific prompt (e.g., "A tutor who has knowledge of 80's music and can answer related questions").

[1066] The generated virtual instructors are stored on the server side, and the user (student) selects the desired virtual instructor from this list to reserve a lesson. Based on the reserved date and time, the server prepares a real-time session and sends a notification to the terminal. The user (student) starts the real-time session by clicking the session start button, and the interaction is stored in the database.

[1067] For example:

[1068] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music," the process would be as follows:

[1069] 1. The creator fills the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[1070] 2. Generative AI generates a virtual instructor based on this prompt and stores the data in a database.

[1071] 3. The student selects this instructor from the list and reserves the lesson on the desired date and time.

[1072] 4. The session will begin at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[1073] This system allows users (students) to receive English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[1074] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1075] Step 1:

[1076] The device displays a registration form for the creator. The form contains input fields for name, email address, and a prompt for the virtual instructor. The input data includes name, email address, and prompt, and is collected using HTML and JavaScript. Based on this data input, appropriate validation is performed to ensure all required fields are filled in. The input data is converted to JSON format and sent to the server.

[1077] Step 2:

[1078] The server passes the received prompt to a generative artificial intelligence model (generative AI model), which generates a virtual tutor specialized in a specific field. The prompt and related metadata are used as input data. The generative AI model generates text data and scripts for the virtual tutor based on the input prompt, and outputs the results in JSON format. The output data includes the tutor ID, creator information, generation date and time, prompt content, etc. This data is returned to the server and stored in a database.

[1079] Step 3:

[1080] The device displays a list of available virtual instructors to the student. This list includes instructor profiles and past evaluations, and the latest instructor information is obtained from the server using an AJAX request. The displayed list is dynamically generated using HTML and CSS, and the user (student) can select the virtual instructor they are interested in. The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. The entered reservation information (student ID, instructor ID, desired date and time) is converted to JSON format and sent to the server.

[1081] Step 4:

[1082] The server prepares the real-time session based on the reserved date and time. Session details are set based on the reservation information, and a reminder notification is sent to the device. This notification is sent using an API such as Firebase Cloud Messaging. The user (student) receives the reminder notification and is given a warning before the session begins. The real-time session begins when the user (student) clicks the session start button. During the real-time session, the chat screen and voice call functions are enabled, and WebRTC technology is used for this.

[1083] Step 5:

[1084] The server records interactions during the real-time session, and the data (text messages, audio data) is stored in a database. This allows the history of the session content to be managed. After the session ends, the server updates the lesson revenue information and distributes revenue to the virtual instructor creator. Revenue is calculated based on the lesson fee and student ratings. Finally, the device updates the creator's dashboard, which displays revenue status and other management information in real time. This dashboard uses UI components such as graphs and tables to present information to the user in an easy-to-understand manner.

[1085] (Application example 1)

[1086] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] Traditional online lesson systems have had issues with users finding instructors who specialize in specific fields, and they are unable to answer questions or receive product explanations in real time. Another factor is the lack of virtual assistants who can provide detailed product explanations and support needed when shopping. This often leads to a poor user experience and delays in making purchasing decisions.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1089] In this invention, the server includes a means for creating a virtual assistant specialized in a specific field using a generative artificial intelligence, a means for storing the virtual assistant created by the generative artificial intelligence in a database, and a means for the virtual assistant to answer questions about products from the user in real time, thereby enabling the user to obtain detailed product descriptions and immediate answers to individual questions.

[1090] "Generative artificial intelligence" is a technology that generates specialized virtual assistants based on prompts entered by the user.

[1091] A "virtual assistant" is a digital agent created by generative artificial intelligence that provides real-time information and assistance in a specific field.

[1092] The "database" is a digital system for systematically storing and managing the created virtual assistant data and session log information.

[1093] A "real-time session" is a means of communication without time delay for a user to interact directly with a virtual assistant.

[1094] "Reminder notification" is a feature that sends advance notifications to users so that they do not forget the time of their reserved session.

[1095] The "dashboard" is an interface that allows the creator of a virtual assistant to view session revenue and management information in real time.

[1096] "Revenue sharing" is a system that distributes revenue to virtual assistant creators based on sessions with users.

[1097] This invention relates to a system that uses generative artificial intelligence to generate a virtual assistant specialized in a specific field, and provides product explanations and purchasing support through the assistant. Specific embodiments are described below.

[1098] Overall system overview

[1099] First, the user (creator) provides input prompts to the generation AI to create a virtual assistant specialized in a specific field. Based on these prompts, the generation AI generates a virtual assistant, and the data is stored in a database. Next, the user (consumer) can select the desired virtual assistant from this list and reserve a session at the desired date and time. At the reserved date and time, a real-time session is conducted, the session data is saved, and revenue is distributed to the virtual assistant creator.

[1100] Program processing flow

[1101] User (creator) registration and prompt input

[1102] The device displays the creator's registration form, which includes input fields for name, email address, and a prompt for a virtual assistant.

[1103] The user (creator) enters this information and fills in the prompts for a detailed description of the particular product.

[1104] The terminal checks the input and sends it to the server.

[1105] Creating and saving an AI assistant

[1106] The server passes the received prompts to a generative AI model to generate a virtual assistant specialized for a specific product (e.g., using GPT-4 as the generative AI model).

[1107] The data of the generated assistant (assistant ID, creator information, generation date and time, prompt content, etc.) is saved in a database (MySQL is used as the database).

[1108] User selection and reservation of AI assistants

[1109] The device displays a list of available virtual assistants, including each assistant's profile and the product categories they support.

[1110] The user selects the desired assistant and enters the desired date and time in the session reservation form.

[1111] The terminal transmits this reservation information to the server.

[1112] Conducting product explanation sessions

[1113] The server prepares the real-time session based on the reserved date and time.

[1114] The device receives a reminder notification and notifies the user before the session begins.

[1115] The user clicks the session start button to start a real-time session. During this time, the chat screen and voice call functions are enabled.

[1116] The virtual assistant answers users' product questions in real time, and the interactions are stored in a database.

[1117] Revenue sharing

[1118] The server updates the revenue information after the session ends and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee, user ratings, etc.

[1119] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[1120] Specific examples

[1121] For example, if a user (creator) wants to create a "virtual assistant that is knowledgeable about the latest smartwatches":

[1122] The creator fills in the form with the prompt, "A virtual assistant who can provide detailed instructions on the features and usage of the latest smartwatch."

[1123] The generative artificial intelligence generates a virtual assistant based on these prompts and stores the data in a database.

[1124] The user selects this assistant from the list and makes a reservation for the desired date and time.

[1125] The session will begin at the scheduled time and date, and the user can ask questions about their smartwatch, with the virtual assistant providing real-time answers.

[1126] Prompt Sentence Examples

[1127] "Generate a virtual shopping assistant that can provide detailed instructions on the features and usage of the latest smartwatch."

[1128] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1129] Step 1:

[1130] To create a specialized virtual assistant, a user (creator) uses a device to enter information into a registration form. Input items include a name, email address, and a prompt for the virtual assistant. The device sends this input data to the server, which receives the input data as a text prompt (e.g., "Please generate a virtual shopping assistant that can provide detailed explanations about the features and usage of the latest smartwatches.").

[1131] Step 2:

[1132] The server passes the received prompt to a generative AI model (e.g., GPT-4) to generate a virtual assistant. The generative AI model analyzes the prompt and generates data (text, voice data, etc.) for an assistant specialized in a specific field. The server stores this generated virtual assistant data in a database (e.g., a MySQL database). The stored data includes the assistant ID, creator information, generation date and time, prompt content, etc.

[1133] Step 3:

[1134] The terminal displays a list of available virtual assistants to the user. This list is obtained from the server and includes each assistant's profile and the product categories they can handle. The user selects the desired virtual assistant from this list.

[1135] Step 4:

[1136] The user selects the desired assistant and enters the desired date and time in the session reservation form via the terminal. The terminal sends this reservation information to the server. The sent reservation information includes the user's ID, the selected assistant ID, and the desired date and time.

[1137] Step 5:

[1138] The server prepares the real-time session based on the reservation information. Before the session starts, the server sends a reminder notification to the terminal to notify the user that the session is about to begin.

[1139] Step 6:

[1140] When the user clicks the session start button, the device starts a real-time session. During the session, the chat screen and voice call function are enabled, and the user can ask questions to the virtual assistant in real time.

[1141] Step 7:

[1142] The virtual assistant answers users' questions in real time, and the server logs these interactions in a database. The stored data includes the session timestamp, the user's question, and the assistant's response.

[1143] Step 8:

[1144] After the session ends, the server updates the revenue information and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee and the user's rating. The device updates the creator's dashboard based on this information and displays the revenue status in real time.

[1145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1146] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. This system can provide more interactive and adaptive lessons by combining it with an emotion engine that recognizes the user's emotions.

[1147] Overall system overview

[1148] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session begins at the reserved date and time, and not only is the session data saved, but the emotion engine recognizes the user's emotions and adjusts the response. This data is also saved as session data. In addition, revenue is distributed to the virtual tutor creator.

[1149] Program processing flow

[1150] Registration and prompt entry

[1151] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[1152] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[1153] The terminal checks these inputs and sends them to the server.

[1154] Generate and save AI instructors

[1155] The server passes the received prompts to the artificial intelligence generator, which then generates a virtual instructor specialized in a specific field. At this time, the prompts are analyzed and the necessary model is selected.

[1156] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[1157] Students can select and book AI instructors

[1158] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[1159] The user (student) selects the desired virtual instructor and enters the desired date and time into the lesson reservation form, which often uses a calendar-style UI.

[1160] The terminal transmits this reservation information to the server.

[1161] Lesson implementation and emotion recognition

[1162] The server prepares the real-time session based on the reserved date and time.

[1163] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[1164] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[1165] The emotion engine built into the device recognizes emotions from the student's facial expressions and voice. For example, it uses a camera and microphone to collect emotional information from facial expressions and voice tone.

[1166] The emotion data recognized by the emotion engine is sent to the server in real time, and the virtual instructor's response is adjusted accordingly. For example, if the student is confused, the virtual instructor will provide a simpler explanation.

[1167] The server stores conversation data and emotion data during the real-time session.

[1168] Revenue sharing

[1169] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[1170] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[1171] Specific examples

[1172] For example, if a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson:

[1173] Based on the generated information, the virtual instructor begins a real-time conversation with the student.

[1174] If a student asks, "I'd like to know more about '80s rock music," the emotion engine analyzes the student's facial expressions and tone of voice and recognizes that they are excited.

[1175] The virtual instructor responds by offering further details or amusing anecdotes, and if the student looks confused, they simplify the explanation or provide additional information.

[1176] After the lesson is over, the conversation content and the student's emotional data are saved, and revenue is distributed based on that.

[1177] This system allows students to receive lessons that best suit their emotions, and allows virtual instructor creators to earn higher revenue.

[1178] The processing flow will be explained below.

[1179] Program processing flow

[1180] Registration and prompt entry

[1181] Step 1:

[1182] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and virtual instructor prompts.

[1183] Step 2:

[1184] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[1185] Step 3:

[1186] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Basic input checks are performed (e.g., ensuring all required fields are filled in).

[1187] Generate and save AI instructors

[1188] Step 4:

[1189] The server accepts the received data and passes prompts to the generative AI to generate a virtual tutor specialized in a specific field. The prompts are analyzed and the necessary AI model is selected.

[1190] Step 5:

[1191] The server stores the generated virtual instructor data (e.g. instructor ID, creator ID, creation date and time, prompt content) in a database for later retrieval and reference.

[1192] Students can select and book AI instructors

[1193] Step 6:

[1194] The terminal retrieves a list of available virtual instructors from the database and displays it to the student. This list includes information such as the profile, evaluation, and areas of expertise of each virtual instructor.

[1195] Step 7:

[1196] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. A calendar UI allows for visual reservations.

[1197] Step 8:

[1198] The terminal sends the student's reservation information (e.g., student ID, selected virtual instructor ID, desired date and time) to the server.

[1199] Step 9:

[1200] The server receives the reservation information and stores it in a database. Once the reservation is confirmed, a reservation confirmation notification can be sent to the student.

[1201] Lesson implementation and emotion recognition

[1202] Step 10:

[1203] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[1204] Step 11:

[1205] The device receives a reminder notification and notifies the user (student) before the lesson begins. Notifications can be sent via browser or in-app notifications.

[1206] Step 12:

[1207] The user (student) clicks the Start Session button at the start of the lesson, which starts a real-time session with the virtual instructor.

[1208] Step 13:

[1209] The terminal starts a real-time session with the virtual instructor via a real-time chat system or a voice call system, and the student's questions and comments are sent to the virtual instructor, who then responds in real time.

[1210] Step 14:

[1211] The emotion engine built into the device uses a camera and microphone to recognize emotions from the students' facial expressions and voices, and this emotional data is analyzed in real time.

[1212] Step 15:

[1213] The server receives the student's emotional data in real time and adjusts the virtual instructor's response accordingly. For example, if the student is confused, the virtual instructor updates the response to provide a simpler explanation.

[1214] Step 16:

[1215] The server stores conversation and emotion data from the real-time session in a database, which can be used for later review and improvement.

[1216] Revenue sharing

[1217] Step 17:

[1218] After the lesson is completed, the server updates the revenue information and distributes the revenue to the virtual instructor creator based on the lesson fee, rating, and student usage.

[1219] Step 18:

[1220] The device updates the creator's dashboard with new revenue and other management information in real time, allowing creators to monitor their virtual tutor's performance and revenue.

[1221] Specific examples

[1222] For example, if a user (creator) wants to create a "virtual instructor who is knowledgeable about 80s music," they would enter "an instructor who has knowledge about 80s music and can answer questions" into the prompt. The generative AI generates a virtual instructor based on this prompt and stores it in a database. When a student selects this instructor, books a lesson, and starts a real-time session, the emotion engine analyzes the student's facial expressions and tone of voice to recognize emotions and adjust the virtual instructor's responses based on that information.

[1223] Example 2

[1224] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1225] Traditional online education systems face challenges in quickly and effectively providing instructors specializing in specific fields. Furthermore, they are unable to recognize students' emotions and adjust responses on the spot, meaning the quality of lessons is heavily influenced by the students' emotional state, making it difficult to provide an optimal learning environment. Furthermore, creators lack a way to monitor the performance and revenue status of virtual instructors in real time.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1227] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using generative artificial intelligence, means for storing the created virtual tutor in a database, and means for recognizing the user's emotions during a real-time session and adjusting responses. This allows students to receive the most appropriate lesson according to their emotions, and the creator can also check and manage the virtual tutor's performance and revenue status in real time.

[1228] "Generative AI" is an AI system that has the ability to generate new information or functions from data using specific algorithms or models.

[1229] A "virtual instructor" is a type of generated digital personality that has specialized knowledge in a particular field and provides educational services through dialogue.

[1230] A "database" is a system for organizing, storing, and retrieving a collection of data in a certain structure.

[1231] A "list" refers to a set of items or entries organized and displayed according to certain criteria.

[1232] "User" refers to an end user who uses the system to receive the services of a virtual instructor.

[1233] "Real-time session" refers to an interactive session in which the virtual instructor and the user interact directly and respond instantly.

[1234] "Means for recognizing emotions and adjusting responses" refers to technology that detects the user's emotional state in real time and adjusts the content and tone of the virtual instructor's responses based on that information.

[1235] "Revenue Sharing Mechanism" means the mechanism by which revenues earned based on the provision of Virtual Instructor Services are distributed to Instructor Creators.

[1236] A "dashboard" refers to an interface that allows users and creators to visually check the system's operational status and performance data.

[1237] This invention is a system that uses generative artificial intelligence to create a virtual instructor specialized in a specific field and then uses the virtual instructor to provide an interactive educational session. The system consists of the following main components:

[1238] Registration and prompt entry

[1239] First, the device displays a registration form for the creator, where the creator enters their name, email address, and a subject-specific prompt (e.g., "Tennis Technique Instructor"). The input is format-checked by a client-side script, such as JavaScript, and if valid, is sent to the server via an HTTP POST request.

[1240] Generate and save AI instructors

[1241] The server passes the received prompts to a generative AI to generate a virtual tutor specialized in a specific field, using a generative AI model such as OpenAI's GPT-4. The generated virtual tutor data is stored in a relational database such as MySQL or PostgreSQL along with the tutor ID, creator ID, generation date and time, and prompt content.

[1242] Students can select and book AI instructors

[1243] Next, the device displays a list of available virtual instructors. The list includes each instructor's profile, ratings, and areas of expertise. The user (student) selects a virtual instructor that best suits their needs and schedules a lesson using a calendar UI. The reservation information is checked again and sent to the server.

[1244] Lesson implementation and emotion recognition

[1245] The server prepares the real-time session based on the reserved date and time. When the lesson starts, the device receives a reminder notification and notifies the user. When the user clicks a link or button to start the session, a voice or video call begins.

[1246] During this time, an emotion engine built into the device recognizes the student's facial expressions and tone of voice in real time. The emotion engine typically uses emotion recognition APIs such as Google Cloud Vision or IBM Watson. The recognized emotion data is sent to a server, and the virtual instructor's responses are adjusted in real time. For example, if a student is confused, the virtual instructor can simplify their explanation.

[1247] Revenue sharing

[1248] After the lesson is over, the server updates the session revenue information and distributes the revenue to the virtual instructor creator. Payments are made via payment processing systems such as PayPal or Stripe. The device also updates the creator's dashboard with the new revenue information and instructor performance data.

[1249] Specific examples

[1250] For example, consider a case where a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson. The virtual teacher will start a real-time conversation based on the generated information. If the student asks, "I want to know more about 80s rock music," the emotion engine will analyze the student's facial expressions and tone of voice and recognize that the student is excited. In response, the virtual teacher will provide more detailed information or an interesting anecdote. If the student looks confused, the virtual teacher will simplify the explanation or provide additional information. After the lesson, the student's emotional data will be saved along with the conversation content, and revenue will be distributed based on that.

[1251] In this way, the system provides highly interactive and adaptive educational sessions, enhancing the learning experience for the student and benefiting the instructor creator.

[1252] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1253] Step 1:

[1254] The device displays the creator's registration form. The creator enters their name, email address, and a prompt (e.g., "Tennis Technique Instructor"). The input is done through an HTML form, and JavaScript is used to check the data for formatting. The entered data is temporarily stored on the client side.

[1255] Example input:

[1256] Name: Taro Yamada

[1257] Email address: yamada@example.com

[1258] Prompt: "An instructor who is knowledgeable about tennis techniques."

[1259] Example output:

[1260] {

[1261] "Name": "Yamada Taro",

[1262] "Email address": "yamada@example.com",

[1263] "prompt": "Tennis technique instructor"

[1264] }

[1265] Step 2:

[1266] The device checks the input and sends it to the server. The check checks whether required fields have been entered and validates the email address format. Once the check is complete, an HTTP POST request is sent to the server using a JavaScript AJAX request. The server receives this data and temporarily stores it in a database.

[1267] Example input:

[1268] {

[1269] "Name": "Yamada Taro",

[1270] "Email address": "yamada@example.com",

[1271] "prompt": "Tennis technique instructor"

[1272] }

[1273] Example output:

[1274] A new record is created in the database.

[1275] Step 3:

[1276] The server passes the received prompt to a generative AI to generate a virtual tutor. The prompt sentence is analyzed using a generative AI model (e.g., GPT-4). A script and dialogue model for the generated virtual tutor are created and returned as the generated result.

[1277] Example input:

[1278] Prompt: "An instructor who is knowledgeable about tennis techniques."

[1279] Example output:

[1280] Virtual instructor model (script, dialogue model)

[1281] Step 4:

[1282] The server saves the generated virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content. The data is inserted into a relational database, such as MySQL or PostgreSQL.

[1283] Example input:

[1284] Virtual instructor model (script, dialogue model), prompt sentences

[1285] Example output:

[1286] {

[1287] "Teacher ID": 12345,

[1288] "Creator ID": "yamada@example.com",

[1289] "Generation date and time": "2023-10-10T10:00:00Z",

[1290] "prompt": "Tennis technique instructor",

[1291] "ModelData": "{...}" / / Virtual instructor details

[1292] }

[1293] Step 5:

[1294] The device displays a list of available virtual instructors. The data to be displayed is retrieved from a database and displayed in the browser in a table or card format using HTML and JavaScript.

[1295] Example input:

[1296] A list of instructors retrieved by a database query

[1297] Example output:

[1298] Display instructor list in HTML format

[1299] Step 6:

[1300] The user (student) selects the desired virtual instructor and books a lesson. The student selects the desired date and time using the calendar UI and enters the information into the booking form. This data is sent to the server using AJAX.

[1301] Example input:

[1302] Instructor ID: 12345, Reservation date and time: "2023-10-15T15:00:00Z"

[1303] Example output:

[1304] {

[1305] "Reservation ID": 67890,

[1306] "Teacher ID": 12345,

[1307] "Student ID": "suzuki@example.com",

[1308] "Reservation date and time": "2023-10-15T15:00:00Z"

[1309] }

[1310] Step 7:

[1311] The server prepares the session. When the reserved date and time arrives, it generates resources for the session (e.g., a room ID for a video conference) and prepares data to be sent as a reminder to the device.

[1312] Example input:

[1313] Reservation Information

[1314] Example output:

[1315] {

[1316] "Session ID": "ROOM12345",

[1317] "Reminder": "Notification 10 minutes before lesson starts"

[1318] }

[1319] Step 8:

[1320] The device receives the reminder notification and notifies the user. The device displays a reminder to the user to start the lesson using a browser notification or a mobile app notification.

[1321] Example input:

[1322] Reminder notification data

[1323] Example output:

[1324] Pop-up notifications

[1325] Step 9:

[1326] A user (student) starts a session. A voice or video call is started by clicking a link or button. A chat screen or voice call function is enabled within the session.

[1327] Example input:

[1328] Signaling the start of a session

[1329] Example output:

[1330] Open the video call screen

[1331] Step 10:

[1332] The device's built-in emotion engine recognizes the student's facial expressions and tone of voice. The emotion engine collects data using a camera and microphone, analyzes the emotion data in real time using Google Cloud Vision and IBM Watson APIs, and sends it to a server.

[1333] Example input:

[1334] Facial expression data, voice data

[1335] Example output:

[1336] Real-time sentiment analysis data

[1337] Step 11:

[1338] The server stores data during real-time sessions, records conversation data and emotion data in a database, and saves the session details.

[1339] Example input:

[1340] Conversation data, emotion data

[1341] Example output:

[1342] Database Update

[1343] Step 12:

[1344] The server updates the revenue information and distributes the revenue to the creator. After the session ends, the revenue is calculated and paid to the creator via, for example, PayPal or Stripe.

[1345] Example input:

[1346] Session data, revenue sharing algorithms

[1347] Example output:

[1348] Payment Transactions

[1349] Step 13:

[1350] The device updates the creator's dashboard and UI to display new revenue information and instructor performance data in real time.

[1351] Example input:

[1352] Updated earnings and performance data

[1353] Example output:

[1354] Updated Dashboard View

[1355] (Application example 2)

[1356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1357] Conventional virtual instructor systems were capable of providing users with interactive lessons specialized in specific fields, but lacked the ability to recognize the user's emotions and adjust responses in real time. As a result, there were problems with user satisfaction and learning effectiveness. It was also difficult to provide the interactivity that users experience in real-life interpersonal communication. Furthermore, the same problem existed in shopping, where users were unable to receive product guidance that responded to their emotions in real time. There is a need to solve these problems.

[1358] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1359] In this invention, the server includes means for creating a virtual instructor specialized in a specific field using a generative artificial intelligence, means for saving the virtual instructor created by the generative artificial intelligence in a database, means for displaying a list of the virtual instructors to allow a user to select one, means for reserving a session with the virtual instructor at a date and time desired by the user, means for conducting a real-time session with the virtual instructor based on the reservation, means for saving data of the real-time session, means for distributing revenue to the creator of the virtual instructor, means for analyzing the user's emotions using emotion recognition technology and adjusting the virtual instructor's responses, and means for saving the emotion recognition data in a database. This enables optimal responses according to the user's emotions and makes it possible to provide real-time interactive lessons and shopping assistant experiences.

[1360] "Generative AI" is an AI technology that generates domain-specific responses and information based on input prompts.

[1361] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, with knowledge and skills in a specific field, who provides interactive lessons to users.

[1362] The "database" is an information system for storing and managing information about the generated virtual instructor, session data, emotion recognition data, etc.

[1363] "Emotion recognition technology" is a technology that analyzes emotions from a user's facial expressions, tone of voice, etc., and adjusts the system's response based on the results.

[1364] A "real-time session" is an online session in which a virtual instructor and a user interact in real time to provide interactive feedback.

[1365] "Revenue sharing" is a system in which the creator of a virtual instructor is given a share of the profits and revenues earned from students.

[1366] A "server" is a computer system that processes user requests, generates virtual instructors, stores data, manages real-time sessions, and so on.

[1367] Overall system overview

[1368] This invention provides a system for creating and using virtual instructors specialized in specific fields. This system includes functions for user registration, virtual instructor creation, selection and reservation, real-time sessions, emotion recognition, data storage, and revenue sharing. This system is primarily implemented by a server, terminals (smartphones, PCs, etc.), and software using emotion recognition technology.

[1369] Hardware and software used

[1370] Hardware: Camera, microphone, smartphone, PC, server

[1371] Software: Generative artificial intelligence models (e.g., OpenAI GPT-4), emotion recognition engines, database management systems (e.g., MySQL), real-time communication tools (e.g., WebRTC)

[1372] A natural language explanation of the program's processing overview

[1373] User Registration

[1374] The device displays a user registration form, where the user enters their name, email address, specific areas of interest, etc. This information is sent to the server, which stores the user data in a database.

[1375] Virtual Instructor Creation

[1376] The server uses a generative artificial intelligence model to generate a virtual tutor based on prompts received from the user, such as "a tutor who is knowledgeable about fashion trends for Spring 2023." The generated virtual tutor is then stored in a database by the server.

[1377] Select and book a virtual teacher

[1378] The terminal retrieves a list of virtual instructors from the database and displays it to the user. The user selects the desired virtual instructor and date and time, and sends the reservation information to the server. The server saves the reservation data.

[1379] Conducting real-time sessions

[1380] At the scheduled time, the device will send a reminder to the user and start the real-time session. During the session, emotion recognition technology will analyze the user's facial expressions and tone of voice and transmit the results in real time to the server. The server will then adjust the virtual instructor's responses based on this emotion data.

[1381] Data storage

[1382] Once the session ends, the server stores the conversation data and emotion recognition data in a database.

[1383] Revenue sharing

[1384] After the session ends, the server updates the revenue data and distributes the revenue to the virtual instructor creator, who can check the revenue status through their device.

[1385] Specific examples

[1386] The user selects "A virtual assistant knowledgeable about fashionable Spring 2023 fashion" and books a shopping session. The real-time session begins, and the user turns on their camera to interact with the assistant. When the user asks, "What spring outfits do you recommend?", the emotion recognition engine analyzes the user's facial expression and recognizes that they are excited. The assistant then presents detailed and glamorous products.

[1387] Prompt Sentence Examples

[1388] Create a fashion-savvy virtual assistant that can provide information on fashion trends, especially for Spring 2023, and guide you through the shopping process in a fun and interactive way.

[1389] The above is a specific embodiment for carrying out the invention.

[1390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1391] Step 1: User Registration

[1392] Input: The user enters their name, email address, and specific areas of interest (e.g., fashion, home appliances) into the device.

[1393] Processing: The terminal checks these input data and sends them to the server.

[1394] Data processing and calculation: The server stores the received data in a database.

[1395] Output: A new user record is created in the database and the user registration is complete.

[1396] Step 2: Create a virtual instructor

[1397] Input: A subject-specific prompt from the user (e.g., "Teacher who is knowledgeable about fashion trends for Spring 2023").

[1398] Processing: The server passes the prompts to a generative artificial intelligence model to generate a virtual instructor specialized in the specific subject area.

[1399] Data processing and calculation: A generative artificial intelligence model analyzes the prompts and generates data for a virtual instructor with domain-specific knowledge.

[1400] Output: The generated virtual instructor data is saved in a database via the server.

[1401] Step 3: Select and book a virtual teacher

[1402] Input: The user browses the list of virtual instructors through the terminal and selects the desired instructor and date and time.

[1403] Processing: The device sends information about the selected instructor and reservation date and time to the server.

[1404] Data processing and calculation: The server verifies the reservation data and stores it in the database.

[1405] Output: The virtual instructor and date / time are booked and a confirmation is sent to the user.

[1406] Step 4: Reminders

[1407] Enter your reservation information shortly before your reservation date and time.

[1408] Processing: The server sends a reminder notification to the device based on the reservation information.

[1409] Data processing and calculation: The server calculates the reservation date and time and sends a notification at the appropriate time.

[1410] Output: A reminder notification is displayed to the user on their device.

[1411] Step 5: Conduct a real-time session

[1412] Input: The user clicks the session start button on the terminal at the scheduled date and time.

[1413] Processing: The device initiates a real-time session and interacts with the server to stream session data.

[1414] Data processing and calculation: The server processes the session data and passes it to the emotion recognition engine in real time.

[1415] Output: A real-time interactive session is initiated and continues.

[1416] Step 6: Emotion recognition and response adjustment

[1417] Input: User's facial expression and voice tone data are sent from the device.

[1418] Processing: The emotion recognition engine analyzes the input data and identifies the user's emotion.

[1419] Data processing and calculation: The server adjusts the virtual instructor's responses based on the emotion recognition results.

[1420] Output: Tailored responses are provided to the user in real time.

[1421] Step 7: Save your data

[1422] Input: Conversation data and emotion recognition data from a real-time session.

[1423] Processing: The server stores these data in a database.

[1424] Data processing and calculations: After the session, the data is organized and stored.

[1425] Output: All session and sentiment data is stored securely in a database.

[1426] Step 8: Revenue sharing

[1427] Input: Revenue data after the session ends.

[1428] Processing: The server calculates the revenue data and distributes it to the creators of the virtual instructors.

[1429] Data processing and calculation: The server calculates the revenue and distributes it appropriately.

[1430] Output: Creator dashboard with revenue information.

[1431] The above are the specific processing steps of the system based on the present invention.

[1432] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1433] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1434] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1435] [Fourth embodiment]

[1436] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1437] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1438] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1439] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1440] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1441] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1442] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1443] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1444] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1445] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1446] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1447] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1448] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1449] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[1450] Overall system overview

[1451] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[1452] Program processing flow

[1453] Registration and prompt entry

[1454] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[1455] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[1456] The terminal checks these inputs and sends them to the server.

[1457] Generate and save AI instructors

[1458] The server passes the received prompts to a generation artificial intelligence to generate a virtual instructor specialized in a specific field.

[1459] The data of the generated virtual instructor (instructor ID, creator information, generation date and time, prompt content, etc.) is saved in the database.

[1460] Students can select and book AI instructors

[1461] The device displays a list of available virtual instructors to the student, including instructor profiles and past evaluations.

[1462] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form.

[1463] The terminal transmits this reservation information to the server.

[1464] Lesson implementation and log saving

[1465] The server prepares the real-time session based on the reserved date and time.

[1466] The device receives a reminder notification and notifies the user (student) before the lesson begins.

[1467] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[1468] The AI ​​instructor answers students' questions in real time, and the exchange is saved as a log in a database.

[1469] Revenue sharing

[1470] The server updates the revenue information after the lesson ends and distributes the revenue to the virtual instructor creator. The revenue calculation is based on the lesson fee and the student's evaluation.

[1471] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[1472] Specific examples

[1473] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music":

[1474] The creator fills in the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[1475] The generative artificial intelligence generates a virtual instructor based on this prompt and stores the data in a database.

[1476] Students select this instructor from the list and book a lesson on the desired date and time.

[1477] The session begins at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[1478] This system allows users (students) to take English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[1479] The processing flow will be explained below.

[1480] Step 1:

[1481] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and a prompt for the virtual instructor.

[1482] Step 2:

[1483] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[1484] Step 3:

[1485] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Input checks include checking whether all fields are filled in and whether the email address format is correct.

[1486] Step 4:

[1487] The server accepts the received data and passes prompts to the generative artificial intelligence to generate a virtual tutor specialized in a specific field, which analyzes the prompts and selects the required model.

[1488] Step 5:

[1489] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[1490] Step 6:

[1491] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[1492] Step 7:

[1493] The user (student) selects the desired instructor from a list of virtual instructors and enters the desired date and time in the lesson reservation form, which often uses a calendar-style UI.

[1494] Step 8:

[1495] The terminal sends the student's reservation information to the server, which includes the student ID, the selected virtual instructor ID, the desired date and time, etc.

[1496] Step 9:

[1497] The server receives the reservation information and stores it in a database, which confirms the reservation and allows a reservation confirmation notice to be sent to the user.

[1498] Step 10:

[1499] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[1500] Step 11:

[1501] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[1502] Step 12:

[1503] The user (student) clicks the session start button at the start time of the lesson, which starts a real-time session with the virtual instructor.

[1504] Step 13:

[1505] The device initiates a real-time session with the virtual instructor via a real-time chat system or voice call system. The student's questions and comments are sent to the virtual instructor, and answers are generated and displayed in real time.

[1506] Step 14:

[1507] The server stores conversation data from real-time sessions, including the questions asked by the student, the responses given by the virtual instructor, and session timestamps.

[1508] Step 15:

[1509] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[1510] Step 16:

[1511] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[1512] Example 1

[1513] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1514] In today's education market, instructors with specialized knowledge in specific fields are scarce and in high demand. Furthermore, there is a need for a system that allows students to conveniently take lessons according to their own schedule. There is also a need for a system that provides fair revenue distribution to virtual instructor creators. A new system is needed to solve these issues.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1516] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using a generating AI, means for saving the virtual tutors created by the generating AI in a database, means for displaying a list of the virtual tutors to allow users to select one, means for reserving a session with the virtual tutor at a date and time desired by the user, means for conducting a real-time session with the virtual tutor based on the reservation, means for enabling a chat screen and voice call function during the real-time session, means for saving data on the real-time session, and means for distributing revenue to the creators of the virtual tutors. This makes it possible to efficiently meet the demand for tutors specialized in a specific field, enable students to take lessons conveniently, and provide appropriate revenue distribution to creators.

[1517] "Generative AI" is AI that uses natural language processing techniques to generate text or scripts based on specific input data or prompts.

[1518] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, who has knowledge of a specific field and provides education and guidance through dialogue with students.

[1519] A "database" is an electronic system or structure for efficiently storing, managing, and retrieving information.

[1520] "User" refers to an individual or organization that uses this system to create a virtual teacher or take English conversation lessons.

[1521] A "list" is a format that displays information about virtual instructors in a centralized manner, allowing users to select the instructors according to their purpose.

[1522] "Reservation" means the process by which a User reserves a session with a Virtual Instructor for a specific date and time.

[1523] A "real-time session" is a session in which a user and a virtual instructor interact in real time at a reserved date and time.

[1524] The "chat screen" is an interface that allows users to have text-based conversations with virtual instructors.

[1525] The "voice call function" is a function that allows users and virtual instructors to communicate via voice during a real-time session.

[1526] "Data" means records and information relating to your interactions and sessions with a Virtual Instructor.

[1527] "Revenue" means profit calculated based on lesson fees paid by users and other income sources.

[1528] "Distribution" means the act of appropriately sharing revenue with the creator of the virtual instructor and other parties.

[1529] A "reminder" is an electronic notification that alerts a user before a session begins.

[1530] A "dashboard" is an interface for visually displaying revenue status and other management information.

[1531] This invention is a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. Specific embodiments of this system are described in detail below.

[1532] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session is conducted, and the session data is saved, with revenue also being distributed to the virtual tutor creator.

[1533] The system is implemented using the following hardware and software:

[1534] Hardware: personal computer, smartphone, server (can be cloud-based)

[1535] Software: Registration form (web application), generative AI (e.g., OpenAI's GPT-4), database (MySQL, PostgreSQL, etc.)

[1536] The user (creator) initiates the process of generating a virtual tutor using a domain-specific prompt (e.g., "A tutor who has knowledge of 80's music and can answer related questions").

[1537] The generated virtual instructors are stored on the server side, and the user (student) selects the desired virtual instructor from this list to reserve a lesson. Based on the reserved date and time, the server prepares a real-time session and sends a notification to the terminal. The user (student) starts the real-time session by clicking the session start button, and the interaction is stored in the database.

[1538] For example:

[1539] For example, if a user (creator) wants to create a "virtual teacher who is knowledgeable about 80's music," the process would be as follows:

[1540] 1. The creator fills the form with the prompt, "A teacher who has knowledge of 80s music and can answer related questions."

[1541] 2. Generative AI generates a virtual instructor based on this prompt and stores the data in a database.

[1542] 3. The student selects this instructor from the list and reserves the lesson on the desired date and time.

[1543] 4. The session will begin at the scheduled date and time, and participants can ask questions about 80s music, with the virtual instructor answering in real time.

[1544] This system allows users (students) to receive English conversation lessons tailored to their interests and needs, and allows users (creators) to earn revenue.

[1545] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1546] Step 1:

[1547] The device displays a registration form for the creator. The form contains input fields for name, email address, and a prompt for the virtual instructor. The input data includes name, email address, and prompt, and is collected using HTML and JavaScript. Based on this data input, appropriate validation is performed to ensure all required fields are filled in. The input data is converted to JSON format and sent to the server.

[1548] Step 2:

[1549] The server passes the received prompt to a generative artificial intelligence model (generative AI model), which generates a virtual tutor specialized in a specific field. The prompt and related metadata are used as input data. The generative AI model generates text data and scripts for the virtual tutor based on the input prompt, and outputs the results in JSON format. The output data includes the tutor ID, creator information, generation date and time, prompt content, etc. This data is returned to the server and stored in a database.

[1550] Step 3:

[1551] The device displays a list of available virtual instructors to the student. This list includes instructor profiles and past evaluations, and the latest instructor information is obtained from the server using an AJAX request. The displayed list is dynamically generated using HTML and CSS, and the user (student) can select the virtual instructor they are interested in. The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. The entered reservation information (student ID, instructor ID, desired date and time) is converted to JSON format and sent to the server.

[1552] Step 4:

[1553] The server prepares the real-time session based on the reserved date and time. Session details are set based on the reservation information, and a reminder notification is sent to the device. This notification is sent using an API such as Firebase Cloud Messaging. The user (student) receives the reminder notification and is given a warning before the session begins. The real-time session begins when the user (student) clicks the session start button. During the real-time session, the chat screen and voice call functions are enabled, and WebRTC technology is used for this.

[1554] Step 5:

[1555] The server records interactions during the real-time session, and the data (text messages, audio data) is stored in a database. This allows the history of the session content to be managed. After the session ends, the server updates the lesson revenue information and distributes revenue to the virtual instructor creator. Revenue is calculated based on the lesson fee and student ratings. Finally, the device updates the creator's dashboard, which displays revenue status and other management information in real time. This dashboard uses UI components such as graphs and tables to present information to the user in an easy-to-understand manner.

[1556] (Application example 1)

[1557] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] Traditional online lesson systems have had issues with users finding instructors who specialize in specific fields, and they are unable to answer questions or receive product explanations in real time. Another factor is the lack of virtual assistants who can provide detailed product explanations and support needed when shopping. This often leads to a poor user experience and delays in making purchasing decisions.

[1559] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1560] In this invention, the server includes a means for creating a virtual assistant specialized in a specific field using a generative artificial intelligence, a means for storing the virtual assistant created by the generative artificial intelligence in a database, and a means for the virtual assistant to answer questions about products from the user in real time, thereby enabling the user to obtain detailed product descriptions and immediate answers to individual questions.

[1561] "Generative artificial intelligence" is a technology that generates specialized virtual assistants based on prompts entered by the user.

[1562] A "virtual assistant" is a digital agent created by generative artificial intelligence that provides real-time information and assistance in a specific field.

[1563] The "database" is a digital system for systematically storing and managing the created virtual assistant data and session log information.

[1564] A "real-time session" is a means of communication without time delay for a user to interact directly with a virtual assistant.

[1565] "Reminder notification" is a feature that sends advance notifications to users so that they do not forget the time of their reserved session.

[1566] The "dashboard" is an interface that allows the creator of a virtual assistant to view session revenue and management information in real time.

[1567] "Revenue sharing" is a system that distributes revenue to virtual assistant creators based on sessions with users.

[1568] This invention relates to a system that uses generative artificial intelligence to generate a virtual assistant specialized in a specific field, and provides product explanations and purchasing support through the assistant. Specific embodiments are described below.

[1569] Overall system overview

[1570] First, the user (creator) provides input prompts to the generation AI to create a virtual assistant specialized in a specific field. Based on these prompts, the generation AI generates a virtual assistant, and the data is stored in a database. Next, the user (consumer) can select the desired virtual assistant from this list and reserve a session at the desired date and time. At the reserved date and time, a real-time session is conducted, the session data is saved, and revenue is distributed to the virtual assistant creator.

[1571] Program processing flow

[1572] User (creator) registration and prompt input

[1573] The device displays the creator's registration form, which includes input fields for name, email address, and a prompt for a virtual assistant.

[1574] The user (creator) enters this information and fills in the prompts for a detailed description of the particular product.

[1575] The terminal checks the input and sends it to the server.

[1576] Creating and saving an AI assistant

[1577] The server passes the received prompts to a generative AI model to generate a virtual assistant specialized for a specific product (e.g., using GPT-4 as the generative AI model).

[1578] The data of the generated assistant (assistant ID, creator information, generation date and time, prompt content, etc.) is saved in a database (MySQL is used as the database).

[1579] User selection and reservation of AI assistants

[1580] The device displays a list of available virtual assistants, including each assistant's profile and the product categories they support.

[1581] The user selects the desired assistant and enters the desired date and time in the session reservation form.

[1582] The terminal transmits this reservation information to the server.

[1583] Conducting product explanation sessions

[1584] The server prepares the real-time session based on the reserved date and time.

[1585] The device receives a reminder notification and notifies the user before the session begins.

[1586] The user clicks the session start button to start a real-time session. During this time, the chat screen and voice call functions are enabled.

[1587] The virtual assistant answers users' product questions in real time, and the interactions are stored in a database.

[1588] Revenue sharing

[1589] The server updates the revenue information after the session ends and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee, user ratings, etc.

[1590] The device updates the creator's dashboard, displaying earnings status and other management information in real time.

[1591] Specific examples

[1592] For example, if a user (creator) wants to create a "virtual assistant that is knowledgeable about the latest smartwatches":

[1593] The creator fills in the form with the prompt, "A virtual assistant who can provide detailed instructions on the features and usage of the latest smartwatch."

[1594] The generative artificial intelligence generates a virtual assistant based on these prompts and stores the data in a database.

[1595] The user selects this assistant from the list and makes a reservation for the desired date and time.

[1596] The session will begin at the scheduled time and date, and the user can ask questions about their smartwatch, with the virtual assistant providing real-time answers.

[1597] Prompt Sentence Examples

[1598] "Generate a virtual shopping assistant that can provide detailed instructions on the features and usage of the latest smartwatch."

[1599] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1600] Step 1:

[1601] To create a specialized virtual assistant, a user (creator) uses a device to enter information into a registration form. Input items include a name, email address, and a prompt for the virtual assistant. The device sends this input data to the server, which receives the input data as a text prompt (e.g., "Please generate a virtual shopping assistant that can provide detailed explanations about the features and usage of the latest smartwatches.").

[1602] Step 2:

[1603] The server passes the received prompt to a generative AI model (e.g., GPT-4) to generate a virtual assistant. The generative AI model analyzes the prompt and generates data (text, voice data, etc.) for an assistant specialized in a specific field. The server stores this generated virtual assistant data in a database (e.g., a MySQL database). The stored data includes the assistant ID, creator information, generation date and time, prompt content, etc.

[1604] Step 3:

[1605] The terminal displays a list of available virtual assistants to the user. This list is obtained from the server and includes each assistant's profile and the product categories they can handle. The user selects the desired virtual assistant from this list.

[1606] Step 4:

[1607] The user selects the desired assistant and enters the desired date and time in the session reservation form via the terminal. The terminal sends this reservation information to the server. The sent reservation information includes the user's ID, the selected assistant ID, and the desired date and time.

[1608] Step 5:

[1609] The server prepares the real-time session based on the reservation information. Before the session starts, the server sends a reminder notification to the terminal to notify the user that the session is about to begin.

[1610] Step 6:

[1611] When the user clicks the session start button, the device starts a real-time session. During the session, the chat screen and voice call function are enabled, and the user can ask questions to the virtual assistant in real time.

[1612] Step 7:

[1613] The virtual assistant answers users' questions in real time, and the server logs these interactions in a database. The stored data includes the session timestamp, the user's question, and the assistant's response.

[1614] Step 8:

[1615] After the session ends, the server updates the revenue information and distributes the revenue to the virtual assistant creator. The revenue calculation is based on the session fee and the user's rating. The device updates the creator's dashboard based on this information and displays the revenue status in real time.

[1616] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1617] This invention relates to a system that uses artificial intelligence to create virtual instructors specialized in specific fields and provides English conversation lessons using these virtual instructors. This system can provide more interactive and adaptive lessons by combining it with an emotion engine that recognizes the user's emotions.

[1618] Overall system overview

[1619] First, the user (creator) provides input prompts to the generation AI to create a virtual tutor specialized in a specific field. Based on these prompts, the generation AI generates a virtual tutor, and the data is stored in a database. Next, the user (student) can select the desired virtual tutor from this list and book a lesson at the desired date and time. A real-time session begins at the reserved date and time, and not only is the session data saved, but the emotion engine recognizes the user's emotions and adjusts the response. This data is also saved as session data. In addition, revenue is distributed to the virtual tutor creator.

[1620] Program processing flow

[1621] Registration and prompt entry

[1622] The device displays the author's registration form, which includes input fields for name, email address, and a prompt for the virtual instructor.

[1623] The user (creator) enters this information and fills in a domain-specific prompt (e.g., "Tennis technique instructor").

[1624] The terminal checks these inputs and sends them to the server.

[1625] Generate and save AI instructors

[1626] The server passes the received prompts to the artificial intelligence generator, which then generates a virtual instructor specialized in a specific field. At this time, the prompts are analyzed and the necessary model is selected.

[1627] The server saves the created virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content.

[1628] Students can select and book AI instructors

[1629] The device displays a list of available virtual instructors to the student, including information about each virtual instructor, such as their profile, ratings, and areas of expertise.

[1630] The user (student) selects the desired virtual instructor and enters the desired date and time into the lesson reservation form, which often uses a calendar-style UI.

[1631] The terminal transmits this reservation information to the server.

[1632] Lesson implementation and emotion recognition

[1633] The server prepares the real-time session based on the reserved date and time.

[1634] The device receives a reminder notification and notifies the user (student) before the lesson starts. Browser notifications or in-app notifications are often used.

[1635] The user (student) clicks the session start button to start the real-time session. During this time, the chat screen and voice call functions are enabled.

[1636] The emotion engine built into the device recognizes emotions from the student's facial expressions and voice. For example, it uses a camera and microphone to collect emotional information from facial expressions and voice tone.

[1637] The emotion data recognized by the emotion engine is sent to the server in real time, and the virtual instructor's response is adjusted accordingly. For example, if the student is confused, the virtual instructor will provide a simpler explanation.

[1638] The server stores conversation data and emotion data during the real-time session.

[1639] Revenue sharing

[1640] The server updates the revenue information after the lesson is completed and distributes the revenue to the virtual teacher creator based on the lesson fee and usage.

[1641] The device updates the creator's dashboard with new revenue and management information in real time, allowing the creator to monitor the performance of their virtual instructor.

[1642] Specific examples

[1643] For example, if a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson:

[1644] Based on the generated information, the virtual instructor begins a real-time conversation with the student.

[1645] If a student asks, "I'd like to know more about '80s rock music," the emotion engine analyzes the student's facial expressions and tone of voice and recognizes that they are excited.

[1646] The virtual instructor responds by offering further details or amusing anecdotes, and if the student looks confused, they simplify the explanation or provide additional information.

[1647] After the lesson is over, the conversation content and the student's emotional data are saved, and revenue is distributed based on that.

[1648] This system allows students to receive lessons that best suit their emotions, and allows virtual instructor creators to earn higher revenue.

[1649] The processing flow will be explained below.

[1650] Program processing flow

[1651] Registration and prompt entry

[1652] Step 1:

[1653] The device displays the creator's registration form on the screen, which includes input fields for name, email address, and virtual instructor prompts.

[1654] Step 2:

[1655] A user (creator) fills out a registration form with their name, email address, and a prompt for a virtual instructor specializing in a specific subject, such as "an instructor who specializes in tennis techniques."

[1656] Step 3:

[1657] The device validates the user (creator) input using JavaScript and sends it to the server in the appropriate format. Basic input checks are performed (e.g., ensuring all required fields are filled in).

[1658] Generate and save AI instructors

[1659] Step 4:

[1660] The server accepts the received data and passes prompts to the generative AI to generate a virtual tutor specialized in a specific field. The prompts are analyzed and the necessary AI model is selected.

[1661] Step 5:

[1662] The server stores the generated virtual instructor data (e.g. instructor ID, creator ID, creation date and time, prompt content) in a database for later retrieval and reference.

[1663] Students can select and book AI instructors

[1664] Step 6:

[1665] The terminal retrieves a list of available virtual instructors from the database and displays it to the student. This list includes information such as the profile, evaluation, and areas of expertise of each virtual instructor.

[1666] Step 7:

[1667] The user (student) selects the desired virtual instructor and enters the desired date and time in the lesson reservation form. A calendar UI allows for visual reservations.

[1668] Step 8:

[1669] The terminal sends the student's reservation information (e.g., student ID, selected virtual instructor ID, desired date and time) to the server.

[1670] Step 9:

[1671] The server receives the reservation information and stores it in a database. Once the reservation is confirmed, a reservation confirmation notification can be sent to the student.

[1672] Lesson implementation and emotion recognition

[1673] Step 10:

[1674] The server will send a reminder to the student's device just before the scheduled time. This notification will be sent via push notification or email.

[1675] Step 11:

[1676] The device receives a reminder notification and notifies the user (student) before the lesson begins. Notifications can be sent via browser or in-app notifications.

[1677] Step 12:

[1678] The user (student) clicks the Start Session button at the start of the lesson, which starts a real-time session with the virtual instructor.

[1679] Step 13:

[1680] The terminal starts a real-time session with the virtual instructor via a real-time chat system or a voice call system, and the student's questions and comments are sent to the virtual instructor, who then responds in real time.

[1681] Step 14:

[1682] The emotion engine built into the device uses a camera and microphone to recognize emotions from the students' facial expressions and voices, and this emotional data is analyzed in real time.

[1683] Step 15:

[1684] The server receives the student's emotional data in real time and adjusts the virtual instructor's response accordingly. For example, if the student is confused, the virtual instructor updates the response to provide a simpler explanation.

[1685] Step 16:

[1686] The server stores conversation and emotion data from the real-time session in a database, which can be used for later review and improvement.

[1687] Revenue sharing

[1688] Step 17:

[1689] After the lesson is completed, the server updates the revenue information and distributes the revenue to the virtual instructor creator based on the lesson fee, rating, and student usage.

[1690] Step 18:

[1691] The device updates the creator's dashboard with new revenue and other management information in real time, allowing creators to monitor their virtual tutor's performance and revenue.

[1692] Specific examples

[1693] For example, if a user (creator) wants to create a "virtual instructor who is knowledgeable about 80s music," they would enter "an instructor who has knowledge about 80s music and can answer questions" into the prompt. The generative AI generates a virtual instructor based on this prompt and stores it in a database. When a student selects this instructor, books a lesson, and starts a real-time session, the emotion engine analyzes the student's facial expressions and tone of voice to recognize emotions and adjust the virtual instructor's responses based on that information.

[1694] Example 2

[1695] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1696] Traditional online education systems face challenges in quickly and effectively providing instructors specializing in specific fields. Furthermore, they are unable to recognize students' emotions and adjust responses on the spot, meaning the quality of lessons is heavily influenced by the students' emotional state, making it difficult to provide an optimal learning environment. Furthermore, creators lack a way to monitor the performance and revenue status of virtual instructors in real time.

[1697] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1698] In this invention, the server includes means for creating a virtual tutor specialized in a specific field using generative artificial intelligence, means for storing the created virtual tutor in a database, and means for recognizing the user's emotions during a real-time session and adjusting responses. This allows students to receive the most appropriate lesson according to their emotions, and the creator can also check and manage the virtual tutor's performance and revenue status in real time.

[1699] "Generative AI" is an AI system that has the ability to generate new information or functions from data using specific algorithms or models.

[1700] A "virtual instructor" is a type of generated digital personality that has specialized knowledge in a particular field and provides educational services through dialogue.

[1701] A "database" is a system for organizing, storing, and retrieving a collection of data in a certain structure.

[1702] A "list" refers to a set of items or entries organized and displayed according to certain criteria.

[1703] "User" refers to an end user who uses the system to receive the services of a virtual instructor.

[1704] "Real-time session" refers to an interactive session in which the virtual instructor and the user interact directly and respond instantly.

[1705] "Means for recognizing emotions and adjusting responses" refers to technology that detects the user's emotional state in real time and adjusts the content and tone of the virtual instructor's responses based on that information.

[1706] "Revenue Sharing Mechanism" means the mechanism by which revenues earned based on the provision of Virtual Instructor Services are distributed to Instructor Creators.

[1707] A "dashboard" refers to an interface that allows users and creators to visually check the system's operational status and performance data.

[1708] This invention is a system that uses generative artificial intelligence to create a virtual instructor specialized in a specific field and then uses the virtual instructor to provide an interactive educational session. The system consists of the following main components:

[1709] Registration and prompt entry

[1710] First, the device displays a registration form for the creator, where the creator enters their name, email address, and a subject-specific prompt (e.g., "Tennis Technique Instructor"). The input is format-checked by a client-side script, such as JavaScript, and if valid, is sent to the server via an HTTP POST request.

[1711] Generate and save AI instructors

[1712] The server passes the received prompts to a generative AI to generate a virtual tutor specialized in a specific field, using a generative AI model such as OpenAI's GPT-4. The generated virtual tutor data is stored in a relational database such as MySQL or PostgreSQL along with the tutor ID, creator ID, generation date and time, and prompt content.

[1713] Students can select and book AI instructors

[1714] Next, the device displays a list of available virtual instructors. The list includes each instructor's profile, ratings, and areas of expertise. The user (student) selects a virtual instructor that best suits their needs and schedules a lesson using a calendar UI. The reservation information is checked again and sent to the server.

[1715] Lesson implementation and emotion recognition

[1716] The server prepares the real-time session based on the reserved date and time. When the lesson starts, the device receives a reminder notification and notifies the user. When the user clicks a link or button to start the session, a voice or video call begins.

[1717] During this time, an emotion engine built into the device recognizes the student's facial expressions and tone of voice in real time. The emotion engine typically uses emotion recognition APIs such as Google Cloud Vision or IBM Watson. The recognized emotion data is sent to a server, and the virtual instructor's responses are adjusted in real time. For example, if a student is confused, the virtual instructor can simplify their explanation.

[1718] Revenue sharing

[1719] After the lesson is over, the server updates the session revenue information and distributes the revenue to the virtual instructor creator. Payments are made via payment processing systems such as PayPal or Stripe. The device also updates the creator's dashboard with the new revenue information and instructor performance data.

[1720] Specific examples

[1721] For example, consider a case where a student selects a "virtual teacher who is knowledgeable about 80s music" and books a lesson. The virtual teacher will start a real-time conversation based on the generated information. If the student asks, "I want to know more about 80s rock music," the emotion engine will analyze the student's facial expressions and tone of voice and recognize that the student is excited. In response, the virtual teacher will provide more detailed information or an interesting anecdote. If the student looks confused, the virtual teacher will simplify the explanation or provide additional information. After the lesson, the student's emotional data will be saved along with the conversation content, and revenue will be distributed based on that.

[1722] In this way, the system provides highly interactive and adaptive educational sessions, enhancing the learning experience for the student and benefiting the instructor creator.

[1723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1724] Step 1:

[1725] The device displays the creator's registration form. The creator enters their name, email address, and a prompt (e.g., "Tennis Technique Instructor"). The input is done through an HTML form, and JavaScript is used to check the data for formatting. The entered data is temporarily stored on the client side.

[1726] Example input:

[1727] Name: Taro Yamada

[1728] Email address: yamada@example.com

[1729] Prompt: "An instructor who is knowledgeable about tennis techniques."

[1730] Example output:

[1731] {

[1732] "Name": "Yamada Taro",

[1733] "Email address": "yamada@example.com",

[1734] "prompt": "Tennis technique instructor"

[1735] }

[1736] Step 2:

[1737] The device checks the input and sends it to the server. The check checks whether required fields have been entered and validates the email address format. Once the check is complete, an HTTP POST request is sent to the server using a JavaScript AJAX request. The server receives this data and temporarily stores it in a database.

[1738] Example input:

[1739] {

[1740] "Name": "Yamada Taro",

[1741] "Email address": "yamada@example.com",

[1742] "prompt": "Tennis technique instructor"

[1743] }

[1744] Example output:

[1745] A new record is created in the database.

[1746] Step 3:

[1747] The server passes the received prompt to a generative AI to generate a virtual tutor. The prompt sentence is analyzed using a generative AI model (e.g., GPT-4). A script and dialogue model for the generated virtual tutor are created and returned as the generated result.

[1748] Example input:

[1749] Prompt: "An instructor who is knowledgeable about tennis techniques."

[1750] Example output:

[1751] Virtual instructor model (script, dialogue model)

[1752] Step 4:

[1753] The server saves the generated virtual instructor data in a database, including the instructor ID, creator ID, creation date and time, and prompt content. The data is inserted into a relational database, such as MySQL or PostgreSQL.

[1754] Example input:

[1755] Virtual instructor model (script, dialogue model), prompt sentences

[1756] Example output:

[1757] {

[1758] "Teacher ID": 12345,

[1759] "Creator ID": "yamada@example.com",

[1760] "Generation date and time": "2023-10-10T10:00:00Z",

[1761] "prompt": "Tennis technique instructor",

[1762] "ModelData": "{...}" / / Virtual instructor details

[1763] }

[1764] Step 5:

[1765] The device displays a list of available virtual instructors. The data to be displayed is retrieved from a database and displayed in the browser in a table or card format using HTML and JavaScript.

[1766] Example input:

[1767] A list of instructors retrieved by a database query

[1768] Example output:

[1769] Display instructor list in HTML format

[1770] Step 6:

[1771] The user (student) selects the desired virtual instructor and books a lesson. The student selects the desired date and time using the calendar UI and enters the information into the booking form. This data is sent to the server using AJAX.

[1772] Example input:

[1773] Instructor ID: 12345, Reservation date and time: "2023-10-15T15:00:00Z"

[1774] Example output:

[1775] {

[1776] "Reservation ID": 67890,

[1777] "Teacher ID": 12345,

[1778] "Student ID": "suzuki@example.com",

[1779] "Reservation date and time": "2023-10-15T15:00:00Z"

[1780] }

[1781] Step 7:

[1782] The server prepares the session. When the reserved date and time arrives, it generates resources for the session (e.g., a room ID for a video conference) and prepares data to be sent as a reminder to the device.

[1783] Example input:

[1784] Reservation Information

[1785] Example output:

[1786] {

[1787] "Session ID": "ROOM12345",

[1788] "Reminder": "Notification 10 minutes before lesson starts"

[1789] }

[1790] Step 8:

[1791] The device receives the reminder notification and notifies the user. The device displays a reminder to the user to start the lesson using a browser notification or a mobile app notification.

[1792] Example input:

[1793] Reminder notification data

[1794] Example output:

[1795] Pop-up notifications

[1796] Step 9:

[1797] A user (student) starts a session. A voice or video call is started by clicking a link or button. A chat screen or voice call function is enabled within the session.

[1798] Example input:

[1799] Signaling the start of a session

[1800] Example output:

[1801] Open the video call screen

[1802] Step 10:

[1803] The device's built-in emotion engine recognizes the student's facial expressions and tone of voice. The emotion engine collects data using a camera and microphone, analyzes the emotion data in real time using Google Cloud Vision and IBM Watson APIs, and sends it to a server.

[1804] Example input:

[1805] Facial expression data, voice data

[1806] Example output:

[1807] Real-time sentiment analysis data

[1808] Step 11:

[1809] The server stores data during real-time sessions, records conversation data and emotion data in a database, and saves the session details.

[1810] Example input:

[1811] Conversation data, emotion data

[1812] Example output:

[1813] Database Update

[1814] Step 12:

[1815] The server updates the revenue information and distributes the revenue to the creator. After the session ends, the revenue is calculated and paid to the creator via, for example, PayPal or Stripe.

[1816] Example input:

[1817] Session data, revenue sharing algorithms

[1818] Example output:

[1819] Payment Transactions

[1820] Step 13:

[1821] The device updates the creator's dashboard and UI to display new revenue information and instructor performance data in real time.

[1822] Example input:

[1823] Updated earnings and performance data

[1824] Example output:

[1825] Updated Dashboard View

[1826] (Application example 2)

[1827] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1828] Conventional virtual instructor systems were capable of providing users with interactive lessons specialized in specific fields, but lacked the ability to recognize the user's emotions and adjust responses in real time. As a result, there were problems with user satisfaction and learning effectiveness. It was also difficult to provide the interactivity that users experience in real-life interpersonal communication. Furthermore, the same problem existed in shopping, where users were unable to receive product guidance that responded to their emotions in real time. There is a need to solve these problems.

[1829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1830] In this invention, the server includes means for creating a virtual instructor specialized in a specific field using a generative artificial intelligence, means for saving the virtual instructor created by the generative artificial intelligence in a database, means for displaying a list of the virtual instructors to allow a user to select one, means for reserving a session with the virtual instructor at a date and time desired by the user, means for conducting a real-time session with the virtual instructor based on the reservation, means for saving data of the real-time session, means for distributing revenue to the creator of the virtual instructor, means for analyzing the user's emotions using emotion recognition technology and adjusting the virtual instructor's responses, and means for saving the emotion recognition data in a database. This enables optimal responses according to the user's emotions and makes it possible to provide real-time interactive lessons and shopping assistant experiences.

[1831] "Generative AI" is an AI technology that generates domain-specific responses and information based on input prompts.

[1832] A "virtual instructor" is a virtual instructor generated by generative artificial intelligence, with knowledge and skills in a specific field, who provides interactive lessons to users.

[1833] The "database" is an information system for storing and managing information about the generated virtual instructor, session data, emotion recognition data, etc.

[1834] "Emotion recognition technology" is a technology that analyzes emotions from a user's facial expressions, tone of voice, etc., and adjusts the system's response based on the results.

[1835] A "real-time session" is an online session in which a virtual instructor and a user interact in real time to provide interactive feedback.

[1836] "Revenue sharing" is a system in which the creator of a virtual instructor is given a share of the profits and revenues earned from students.

[1837] A "server" is a computer system that processes user requests, generates virtual instructors, stores data, manages real-time sessions, and so on.

[1838] Overall system overview

[1839] This invention provides a system for creating and using virtual instructors specialized in specific fields. This system includes functions for user registration, virtual instructor creation, selection and reservation, real-time sessions, emotion recognition, data storage, and revenue sharing. This system is primarily implemented by a server, terminals (smartphones, PCs, etc.), and software using emotion recognition technology.

[1840] Hardware and software used

[1841] Hardware: Camera, microphone, smartphone, PC, server

[1842] Software: Generative artificial intelligence models (e.g., OpenAI GPT-4), emotion recognition engines, database management systems (e.g., MySQL), real-time communication tools (e.g., WebRTC)

[1843] A natural language explanation of the program's processing overview

[1844] User Registration

[1845] The device displays a user registration form, where the user enters their name, email address, specific areas of interest, etc. This information is sent to the server, which stores the user data in a database.

[1846] Virtual Instructor Creation

[1847] The server uses a generative artificial intelligence model to generate a virtual tutor based on prompts received from the user, such as "a tutor who is knowledgeable about fashion trends for Spring 2023." The generated virtual tutor is then stored in a database by the server.

[1848] Select and book a virtual teacher

[1849] The terminal retrieves a list of virtual instructors from the database and displays it to the user. The user selects the desired virtual instructor and date and time, and sends the reservation information to the server. The server saves the reservation data.

[1850] Conducting real-time sessions

[1851] At the scheduled time, the device will send a reminder to the user and start the real-time session. During the session, emotion recognition technology will analyze the user's facial expressions and tone of voice and transmit the results in real time to the server. The server will then adjust the virtual instructor's responses based on this emotion data.

[1852] Data storage

[1853] Once the session ends, the server stores the conversation data and emotion recognition data in a database.

[1854] Revenue sharing

[1855] After the session ends, the server updates the revenue data and distributes the revenue to the virtual instructor creator, who can check the revenue status through their device.

[1856] Specific examples

[1857] The user selects "A virtual assistant knowledgeable about fashionable Spring 2023 fashion" and books a shopping session. The real-time session begins, and the user turns on their camera to interact with the assistant. When the user asks, "What spring outfits do you recommend?", the emotion recognition engine analyzes the user's facial expression and recognizes that they are excited. The assistant then presents detailed and glamorous products.

[1858] Prompt Sentence Examples

[1859] Create a fashion-savvy virtual assistant that can provide information on fashion trends, especially for Spring 2023, and guide you through the shopping process in a fun and interactive way.

[1860] The above is a specific embodiment for carrying out the invention.

[1861] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1862] Step 1: User Registration

[1863] Input: The user enters their name, email address, and specific areas of interest (e.g., fashion, home appliances) into the device.

[1864] Processing: The terminal checks these input data and sends them to the server.

[1865] Data processing and calculation: The server stores the received data in a database.

[1866] Output: A new user record is created in the database and the user registration is complete.

[1867] Step 2: Create a virtual instructor

[1868] Input: A subject-specific prompt from the user (e.g., "Teacher who is knowledgeable about fashion trends for Spring 2023").

[1869] Processing: The server passes the prompts to a generative artificial intelligence model to generate a virtual instructor specialized in the specific subject area.

[1870] Data processing and calculation: A generative artificial intelligence model analyzes the prompts and generates data for a virtual instructor with domain-specific knowledge.

[1871] Output: The generated virtual instructor data is saved in a database via the server.

[1872] Step 3: Select and book a virtual teacher

[1873] Input: The user browses the list of virtual instructors through the terminal and selects the desired instructor and date and time.

[1874] Processing: The device sends information about the selected instructor and reservation date and time to the server.

[1875] Data processing and calculation: The server verifies the reservation data and stores it in the database.

[1876] Output: The virtual instructor and date / time are booked and a confirmation is sent to the user.

[1877] Step 4: Reminders

[1878] Enter your reservation information shortly before your reservation date and time.

[1879] Processing: The server sends a reminder notification to the device based on the reservation information.

[1880] Data processing and calculation: The server calculates the reservation date and time and sends a notification at the appropriate time.

[1881] Output: A reminder notification is displayed to the user on their device.

[1882] Step 5: Conduct a real-time session

[1883] Input: The user clicks the session start button on the terminal at the scheduled date and time.

[1884] Processing: The device initiates a real-time session and interacts with the server to stream session data.

[1885] Data processing and calculation: The server processes the session data and passes it to the emotion recognition engine in real time.

[1886] Output: A real-time interactive session is initiated and continues.

[1887] Step 6: Emotion recognition and response adjustment

[1888] Input: User's facial expression and voice tone data are sent from the device.

[1889] Processing: The emotion recognition engine analyzes the input data and identifies the user's emotion.

[1890] Data processing and calculation: The server adjusts the virtual instructor's responses based on the emotion recognition results.

[1891] Output: Tailored responses are provided to the user in real time.

[1892] Step 7: Save your data

[1893] Input: Conversation data and emotion recognition data from a real-time session.

[1894] Processing: The server stores these data in a database.

[1895] Data processing and calculations: After the session, the data is organized and stored.

[1896] Output: All session and sentiment data is stored securely in a database.

[1897] Step 8: Revenue sharing

[1898] Input: Revenue data after the session ends.

[1899] Processing: The server calculates the revenue data and distributes it to the creators of the virtual instructors.

[1900] Data processing and calculation: The server calculates the revenue and distributes it appropriately.

[1901] Output: Creator dashboard with revenue information.

[1902] The above are the specific processing steps of the system based on the present invention.

[1903] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1904] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1905] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1906] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1907] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1908] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1909] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1910] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1911] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1912] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1913] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1914] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1915] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1916] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1917] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1918] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1919] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1920] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1921] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1922] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1923] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1924] The following is further disclosed regarding the above embodiment.

[1925] (Claim 1)

[1926] A means for creating domain-specific virtual instructors using generative artificial intelligence;

[1927] A means for storing the virtual instructor created by the generating artificial intelligence in a database;

[1928] means for displaying a list of the virtual instructors to allow a user to select one;

[1929] a means for reserving a session with the virtual instructor at a date and time desired by the user;

[1930] means for conducting a real-time session with the virtual instructor based on the reservation;

[1931] means for storing data of said real-time session;

[1932] The system includes a means for distributing revenue to the creator of said virtual instructor.

[1933] (Claim 2)

[1934] The system of claim 1 , further comprising: means for sending a session reminder to the user.

[1935] (Claim 3)

[1936] The system of claim 1 , further comprising means for providing a dashboard for the creator of the virtual instructor to view the revenue status of the session.

[1937] "Example 1"

[1938] (Claim 1)

[1939] A means for creating domain-specific virtual instructors using generative artificial intelligence;

[1940] A means for storing the virtual instructor created by the generating artificial intelligence in a database;

[1941] means for displaying a list of the virtual instructors to allow a user to select one;

[1942] a means for reserving a session with the virtual instructor at a date and time desired by the user;

[1943] means for conducting a real-time session with the virtual instructor based on the reservation;

[1944] means for enabling a chat screen and a voice call function during the real-time session;

[1945] means for storing data of said real-time session;

[1946] The system includes a means for distributing revenue to the creator of said virtual instructor.

[1947] (Claim 2)

[1948] The system of claim 1 , further comprising: means for sending a session reminder to the user.

[1949] (Claim 3)

[1950] The system of claim 1 , further comprising means for providing a dashboard for the creator of the virtual instructor to view the revenue status of the session.

[1951] "Application Example 1"

[1952] (Claim 1)

[1953] A means of creating specialized virtual assistants using generative artificial intelligence;

[1954] A means for storing the virtual assistant created by the generating artificial intelligence in a database;

[1955] A means for displaying a list of the virtual assistants so that a user can select one;

[1956] A means for booking a session with the virtual assistant at a date and time desired by the user;

[1957] A means for conducting a real-time session with the virtual assistant based on the reservation;

[1958] means for storing data of said real-time session;

[1959] A means for the virtual assistant to answer questions about the user's products in real time;

[1960] A system including a means for distributing revenue to the creator of the virtual assistant.

[1961] (Claim 2)

[1962] The system of claim 1 , further comprising: means for sending a session reminder to the user.

[1963] (Claim 3)

[1964] 10. The system of claim 1, further comprising means for providing a dashboard for the creator of the virtual assistant to view the revenue status of the session.

[1965] "Example 2: Combining Emotion Engines"

[1966] (Claim 1)

[1967] A means for creating domain-specific virtual instructors using generative artificial intelligence;

[1968] A means for storing the virtual instructor created by the generating artificial intelligence in a database;

[1969] means for displaying a list of the virtual instructors to allow a user to select one;

[1970] a means for reserving a session with the virtual instructor at a date and time desired by the user;

[1971] means for conducting a real-time session with the virtual instructor based on the reservation;

[1972] means for recognizing user emotions and adjusting responses during said real-time session;

[1973] means for storing data of said real-time session;

[1974] The system includes a means for distributing revenue to the creator of said virtual instructor.

[1975] (Claim 2)

[1976] The system of claim 1 , further comprising: means for sending a session reminder to the user.

[1977] (Claim 3)

[1978] The system of claim 1 , further comprising means for providing a dashboard for the creator of the virtual instructor to view the revenue status of the session.

[1979] "Application example 2 when combining emotion engines"

[1980] (Claim 1)

[1981] A means for creating domain-specific virtual instructors using generative artificial intelligence;

[1982] A means for storing the virtual instructor created by the generating artificial intelligence in a database;

[1983] means for displaying a list of the virtual instructors to allow a user to select one;

[1984] a means for reserving a session with the virtual instructor at a date and time desired by the user;

[1985] means for conducting a real-time session with the virtual instructor based on the reservation;

[1986] means for storing data of said real-time session;

[1987] means for distributing revenue to creators of said virtual instructors;

[1988] means for analyzing the user's emotions using emotion recognition technology and adjusting the responses of the virtual instructor;

[1989] The system includes a means for storing emotion recognition data in a database.

[1990] (Claim 2)

[1991] The system of claim 1 , further comprising: means for sending a session reminder to the user.

[1992] (Claim 3)

[1993] The system of claim 1 , further comprising means for providing a dashboard for the creator of the virtual instructor to view the revenue status of the session. [Explanation of symbols]

[1994] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for creating domain-specific virtual instructors using generative artificial intelligence; A means for storing the virtual instructor created by the generating artificial intelligence in a database; means for displaying a list of the virtual instructors to allow a user to select one; a means for reserving a session with the virtual instructor at a date and time desired by the user; means for conducting a real-time session with the virtual instructor based on the reservation; means for storing data of said real-time session; The system includes a means for distributing revenue to the creator of said virtual instructor.

2. The system of claim 1 , further comprising: means for sending a session reminder to the user.

3. The system of claim 1 , further comprising means for providing a dashboard for the creator of the virtual instructor to view the revenue status of the session.

Images