system

Abstract

Provide a system. A means for receiving basic information of a user and storing it in a database; means for analyzing a user's learning history and progress to generate a profile based on the individual user; A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile; A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery; a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan; A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, learning should be a continuous process for everyone, from children to adults. However, it is difficult to study on one's own, and many people are unable to find a good instructor or mentor at the right time. This presents a challenge for learners, as it makes it difficult to efficiently progress through learning in line with their own interests and goals, and to receive appropriate guidance to develop essential problem-solving skills. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides a system including: means for receiving a user's basic information and storing it in a database; means for analyzing the user's learning history and progress and generating a profile based on the individual user; means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile; means for providing the generated feedback to the user and prompting the user with additional questions to encourage the user's awareness and discovery; means for continuously recording the user's learning progress, providing an evaluation based on the progress and adjusting the learning plan as needed; and means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation. Furthermore, the system achieves effective learning support by using a natural language processing model to provide optimal information based on the user's characteristics and learning level in generating feedback, and by periodically evaluating the user's progress and adjusting the next steps and learning plan based on the user's learning history.

[0006] "Basic Information" refers to personal information collected during initial registration, such as the user's name, age, areas of interest, and learning goals.

[0007] "Database" refers to a storage device within the system that stores and manages a user's basic information, learning history, and progress data.

[0008] "Study history" is a record of the user's past learning content, progress, grades, etc.

[0009] A "profile" is information that represents an individual user's characteristics and learning tendencies, generated based on the user's learning history and basic information.

[0010] "Personalized feedback" refers to answers and advice that are customized based on the user's characteristics and learning situation.

[0011] "Questioning" is the act of asking questions or raising doubts to make users think and deepen their understanding.

[0012] A "natural language processing model" is an algorithm and technology that enables computers to understand human language and generate appropriate answers and feedback.

[0013] "Progress" refers to indicators or situations that indicate how much of the learning goal set by the user has been achieved.

[0014] "Evaluation" is the process of checking and analyzing the user's learning progress and results.

[0015] A "study plan" is a plan or schedule set up to help a user study efficiently.

[0016] "Encouragement and praise" are messages that recognize the efforts and achievements of users when they achieve their learning goals or make progress, and increase their motivation. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support based on the user's characteristics and learning situation. This system enables users to study efficiently in line with their interests and goals, and develop essential problem-solving skills. Below, we will explain the system's program and its processing in natural language, including concrete examples.

[0039] System Overview

[0040] 1. User registration and profile creation

[0041] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[0042] Terminal: Receives user input and sends it to the server.

[0043] Server: Stores the received basic information in a database and generates an initial profile.

[0044] 2. Start the study session and take questions

[0045] User: Starts a learning session and enters questions and inquiries.

[0046] Terminal: Receives user input and sends it to the server.

[0047] Server: Analyzes the received question or inquiry and passes it to the appropriate dialogue model.

[0048] 3. Generate personalized feedback

[0049] Server: Based on the user's profile and learning history, the generative AI model analyzes the data and generates feedback and suggestions appropriate for the user.

[0050] Server: Generates a message containing a question along with feedback and sends it to the device.

[0051] 4. Feedback and Questioning

[0052] Terminal: Displays feedback and prompts provided by the server to the user.

[0053] User: Enter your understanding and opinion on the question and submit.

[0054] Terminal: Sends the user's answer to the server.

[0055] Server: Analyzes the user's response and provides additional feedback or explanation.

[0056] 5. Record and evaluate progress

[0057] Server: Records user progress in a database.

[0058] Server: Regularly evaluate progress and make next steps or adjustments to the plan.

[0059] Server: Based on the progress assessment, it provides feedback to the user and suggests any necessary details.

[0060] 6. Maintaining motivation to learn

[0061] Server: Generates and sends encouragement and praise to users based on their progress and achievement of learning goals.

[0062] On-device: Show encouragement and praise to users to increase their motivation.

[0063] Specific examples

[0064] For example, suppose User A wants to learn about vectors in mathematics. User A accesses the system and enters, "I'd like to learn about vector addition." The server uses User A's basic information and past learning history to provide a brief explanation of the basic concept of vectors and asks, "How much do you currently understand?" If User A then replies, "I understand the basics of addition, but I'm having trouble applying it," the server will provide specific application examples and additional materials. It will also record User A's progress and send an encouraging message saying, "You've achieved today's learning goal. Is there anything you'd like to advance to next?"

[0065] This system allows users to learn at their own pace and receives constant personalized feedback, resulting in high learning outcomes.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] User: Accessing the system for the first time and opening the user registration page.

[0069] Terminal: Displays the registration page to the user and provides fields for entering the necessary information.

[0070] Step 2:

[0071] User: Enter basic information such as name, age, areas of interest, and learning goals, then press the submit button.

[0072] Terminal: Sends the entered information to the server.

[0073] Step 3:

[0074] Server: Stores the received basic information in a database.

[0075] Server: Generates an initial profile based on basic information.

[0076] Step 4:

[0077] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[0078] Terminal: Sends user input to the server.

[0079] Step 5:

[0080] Server: Analyzes the received questions and inquiries.

[0081] Server: Reads the user's profile and past learning history from the database and understands the user's characteristics.

[0082] Step 6:

[0083] Server: Calls the generative AI model based on the analysis results and generates personalized feedback.

[0084] Server: Generates a message containing a question to get the user to think along with the generated feedback.

[0085] Step 7:

[0086] Terminal: Displays feedback and prompts from the server to the user.

[0087] User: Check the feedback, enter their understanding and opinion of the question, and press the submit button.

[0088] Step 8:

[0089] Terminal: Sends the user's answer to the server.

[0090] Server: Parses the received answer and generates a message to provide additional feedback or clarification.

[0091] Step 9:

[0092] Server: Records user progress in a database and tracks progress continuously.

[0093] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[0094] Step 10:

[0095] Terminal: displays generated messages to the user.

[0096] Server: Generates encouragement and praise based on the user's achievement of learning goals and progress.

[0097] Server: Generates messages containing encouragement and praise and sends them to the device.

[0098] Step 11:

[0099] On-device: Display encouragement and praise to the user to increase their motivation.

[0100] User: Enter a thank you message or next learning goal if desired, and click the send button.

[0101] These are the specific processing steps of this interactive AI mentor system, which allows users to receive individually customized learning support and progress effectively at their own pace.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional learning support systems lacked personalized support tailored to each user's characteristics and learning progress, making efficient learning difficult. Furthermore, there were limited ways to continuously maintain users' motivation to learn, which meant that learning effectiveness was not maximized. Furthermore, the quality and timing of feedback often did not meet users' needs, which reduced learning efficiency.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for receiving basic information about a user and saving it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and asking additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing optimal feedback and suggestions to the user using a generative AI model, and means for generating prompt sentences, inputting the prompt sentences into the generative AI model, and providing the generated output to the user. This enables personalized support based on the user's characteristics and learning situation, maintaining motivation to learn, and providing effective feedback.

[0107] "User" refers to an individual who uses this system to carry out learning activities or ask questions.

[0108] "Basic information" refers to information entered during initial registration, such as the user's name, age, areas of interest, and learning goals.

[0109] "Database" refers to a digital information storage location for storing basic information, learning history, and progress data of users.

[0110] A "profile" refers to a collection of information specific to an individual user, generated based on the user's basic information, learning history, and progress.

[0111] "Feedback" refers to answers, suggestions, comments, etc. provided in response to users' questions or inquiries.

[0112] "Generative AI model" refers to an artificial intelligence model that generates optimal feedback and suggestions for users.

[0113] "Natural language processing model" refers to a software model used to analyze user input and generate an appropriate response.

[0114] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0115] "Progress" refers to the results of evaluating a user's learning situation and level of achievement.

[0116] "Study plan" refers to a plan or program designed to help users study efficiently.

[0117] "Encouragement and praise" refers to messages and comments provided to motivate users to learn.

[0118] "Continuously recording" means that the user's learning progress is periodically saved to a database.

[0119] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support according to the user's characteristics and learning situation. The program processing of this system is explained in detail below.

[0120] System configuration

[0121] This system mainly consists of three elements: the server, the terminal, and the user.

[0122] server

[0123] The server plays a central role in storing and analyzing data and generating feedback using generative AI models. The hardware used is a cloud-based server, and the corresponding software includes the following:

[0124] Database: MySQL (registered trademark)

[0125] Natural language processing library: spaCy

[0126] Generative AI model: GPT-4 (registered trademark) from OpenAI (registered trademark)

[0127] Programming language: Python

[0128] The server does the following:

[0129] 1. Store the basic information received from the user in a MySQL database.

[0130] 1. Use spaCy to parse user input and extract keywords.

[0131] 1. Use GPT-4 to generate feedback and suggestions.

[0132] 1. Evaluate your learning progress and adjust your next steps and learning plan.

[0133] Terminal

[0134] The terminal is responsible for providing an interface with the user, sending user input to the server, and displaying feedback from the server. The terminal's functions include:

[0135] 1. Receive user input in real time and send it to the server.

[0136] 1. Display feedback and questions received from the server to the user.

[0137] user

[0138] Users use the system to carry out learning activities and receive feedback to progress their learning. Users perform the following actions:

[0139] 1. Enter your basic information and learning goals.

[0140] 1. Enter any questions or concerns you may have while studying.

[0141] 1. Type your response to the feedback or questions provided.

[0142] Specific actions

[0143] The operation of this system will be explained using a specific example.

[0144] User Registration

[0145] When a user first accesses the system, they enter basic information such as their name, age, areas of interest, and learning goals. The device receives the information and sends it to the server in real time. The server stores the information in a MySQL database and creates an initial profile.

[0146] Learning sessions and questions

[0147] If a user wants to learn about "vector addition," they type that into their device. The device receives the question and sends it to the server, which uses spaCy to parse it and GPT-4 to generate appropriate feedback.

[0148] Providing feedback and asking questions

[0149] The server generates feedback and sends it to the device, which then displays it to the user. For example, if you type "Teach me about vector addition," the server will explain the basic concept of vectors and ask, "How well do you understand it now?"

[0150] Giving and receiving feedback

[0151] If the user responds, "I understand the basics of addition, but I'm having trouble applying it," the server will again display specific application examples and additional materials. By repeating this process, the server can provide optimal learning support to the user.

[0152] Prompt Sentence Examples

[0153] For example, the following prompt sentence is input to the generative AI model:

[0154] "If you're a 15-year-old student who's good at math, explain the basic concept of vector addition and ask if they'd like to see some applications. Also, ask follow-up questions to check their understanding."

[0155] By providing the user with feedback messages generated based on these prompts, more effective learning support is possible.

[0156] summary

[0157] This interactive AI mentoring system allows users to receive personalized support based on their individual characteristics and learning progress. The server provides feedback using a generative AI model, enabling efficient learning and maintaining motivation.

[0158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0159] Step 1:

[0160] User registration and profile creation

[0161] 1.1. Collecting User Input

[0162] Users enter basic information such as name, age, interests, and learning goals.

[0163] Input: A user enters information into a web form (e.g., "Taro Tanaka, 17 years old, mathematics, university exams").

[0164] Output: The basic information entered is sent to the terminal.

[0165] What happens: A user fills out a form and clicks the submit button.

[0166] 1.2. Transmission of User Information

[0167] The device receives the user's input and transmits it to the server in real time.

[0168] Input: Basic information submitted via the web form.

[0169] Output: Basic information is sent to the server.

[0170] Specific operation: The front-end script takes the input information and sends it to the server via the API.

[0171] 1.3. Server-based information storage and profile generation

[0172] The server stores the received basic information in a database and generates an initial profile.

[0173] Input: User basic information.

[0174] Output: The initial profile that will be saved in the database.

[0175] What it does: Executes an insert query in the MySQL database to save the information. Generates a profile ID.

[0176] Step 2:

[0177] Start a study session and take questions

[0178] 2.1. User Question Input

[0179] A user begins a learning session by typing in a question: "Teach me about vector addition."

[0180] Input: The user types a question into the terminal.

[0181] Output: The entered question is received by the terminal.

[0182] What it does: A user starts a learning session and uses a form field to enter a question.

[0183] 2.2. Submitting a Question

[0184] The device receives the user's question and sends it to the server.

[0185] Input: The question entered by the user.

[0186] Output: The question is sent to the server.

[0187] What happens: JavaScript or other front-end scripts send questions to the server via the API.

[0188] 2.3. Query analysis by the server

[0189] The server parses the question and passes it to the appropriate dialogue model.

[0190] Input: Question sent from terminal.

[0191] Output: Feedback generated based on analysis results.

[0192] What it does: Uses spaCy to extract keywords from the question and prepares them for passing to the AI ​​model.

[0193] Step 3:

[0194] Generate personalized feedback

[0195] 3.1. Obtaining user information and learning history

[0196] The server retrieves the user's profile information and learning history from the database.

[0197] Input: User profile ID.

[0198] Output: Profile information and learning history.

[0199] Specific behavior: Executes a SELECT query based on the user ID in the MySQL database to retrieve information.

[0200] 3.2. Analysis and feedback generation using AI models

[0201] The server uses a generative AI model such as GPT-4 to generate appropriate feedback and suggestions.

[0202] Input: User information, learning history, question keywords.

[0203] Output: The generated feedback.

[0204] Specific operation: Generate a prompt sentence, input it into GPT-4, and generate feedback.

[0205] 3.3. Sending Feedback

[0206] The server transmits the generated feedback to the terminal.

[0207] Input: Generated feedback.

[0208] Output: Feedback is sent to the device.

[0209] Specific behavior: Sends feedback to the device using an API call.

[0210] Step 4:

[0211] Feedback and questions

[0212] 4.1. Display of Feedback

[0213] The device displays the feedback it receives from the server to the user.

[0214] Input: Feedback received from the server.

[0215] Output: Feedback that is displayed on the user's screen.

[0216] Specific behavior: Display a feedback message on the screen.

[0217] 4.2. User response input

[0218] Users enter responses to the feedback and questions provided.

[0219] Input: The user's response.

[0220] Output: The reply is received at the terminal.

[0221] Specific behavior: The user types a response and presses the send button.

[0222] 4.3. Sending a Reply

[0223] The device receives the user's response and sends it to the server.

[0224] Input: The user's response.

[0225] Output: The reply is sent to the server.

[0226] What happens: JavaScript or other front-end scripts send a response to the server via an API.

[0227] 4.4. Server Response Analysis and Additional Feedback Generation

[0228] The server analyzes the user's response and provides additional feedback or explanation.

[0229] Input: The user's response.

[0230] Output: Additional feedback.

[0231] What it does: We again use spaCy to parse the response and GPT-4 to generate additional feedback.

[0232] Step 5:

[0233] Recording and evaluating progress

[0234] 5.1. Recording progress

[0235] The server records the user's learning progress in a database.

[0236] Input: User's learning progress information.

[0237] Output: Progress information is saved to the database.

[0238] What it does: Executes an insert query into a MySQL database to store progress information.

[0239] 5.2. Periodic evaluation

[0240] The server periodically evaluates the user's progress and determines the next learning step.

[0241] Input: User's learning progress information.

[0242] Output: Progress assessment results.

[0243] Specific operation: Evaluation is performed using scikit-learn based on learning progress information.

[0244] 5.3. Adjusting the plan and proposing next steps

[0245] Based on the evaluation results, the server will suggest the user the best next learning steps and plan.

[0246] Input: Progress assessment results.

[0247] Output: Suggested next learning step.

[0248] Specific operation: Determine the next step based on the evaluation results, generate a prompt for GPT-4, and create a proposed sentence.

[0249] Step 6:

[0250] Maintaining motivation to learn

[0251] 6.1. Generating encouragement and praise

[0252] The server generates encouragement and praise based on the user's achievement of learning goals and progress.

[0253] Input: User's learning progress information.

[0254] Output: Encouragement and praise.

[0255] Specific behavior: Using GPT-4 to generate encouragement and praise.

[0256] 6.2. Showing encouragement and praise

[0257] The device displays encouragement and praise to the user, increasing motivation to learn.

[0258] Input: Encouragement or praise received from the server.

[0259] Output: Encouragement or praise that appears on the user's screen.

[0260] Specific behavior: Displaying encouragement and praise on the screen.

[0261] (Application example 1)

[0262] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0263] Conventional learning support systems lack the ability to provide personalized feedback based on user characteristics and progress, making it difficult to achieve efficient learning. Furthermore, they lack interactive learning support based on specific digital content, making it difficult for users to deepen their understanding of the content they are viewing. To address these issues, a more advanced interactive system based on generative AI models is needed.

[0264] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0265] In this invention, the server includes means for receiving basic information of a user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on each individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and prompting the user with additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing interactive learning support based on specific digital content, means for the user to input questions and inquiries about the content being viewed, and means for generating personalized feedback about the content being viewed using a generative AI model. This allows the user to deepen their understanding of the content being viewed and to study more efficiently.

[0266] "Basic User Information" refers to personal data such as name, age, interests, and goals.

[0267] A "database" is a system for systematically storing and managing information.

[0268] "Study history" is a record of the content and progress of the user's studies to date.

[0269] A "profile" is individual information generated based on a user's characteristics and learning history.

[0270] "Personalized feedback" refers to individual answers and suggestions provided based on the user's characteristics and learning situation.

[0271] "Questions" are questions or confirmation messages sent to users.

[0272] "Realizations and discoveries" are new things that users understand or discover through learning.

[0273] "Progress" refers to the progress of learning or work.

[0274] "Evaluation" refers to judgments and assessments made based on progress and learning content.

[0275] "Adjusting your study plan" refers to modifying and optimizing your study plan based on your progress and feedback.

[0276] "Encouragement and praise" refers to positive messages provided to improve user motivation.

[0277] "Specific digital content" refers to specific video, audio, and text data intended for educational or informational purposes.

[0278] "Interactive learning support" refers to learning support that is provided through two-way interaction.

[0279] A "generative AI model" is a model of artificial intelligence used to conduct dialogue and analyze data.

[0280] "Currently viewed content" refers to digital content that a user is currently viewing.

[0281] This invention applies an interactive AI mentor system based on a generative AI model to a content distribution service. This system allows users to receive interactive learning support while viewing specific digital content. The following describes the program for this system and how to execute it.

[0282] Hardware and software used

[0283] Hardware: Smartphones, tablets

[0284] Software: Generative AI models (e.g., OpenAI's GPT-4), database management systems (e.g., MySQL), web servers (e.g., Apache®)

[0285] Program Overview

[0286] The server executes a program that includes the following means:

[0287] 1. A means of receiving basic user information and storing it in a database

[0288] Users use their smartphone or tablet to enter basic information such as their name, age, areas of interest, and goals.

[0289] The terminal transmits the input information to the server.

[0290] The server stores this basic information in a database.

[0291] 2. A means to analyze a user's learning history and progress and generate a profile based on the individual user.

[0292] The server generates an individual profile based on the user's basic information and past learning history stored in a database.

[0293] 3. A means of generating personalized feedback in response to user questions or inquiries based on the generated profile.

[0294] The user inputs a question about the digital content being viewed into the terminal and transmits it to the server.

[0295] The server analyzes the profile and questions and generates personalized feedback using a generative AI model (e.g., GPT-4).

[0296] 4. A means to provide generated feedback to users and prompt them with follow-up questions to encourage their awareness and discovery.

[0297] The feedback generated by the server is sent to the terminal and displayed to the user.

[0298] The server then asks additional questions to help the user gain a deeper understanding.

[0299] 5. A means to continually track users' learning progress and provide progress-based assessments and necessary adjustments to their learning plans.

[0300] The server records the user's responses and progress in a database and evaluates them periodically.

[0301] Based on progress, we will adjust your next learning steps and plan.

[0302] 6. A means to generate encouragement and praise according to the user's achievement of learning goals and progress, and to help maintain motivation

[0303] The server generates words of encouragement and praise based on the user's progress in achieving the learning goal and sends them to the terminal.

[0304] 7. Means of providing interactive learning support based on specific digital content

[0305] The server provides learning support corresponding to the digital content that the user is currently viewing.

[0306] 8. A means for users to input questions or inquiries about the content they are viewing

[0307] The terminal provides an interface for users to input questions or inquiries about the content they are viewing.

[0308] Example prompt sentences

[0309] Below are some examples of prompts that can be passed to a generative AI model based on a user question:

[0310] Please answer the following question about the science documentary you are watching: "I would like to know more about how this experiment was done."

[0311] Based on the user's profile, they are interested in science and would like detailed instructions for experiments. Please explain them in an easy-to-understand manner for beginners.

[0312] The feedback generated based on this prompt sentence is provided to the user, realizing interactive learning support.

[0313] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0314] Step 1:

[0315] Users use their smartphones or tablets to enter basic information such as their name, age, areas of interest, and goals. This input form is provided by the device. The entered basic information is temporarily stored on the device.

[0316] Step 2:

[0317] The device sends the temporarily stored basic information to the server, which then stores the received basic information in a database. The input data is user information (name, age, areas of interest, and goals), and the output is the status of saving to the database.

[0318] Step 3:

[0319] The server retrieves the user's basic information and past learning history from the database and generates an individual profile based on that information. This profile includes the user's progress and areas of interest. The generated profile is then saved back into the database. The input is the user information and learning history data, and the output is the generated profile.

[0320] Step 4:

[0321] The user inputs a question about the digital content being viewed via the terminal. The question is sent from the terminal to the server. The input is the user's question, and the output is the status of transmission to the server.

[0322] Step 5:

[0323] The server analyzes the user's profile and question, sends prompts to a generative AI model (e.g., GPT-4) and generates personalized feedback. In this process, an appropriate response is generated based on the profile and question. The input is the prompt, and the output is the generated feedback.

[0324] Step 6:

[0325] The generated feedback is sent from the server to the terminal and displayed to the user. The user receives the feedback and deepens their understanding. The input is the generated feedback, and the output is the user's perception.

[0326] Step 7:

[0327] The server also generates questions for the user to further understand and sends them to the terminal. The questions are displayed to the user, and the user inputs answers accordingly. The input is the user's answer, and the output is the transmission status to the server.

[0328] Step 8:

[0329] The server records the user's answers and learning progress in a database and periodically evaluates the progress. The progress data is used to determine the next learning step and adjust the plan. The input is the user's answers and progress data, and the output is suggested adjustments to the learning plan.

[0330] Step 9:

[0331] Based on the user's achievement of learning goals and progress, the server generates encouragement and praise and sends them to the device. The user receives these and stays motivated. The input is progress data, and the output is encouragement and praise.

[0332] Step 10:

[0333] The server provides interactive learning support based on specific digital content. Users use their devices to input questions or inquiries about the content they are viewing. The server responds to these questions and generates appropriate feedback. The input is the user's question, and the output is the generated feedback.

[0334] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0335] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize the user's learning and provide emotional support. Below, we will explain the program and processing of this system in detail, using concrete examples.

[0336] System Overview

[0337] 1. User registration and profile creation

[0338] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[0339] Terminal: Receives user input and sends it to the server.

[0340] Server: Stores the received basic information in a database and generates an initial profile.

[0341] 2. Start the study session and take questions

[0342] User: Starts a learning session and enters the area they want to learn about and the question they have.

[0343] Terminal: Sends user input to the server.

[0344] Server: Analyzes the received questions and inquiries and simultaneously operates the emotion engine.

[0345] 3. Emotion Recognition and Feedback Generation

[0346] Server: The emotion engine analyzes the user's input and dialogue history to recognize the user's emotions. For example, the emotion engine detects emotions such as "excitement," "disappointment," and "anxiety."

[0347] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[0348] Server: Generates feedback and prompts or encouraging messages according to the user's emotions.

[0349] 4. Feedback and Questioning

[0350] Terminal: Displays feedback and prompts provided by the server to the user.

[0351] User: Review the feedback and enter answers based on the questions and emotions.

[0352] Terminal: Sends the user's answer to the server.

[0353] Server: Re-analyzes the user's response and provides appropriate additional feedback or explanation.

[0354] 5. Record and evaluate progress

[0355] Server: Records user progress in a database and keeps track of progress.

[0356] Server: Periodically evaluates progress, adjusts next learning steps and plans, and generates messages with suggestions.

[0357] Server: Provides special suggestions and support based on the user's emotions.

[0358] 6. Maintaining motivation for learning and emotional care

[0359] Server: Generates encouragement and praise based on emotion recognition according to the user's achievement of learning goals and progress.

[0360] Server: For example, if the user is feeling anxious, provide specific encouragement such as, "You're doing great! You're almost there."

[0361] On-device: Show encouragement and praise to users to increase their motivation.

[0362] User: Optionally enter a thank you message or next learning goal and submit.

[0363] Specific examples

[0364] For example, let's say User B wants to learn about the equations of motion in physics. User B inputs, "Please teach me about the equations of motion." The server performs an analysis based on User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concepts of the equations of motion. Which part do you find particularly difficult?"

[0365] If User B responds, "I don't understand the concept of acceleration," the server uses the emotion engine's recognition to provide a specific explanation, including an encouraging message such as, "Everyone has things they don't understand. I'll explain them in detail from the beginning." The server also records the progress and provides further emotional support, asking, "Your understanding of acceleration is great progress. What part would you like to learn next?"

[0366] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[0367] The processing flow will be explained below.

[0368] Step 1:

[0369] User: Accessing the system for the first time and opening the user registration page.

[0370] Device: Presents the user with a registration page, providing fields for input such as name, age, areas of interest, and learning goals.

[0371] Step 2:

[0372] User: Enter basic information and press the send button.

[0373] Terminal: Validates the entered information and sends it to the server.

[0374] Step 3:

[0375] Server: Stores the received basic information in a database and generates an initial profile.

[0376] Step 4:

[0377] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[0378] Terminal: Sends user input to the server.

[0379] Step 5:

[0380] Server: Analyzes the received questions and inquiries. At the same time, it starts the emotion engine and analyzes the user's input.

[0381] Step 6:

[0382] Server: Uses an emotion engine to recognize the user's emotions (e.g., "confused," "excited").

[0383] Step 7:

[0384] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[0385] Step 8:

[0386] Server: Generates messages including the generated feedback, questions, and even encouraging messages according to the user's emotions, and sends them to the device.

[0387] Step 9:

[0388] Terminal: Displays server-provided feedback, prompts, and encouraging messages to the user.

[0389] Step 10:

[0390] User: Check the feedback, enter an answer based on the question or emotion, and press the send button.

[0391] Step 11:

[0392] Terminal: Sends the user's answer to the server.

[0393] Step 12:

[0394] Server: Again analyzes the user's response and generates appropriate additional feedback and explanations, including special care and advice if needed, based on the emotions recognized by the emotion engine.

[0395] Step 13:

[0396] Server: Records user progress in a database and tracks progress continuously.

[0397] Step 14:

[0398] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[0399] Step 15:

[0400] Terminal: displays generated messages to the user.

[0401] Step 16:

[0402] Server: Based on the user's progress and achievement of learning goals, the server performs emotion recognition again and generates words of encouragement or praise. For example, "Your efforts are great. What would you like to learn next?"

[0403] Step 17:

[0404] On-device: Display encouragement and praise to the user to increase their motivation.

[0405] Step 18:

[0406] User: Enter a thank you message or next learning goal and submit.

[0407] By combining this conversational AI mentor system with an emotion engine, users can receive individually customized learning support and emotional care. As a concrete example, if a user is confused about the equations of motion in physics, the system will ask, "I'll start by explaining the basic concepts of the equations of motion. Which parts do you find particularly difficult?" After analyzing the user's confusion, the system will provide an encouraging message saying, "Everyone has parts they don't understand. I'll explain them in detail from the beginning." This will achieve high learning effectiveness and satisfaction.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] Conventional learning support systems often provide uniform feedback without considering the user's emotions, making it difficult to maintain the user's motivation and willingness to learn. Another issue is that feedback is not sufficiently personalized for each user, resulting in reduced learning efficiency. To address these issues, the present invention aims to provide an interactive AI mentor system combined with an emotion recognition engine, thereby realizing personalized learning and emotional care.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server includes: means for receiving basic information about a user and saving it in a database; means for analyzing the user's learning history and progress and generating a profile based on the individual user; means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile; means for providing the generated feedback to the user and prompting the user with additional questions to encourage awareness and discovery; means including an emotion engine that recognizes the user's emotions and generating encouraging and support messages according to the user's emotions; means for continuously recording the user's learning progress and providing evaluations based on the progress and adjustments to the learning plan as needed; and means for generating encouraging and praise words according to the user's achievement of learning goals and progress to help maintain motivation. This makes it possible to individually personalize the user's learning and provide emotional support, thereby improving learning effectiveness and satisfaction.

[0413] "User" refers to an individual who uses this system to study.

[0414] "Basic Information" refers to information provided at the time of initial registration, such as the user's name, age, areas of interest, and learning goals.

[0415] "Database" refers to a system for recording and managing users' basic information, learning history, and progress information.

[0416] "Profile" refers to a set of learning information for each user that is generated based on the user's basic information and learning history.

[0417] "Feedback" refers to the response generated by the system in response to a user's question or inquiry.

[0418] A "natural language processing model" refers to an algorithm that uses a generative AI model to generate appropriate answers to users' questions and inquiries.

[0419] An "emotion engine" refers to a system that analyzes the user's input and dialogue history to recognize the user's emotions.

[0420] "Support messages" refer to messages containing encouragement or specific guidance that are generated by the system based on the user's emotions.

[0421] "Study progress" refers to information indicating how much content a user has studied and the level of understanding of the content.

[0422] "Study plan" refers to the combination of study schedules and tasks suggested by the system to help users study effectively.

[0423] "Maintaining motivation" refers to activities that support users so that they can maintain their motivation to learn during the learning process.

[0424] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize learning and provide emotional support. This system consists of a terminal used by the user to study, a server, and a network for communication between them.

[0425] 1. User registration and profile creation

[0426] First, the user enters basic information from their device, such as their name, age, areas of interest, and learning goals. This information is then sent from the device to the server, which then stores the information in a database (e.g., MySQL) and creates an initial profile.

[0427] 2. Start the study session and take questions

[0428] A user starts a learning session and inputs the subject they want to learn and a question. For example, they input a specific question such as, "Please teach me about the equation of motion." This input is sent to the server via their device. The server analyzes the received question or inquiry using a natural language processing engine (e.g., OpenAI's GPT-4) and simultaneously runs an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer). This allows the emotion to be recognized from the user's input.

[0429] 3. Emotion Recognition and Feedback Generation

[0430] The server uses an emotion engine to analyze the user's input and dialogue history to recognize the user's emotions. For example, it detects "confusion" or "excitement." It then generates personalized feedback using a generative AI model (such as OpenAI's GPT-4) based on the user's profile, past learning history, and the recognized emotions. Specifically, it generates feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?"

[0431] 4. Feedback and Questioning

[0432] The generated feedback is displayed to the user via the device. The user checks the feedback and inputs a response in response to the question. For example, the user might reply, "I don't understand the concept of acceleration." This response is then sent back to the server from the device, and the server generates additional feedback and encouraging messages to support the user. For example, the server might say, "Everyone has parts they don't understand. I'll explain them in detail from the beginning."

[0433] 5. Record and evaluate progress

[0434] The server records the user's progress in a database and continuously tracks it. For example, it might record information such as "What the user learned yesterday: understanding the equation of motion" and "What the user learned today: the concept of acceleration." It also periodically evaluates the progress and adjusts the next learning step or plan. For example, it generates suggestions such as "Let's learn the concept of force next."

[0435] 6. Maintaining motivation for learning and emotional care

[0436] The system generates words of encouragement and praise based on emotion recognition according to the user's progress and achievement of learning goals. For example, if the user is feeling anxious, it will provide specific encouragement such as, "Your efforts are great. You're almost there." These messages are displayed to the user via their device, improving their motivation and helping them maintain their enthusiasm for learning.

[0437] Specific examples

[0438] For example, suppose User B wants to learn about the equation of motion in physics. User B enters, "Teach me about the equation of motion." The server analyzes User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concept of the equation of motion. Which part do you find particularly difficult?" If User B replies, "I don't understand the concept of acceleration," the server provides a specific explanation with an encouraging message, "Everyone has parts they don't understand. I'll explain in detail from the beginning." Progress is also recorded, and further questions and emotional support are provided, such as, "Your understanding of acceleration has improved, which is great. What part would you like to learn next?"

[0439] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[0440] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0441] Step 1:

[0442] Enter and submit basic information

[0443] When a user uses the system for the first time, they enter basic information such as their name, age, areas of interest, and goals. This input is entered into the terminal and then sent to the server. Specifically, when the user enters information into the form and presses the "Submit" button, the data is sent from the terminal to the server. Examples of input are "Name: Tanaka Taro," "Age: 20 years old," "Area of ​​interest: Chemistry," and "Goal: I want to gain a deep understanding from the basics." The output is the basic information data that reaches the server.

[0444] Step 2:

[0445] Generate a profile

[0446] The server stores the received basic information in a database (e.g., MySQL). If the storage is successful, the server generates an initial profile. Specifically, it generates a unique profile ID for each user based on the content stored in the database, associates it with related information, and stores it. The input is the basic information sent from the device, and the output is the user profile data recorded in the database.

[0447] Step 3:

[0448] Start a study session and enter your questions

[0449] A user starts a learning session and inputs the area they want to learn about and a specific question. For example, they might input, "Tell me about the equation of motion." This input is sent from the device to the server. An example input might be, "Could you tell me which part you don't understand?" The output is the specific question data sent to the server.

[0450] Step 4:

[0451] Question analysis and emotion recognition

[0452] The received question or consultation content is analyzed using a natural language processing engine (e.g., a generative AI model). An emotion engine runs in parallel to recognize emotions from the user's input. Specifically, when identifying the emotion "confused," the server calculates an emotion evaluation score based on the relevant keywords and context. The input is the user's question, and the output is the analysis result and emotion recognition result.

[0453] Step 5:

[0454] Feedback Generation

[0455] The server generates personalized feedback based on the user's recognized emotions, profile information, and past learning history. It uses a generative AI model to construct appropriate answers and include encouraging messages. For example, feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?" is generated. The input is emotions and analysis results, and the output is the generated feedback.

[0456] Step 6:

[0457] Displaying feedback and user responses

[0458] The generated feedback is displayed to the user through the device. The user checks the feedback and inputs a response to the question. For example, the user might reply, "I don't understand the concept of acceleration." The device receives this response and sends it to the server. The input is the feedback from the server, and the output is the user's response data.

[0459] Step 7:

[0460] Generate additional feedback and encouraging messages

[0461] The server analyzes the user's response again and generates appropriate additional feedback. It also includes data from the emotion engine and adds an encouraging message. For example, it provides feedback such as, "Everyone has some things they don't understand. I'll explain them in detail from the beginning." The input is the user's response data, and the output is the generated additional feedback.

[0462] Step 8:

[0463] Recording and evaluating progress

[0464] The server records the user's progress in a database. For example, it may record information such as "Today's learning: Equation of motion" or "Level of understanding: Medium." Furthermore, it periodically evaluates the user's progress and adjusts the next learning step and plan. The input is the user's learning data, and the output is the progress evaluation results and a new learning plan proposal.

[0465] Step 9:

[0466] Maintaining motivation for learning and emotional care

[0467] The server generates encouraging and praising messages based on emotion recognition according to the user's achievement of learning goals and progress. For example, it generates messages such as "Your efforts are great. You're almost there." This increases the user's motivation. The input is learning progress and evaluation data, and the output is encouraging and praising messages.

[0468] In this way, the system personalizes the user's learning and provides emotional support through a series of steps.

[0469] (Application example 2)

[0470] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0471] While conventional conversational AI systems can provide feedback to users' questions, they have difficulty providing appropriate responses and encouragement based on the user's emotional state. This can lead to users feeling stressed or losing motivation to learn without receiving appropriate support. Furthermore, in customer service at brick-and-mortar stores, there was a lack of support that took into account the customer's emotions, leading to concerns about a decline in customer satisfaction.

[0472] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving basic information about the user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, and means for providing appropriate feedback and support messages in response to the user's emotions based on emotion recognition. This makes it possible to respond according to the user's emotional state, thereby reducing stress, increasing motivation to learn, and improving customer satisfaction.

[0473] "User" refers to a person who uses the system.

[0474] "Basic information" refers to the initial personal information a user provides to the system, such as name, age, areas of interest, and goals.

[0475] "Database" refers to the digital data structure that the system uses to store basic information about users, their learning history, and so on.

[0476] "Profile" refers to an individual learning model generated by analyzing a user's basic information, learning history, and progress data.

[0477] "Feedback" refers to the responses and advice the system provides to users in response to their questions or inquiries.

[0478] "Emotion recognition" refers to the process of analyzing a user's input and dialogue history to identify the user's emotional state at that time.

[0479] A "natural language processing model" refers to machine learning technology that understands users' questions and inquiries and generates appropriate feedback.

[0480] "Support messages" refer to messages generated based on emotion recognition to encourage and support users.

[0481] "Learning history" refers to the record of learning that a user has done through the system.

[0482] "Personalization" refers to responses that are customized to suit the characteristics and needs of each individual user.

[0483] "Progress assessment" refers to the process of periodically assessing a user's learning progress.

[0484] "Study Plan" means the specific steps or plan established to achieve a User's learning goals.

[0485] "Maintaining motivation" refers to supporting users to maintain their desire to learn.

[0486] "Customer" refers to a person who visits a physical store in search of a product or service.

[0487] This invention can be implemented as an interactive AI shopping assistant system that can be used in brick-and-mortar stores. A specific example of this system will now be described.

[0488] System configuration

[0489] The system consists of a smartphone, a server, an emotion engine, and a generative AI model. The smartphone is a device that the user can carry around and provides an interactive user interface. The server is the main processing unit that processes input from the user and generates the necessary feedback. The emotion engine is software that analyzes the user's emotional state from text input, and the generative AI model is a system that generates appropriate responses through natural language processing.

[0490] System Operation

[0491] 1. User registration and profile creation: The device (smartphone) receives the user's basic information (name, areas of interest, goals, etc.) and sends it to the server. The server stores this information in a database and creates a user profile.

[0492] 2. Question acceptance: A user uses a smartphone to input a question in a physical store, for example, "Where is this product?"

[0493] 3. Emotion recognition and feedback generation: The server receives the user's text input and analyzes their emotional state using an emotion engine. It then uses a generative AI model to generate appropriate feedback. For example, if the emotion engine detects that the user is confused, it can generate a response like, "I'm sorry, what are you looking for? I'll help you." It can also provide specific location information based on the user's input, such as, "Your item is at Aisle 5."

[0494] 4. Feedback presentation: The smartphone displays the feedback to the user, who can then head to the recommended location. If the user enters additional questions, the server again generates a response using the emotion engine and generative AI model.

[0495] Software and hardware used

[0496] Hardware: Smartphones, servers

[0497] Software: OpenAI generative AI models (e.g., GPT-3 (registered trademark)), EmotionEngine

[0498] The process involves the smartphone receiving the user's input and sending it to the server, which then analyzes the input and uses an emotion engine and generative AI model to generate appropriate feedback, which is then sent to the user's smartphone and displayed.

[0499] Specific examples

[0500] For example, if a user types "I don't know where this product is" into their smartphone, the emotion engine will detect "confusion," and the server will provide an encouraging message saying, "I'm sorry, what are you looking for? I'll help you," along with a specific recommended location, such as "That product is in Aisle 5."

[0501] Prompt Sentence Examples

[0502] An example of a prompt that a user might enter into the system is:

[0503] text

[0504] User Question: I don't know where this item is.

[0505] Provide appropriate assistance.

[0506] In this way, it is possible to increase user satisfaction by generating optimal feedback and support messages based on emotion recognition.

[0507] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0508] Step 1: The user uses their smartphone to enter basic information (such as their name, areas of interest, and goals). The device receives this input and sends it to the server.

[0509] Step 2: The server stores the received basic information in a database. It also generates a user profile and integrates it with other information stored in the database. The input is the user's basic information, and the output is the generated profile.

[0510] Step 3: The user uses their smartphone to input a specific question (e.g., "Where is this product?"). The device receives the user's input and sends it to the server.

[0511] Step 4: The server analyzes the received question using an emotion engine to detect the user's emotional state. The input is the user's text input, and the output is the detected emotional state. Specifically, the emotion engine analyzes the emotional nuances of the text and identifies emotions such as "confused" or "excited."

[0512] Step 5: The server uses the generative AI model to generate appropriate feedback for the user's question. The input is the user's question and the detected emotional state, and the output is personalized feedback. For example, if the user types, "I don't know where the product is," the generative AI model will generate the response, "The product is on Aisle 5."

[0513] Step 6: Based on the emotional state, the server further generates an appropriate support message for the user. The input is the detected emotional state, and the output is a support message. For example, if "confusion" is detected, the server generates a message saying "I'm sorry, what are you looking for? I'm here to help you."

[0514] Step 7: The server sends the generated feedback and support message to the terminal, which displays it to the user. The user confirms the feedback and enters further questions if necessary.

[0515] Step 8: The server continuously records the user's progress and updates their profile based on that data, making future interactions more personalized. The input is the user's feedback and emotion history, and the output is an updated profile.

[0516] These steps provide personalized feedback and support messages tailored to the user's emotional state.

[0517] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0518] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0519] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0520] [Second embodiment]

[0521] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0522] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0523] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0524] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0525] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0526] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0527] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0528] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0529] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0530] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0531] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0532] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0533] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support based on the user's characteristics and learning situation. This system enables users to study efficiently in line with their interests and goals, and develop essential problem-solving skills. Below, we will explain the system's program and its processing in natural language, including concrete examples.

[0534] System Overview

[0535] 1. User registration and profile creation

[0536] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[0537] Terminal: Receives user input and sends it to the server.

[0538] Server: Stores the received basic information in a database and generates an initial profile.

[0539] 2. Start the study session and take questions

[0540] User: Starts a learning session and enters questions and inquiries.

[0541] Terminal: Receives user input and sends it to the server.

[0542] Server: Analyzes the received question or inquiry and passes it to the appropriate dialogue model.

[0543] 3. Generate personalized feedback

[0544] Server: Based on the user's profile and learning history, the generative AI model analyzes the data and generates feedback and suggestions appropriate for the user.

[0545] Server: Generates a message containing a question along with feedback and sends it to the device.

[0546] 4. Feedback and Questioning

[0547] Terminal: Displays feedback and prompts provided by the server to the user.

[0548] User: Enter your understanding and opinion on the question and submit.

[0549] Terminal: Sends the user's answer to the server.

[0550] Server: Analyzes the user's response and provides additional feedback or explanation.

[0551] 5. Record and evaluate progress

[0552] Server: Records user progress in a database.

[0553] Server: Regularly evaluate progress and make next steps or adjustments to the plan.

[0554] Server: Based on the progress assessment, it provides feedback to the user and suggests any necessary details.

[0555] 6. Maintaining motivation to learn

[0556] Server: Generates and sends encouragement and praise to users based on their progress and achievement of learning goals.

[0557] On-device: Show encouragement and praise to users to increase their motivation.

[0558] Specific examples

[0559] For example, suppose User A wants to learn about vectors in mathematics. User A accesses the system and enters, "I'd like to learn about vector addition." The server uses User A's basic information and past learning history to provide a brief explanation of the basic concept of vectors and asks, "How much do you currently understand?" If User A then replies, "I understand the basics of addition, but I'm having trouble applying it," the server will provide specific application examples and additional materials. It will also record User A's progress and send an encouraging message saying, "You've achieved today's learning goal. Is there anything you'd like to advance to next?"

[0560] This system allows users to learn at their own pace and receives constant personalized feedback, resulting in high learning outcomes.

[0561] The processing flow will be explained below.

[0562] Step 1:

[0563] User: Accessing the system for the first time and opening the user registration page.

[0564] Terminal: Displays the registration page to the user and provides fields for entering the necessary information.

[0565] Step 2:

[0566] User: Enter basic information such as name, age, areas of interest, and learning goals, then press the submit button.

[0567] Terminal: Sends the entered information to the server.

[0568] Step 3:

[0569] Server: Stores the received basic information in a database.

[0570] Server: Generates an initial profile based on basic information.

[0571] Step 4:

[0572] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[0573] Terminal: Sends user input to the server.

[0574] Step 5:

[0575] Server: Analyzes the received questions and inquiries.

[0576] Server: Reads the user's profile and past learning history from the database and understands the user's characteristics.

[0577] Step 6:

[0578] Server: Calls the generative AI model based on the analysis results and generates personalized feedback.

[0579] Server: Generates a message containing a question to get the user to think along with the generated feedback.

[0580] Step 7:

[0581] Terminal: Displays feedback and prompts from the server to the user.

[0582] User: Check the feedback, enter their understanding and opinion of the question, and press the submit button.

[0583] Step 8:

[0584] Terminal: Sends the user's answer to the server.

[0585] Server: Parses the received answer and generates a message to provide additional feedback or clarification.

[0586] Step 9:

[0587] Server: Records user progress in a database and tracks progress continuously.

[0588] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[0589] Step 10:

[0590] Terminal: displays generated messages to the user.

[0591] Server: Generates encouragement and praise based on the user's achievement of learning goals and progress.

[0592] Server: Generates messages containing encouragement and praise and sends them to the device.

[0593] Step 11:

[0594] On-device: Display encouragement and praise to the user to increase their motivation.

[0595] User: Enter a thank you message or next learning goal if desired, and click the send button.

[0596] These are the specific processing steps of this interactive AI mentor system, which allows users to receive individually customized learning support and progress effectively at their own pace.

[0597] Example 1

[0598] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0599] Conventional learning support systems lacked personalized support tailored to each user's characteristics and learning progress, making efficient learning difficult. Furthermore, there were limited ways to continuously maintain users' motivation to learn, which meant that learning effectiveness was not maximized. Furthermore, the quality and timing of feedback often did not meet users' needs, which reduced learning efficiency.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0601] In this invention, the server includes means for receiving basic information about a user and saving it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and asking additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing optimal feedback and suggestions to the user using a generative AI model, and means for generating prompt sentences, inputting the prompt sentences into the generative AI model, and providing the generated output to the user. This enables personalized support based on the user's characteristics and learning situation, maintaining motivation to learn, and providing effective feedback.

[0602] "User" refers to an individual who uses this system to carry out learning activities or ask questions.

[0603] "Basic information" refers to information entered during initial registration, such as the user's name, age, areas of interest, and learning goals.

[0604] "Database" refers to a digital information storage location for storing basic information, learning history, and progress data of users.

[0605] A "profile" refers to a collection of information specific to an individual user, generated based on the user's basic information, learning history, and progress.

[0606] "Feedback" refers to answers, suggestions, comments, etc. provided in response to users' questions or inquiries.

[0607] "Generative AI model" refers to an artificial intelligence model that generates optimal feedback and suggestions for users.

[0608] "Natural language processing model" refers to a software model used to analyze user input and generate an appropriate response.

[0609] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0610] "Progress" refers to the results of evaluating a user's learning situation and level of achievement.

[0611] "Study plan" refers to a plan or program designed to help users study efficiently.

[0612] "Encouragement and praise" refers to messages and comments provided to motivate users to learn.

[0613] "Continuously recording" means that the user's learning progress is periodically saved to a database.

[0614] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support according to the user's characteristics and learning situation. The program processing of this system is explained in detail below.

[0615] System configuration

[0616] This system mainly consists of three elements: the server, the terminal, and the user.

[0617] server

[0618] The server plays a central role in storing and analyzing data and generating feedback using generative AI models. The hardware used is a cloud-based server, and the corresponding software includes the following:

[0619] Database: MySQL

[0620] Natural language processing library: spaCy

[0621] Generative AI model: OpenAI's GPT-4

[0622] Programming language: Python

[0623] The server does the following:

[0624] 1. Store the basic information received from the user in a MySQL database.

[0625] 1. Use spaCy to parse user input and extract keywords.

[0626] 1. Use GPT-4 to generate feedback and suggestions.

[0627] 1. Evaluate your learning progress and adjust your next steps and learning plan.

[0628] Terminal

[0629] The terminal is responsible for providing an interface with the user, sending user input to the server, and displaying feedback from the server. The terminal's functions include:

[0630] 1. Receive user input in real time and send it to the server.

[0631] 1. Display feedback and questions received from the server to the user.

[0632] user

[0633] Users use the system to carry out learning activities and receive feedback to progress their learning. Users perform the following actions:

[0634] 1. Enter your basic information and learning goals.

[0635] 1. Enter any questions or concerns you may have while studying.

[0636] 1. Type your response to the feedback or questions provided.

[0637] Specific actions

[0638] The operation of this system will be explained using a specific example.

[0639] User Registration

[0640] When a user first accesses the system, they enter basic information such as their name, age, areas of interest, and learning goals. The device receives the information and sends it to the server in real time. The server stores the information in a MySQL database and creates an initial profile.

[0641] Learning sessions and questions

[0642] If a user wants to learn about "vector addition," they type that into their device. The device receives the question and sends it to the server, which uses spaCy to parse it and GPT-4 to generate appropriate feedback.

[0643] Providing feedback and asking questions

[0644] The server generates feedback and sends it to the device, which then displays it to the user. For example, if you type "Teach me about vector addition," the server will explain the basic concept of vectors and ask, "How well do you understand it now?"

[0645] Giving and receiving feedback

[0646] If the user responds, "I understand the basics of addition, but I'm having trouble applying it," the server will again display specific application examples and additional materials. By repeating this process, the server can provide optimal learning support to the user.

[0647] Prompt Sentence Examples

[0648] For example, the following prompt sentence is input to the generative AI model:

[0649] "If you're a 15-year-old student who's good at math, explain the basic concept of vector addition and ask if they'd like to see some applications. Also, ask follow-up questions to check their understanding."

[0650] By providing the user with feedback messages generated based on these prompts, more effective learning support is possible.

[0651] summary

[0652] This interactive AI mentoring system allows users to receive personalized support based on their individual characteristics and learning progress. The server provides feedback using a generative AI model, enabling efficient learning and maintaining motivation.

[0653] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0654] Step 1:

[0655] User registration and profile creation

[0656] 1.1. Collecting User Input

[0657] Users enter basic information such as name, age, interests, and learning goals.

[0658] Input: A user enters information into a web form (e.g., "Taro Tanaka, 17 years old, mathematics, university exams").

[0659] Output: The basic information entered is sent to the terminal.

[0660] What happens: A user fills out a form and clicks the submit button.

[0661] 1.2. Transmission of User Information

[0662] The device receives the user's input and transmits it to the server in real time.

[0663] Input: Basic information submitted via the web form.

[0664] Output: Basic information is sent to the server.

[0665] Specific operation: The front-end script takes the input information and sends it to the server via the API.

[0666] 1.3. Server-based information storage and profile generation

[0667] The server stores the received basic information in a database and generates an initial profile.

[0668] Input: User basic information.

[0669] Output: The initial profile that will be saved in the database.

[0670] What it does: Executes an insert query in the MySQL database to save the information. Generates a profile ID.

[0671] Step 2:

[0672] Start a study session and take questions

[0673] 2.1. User Question Input

[0674] A user begins a learning session by typing in a question: "Teach me about vector addition."

[0675] Input: The user types a question into the terminal.

[0676] Output: The entered question is received by the terminal.

[0677] What it does: A user starts a learning session and uses a form field to enter a question.

[0678] 2.2. Submitting a Question

[0679] The device receives the user's question and sends it to the server.

[0680] Input: The question entered by the user.

[0681] Output: The question is sent to the server.

[0682] What happens: JavaScript or other front-end scripts submit questions to the server via an API.

[0683] 2.3. Query analysis by the server

[0684] The server parses the question and passes it to the appropriate dialogue model.

[0685] Input: Question sent from terminal.

[0686] Output: Feedback generated based on analysis results.

[0687] What it does: Uses spaCy to extract keywords from the question and prepares them for passing to the AI ​​model.

[0688] Step 3:

[0689] Generate personalized feedback

[0690] 3.1. Obtaining user information and learning history

[0691] The server retrieves the user's profile information and learning history from the database.

[0692] Input: User profile ID.

[0693] Output: Profile information and learning history.

[0694] Specific behavior: Executes a SELECT query based on the user ID in the MySQL database to retrieve information.

[0695] 3.2. Analysis and feedback generation using AI models

[0696] The server uses a generative AI model such as GPT-4 to generate appropriate feedback and suggestions.

[0697] Input: User information, learning history, question keywords.

[0698] Output: The generated feedback.

[0699] Specific operation: Generate a prompt sentence, input it into GPT-4, and generate feedback.

[0700] 3.3. Sending Feedback

[0701] The server transmits the generated feedback to the terminal.

[0702] Input: Generated feedback.

[0703] Output: Feedback is sent to the device.

[0704] Specific behavior: Sends feedback to the device using an API call.

[0705] Step 4:

[0706] Feedback and questions

[0707] 4.1. Display of Feedback

[0708] The device displays the feedback it receives from the server to the user.

[0709] Input: Feedback received from the server.

[0710] Output: Feedback that is displayed on the user's screen.

[0711] Specific behavior: Display a feedback message on the screen.

[0712] 4.2. User response input

[0713] Users enter responses to the feedback and questions provided.

[0714] Input: The user's response.

[0715] Output: The reply is received at the terminal.

[0716] Specific behavior: The user types a response and presses the send button.

[0717] 4.3. Sending a Reply

[0718] The device receives the user's response and sends it to the server.

[0719] Input: The user's response.

[0720] Output: The reply is sent to the server.

[0721] What happens: JavaScript or other front-end scripts send a response to the server via an API.

[0722] 4.4. Server Response Analysis and Additional Feedback Generation

[0723] The server analyzes the user's response and provides additional feedback or explanation.

[0724] Input: The user's response.

[0725] Output: Additional feedback.

[0726] What it does: We again use spaCy to parse the response and GPT-4 to generate additional feedback.

[0727] Step 5:

[0728] Recording and evaluating progress

[0729] 5.1. Recording progress

[0730] The server records the user's learning progress in a database.

[0731] Input: User's learning progress information.

[0732] Output: Progress information is saved to the database.

[0733] What it does: Executes an insert query into a MySQL database to store progress information.

[0734] 5.2. Periodic evaluation

[0735] The server periodically evaluates the user's progress and determines the next learning step.

[0736] Input: User's learning progress information.

[0737] Output: Progress assessment results.

[0738] Specific operation: Evaluation is performed using scikit-learn based on learning progress information.

[0739] 5.3. Adjusting the plan and proposing next steps

[0740] Based on the evaluation results, the server will suggest the user the best next learning steps and plan.

[0741] Input: Progress assessment results.

[0742] Output: Suggested next learning step.

[0743] Specific operation: Determine the next step based on the evaluation results, generate a prompt for GPT-4, and create a proposed sentence.

[0744] Step 6:

[0745] Maintaining motivation to learn

[0746] 6.1. Generating encouragement and praise

[0747] The server generates encouragement and praise based on the user's achievement of learning goals and progress.

[0748] Input: User's learning progress information.

[0749] Output: Encouragement and praise.

[0750] Specific behavior: Using GPT-4 to generate encouragement and praise.

[0751] 6.2. Showing encouragement and praise

[0752] The device displays encouragement and praise to the user, increasing motivation to learn.

[0753] Input: Encouragement or praise received from the server.

[0754] Output: Encouragement or praise that appears on the user's screen.

[0755] Specific behavior: Displaying encouragement and praise on the screen.

[0756] (Application example 1)

[0757] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0758] Conventional learning support systems lack the ability to provide personalized feedback based on user characteristics and progress, making it difficult to achieve efficient learning. Furthermore, they lack interactive learning support based on specific digital content, making it difficult for users to deepen their understanding of the content they are viewing. To address these issues, a more advanced interactive system based on generative AI models is needed.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0760] In this invention, the server includes means for receiving basic information of a user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on each individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and prompting the user with additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing interactive learning support based on specific digital content, means for the user to input questions and inquiries about the content being viewed, and means for generating personalized feedback about the content being viewed using a generative AI model. This allows the user to deepen their understanding of the content being viewed and to study more efficiently.

[0761] "Basic User Information" refers to personal data such as name, age, interests, and goals.

[0762] A "database" is a system for systematically storing and managing information.

[0763] "Study history" is a record of the content and progress of the user's studies to date.

[0764] A "profile" is individual information generated based on a user's characteristics and learning history.

[0765] "Personalized feedback" refers to individual answers and suggestions provided based on the user's characteristics and learning situation.

[0766] "Questions" are questions or confirmation messages sent to users.

[0767] "Realizations and discoveries" are new things that users understand or discover through learning.

[0768] "Progress" refers to the progress of learning or work.

[0769] "Evaluation" refers to judgments and assessments made based on progress and learning content.

[0770] "Adjusting your study plan" refers to modifying and optimizing your study plan based on your progress and feedback.

[0771] "Encouragement and praise" refers to positive messages provided to improve user motivation.

[0772] "Specific digital content" refers to specific video, audio, and text data intended for educational or informational purposes.

[0773] "Interactive learning support" refers to learning support that is provided through two-way interaction.

[0774] A "generative AI model" is a model of artificial intelligence used to conduct dialogue and analyze data.

[0775] "Currently viewed content" refers to digital content that a user is currently viewing.

[0776] This invention applies an interactive AI mentor system based on a generative AI model to a content distribution service. This system allows users to receive interactive learning support while viewing specific digital content. The following describes the program for this system and how to execute it.

[0777] Hardware and software used

[0778] Hardware: Smartphones, tablets

[0779] Software: Generative AI models (e.g., OpenAI's GPT-4), database management systems (e.g., MySQL), web servers (e.g., Apache)

[0780] Program Overview

[0781] The server executes a program that includes the following means:

[0782] 1. A means of receiving basic user information and storing it in a database

[0783] Users use their smartphone or tablet to enter basic information such as their name, age, areas of interest, and goals.

[0784] The terminal transmits the input information to the server.

[0785] The server stores this basic information in a database.

[0786] 2. A means to analyze a user's learning history and progress and generate a profile based on the individual user.

[0787] The server generates an individual profile based on the user's basic information and past learning history stored in a database.

[0788] 3. A means of generating personalized feedback in response to user questions or inquiries based on the generated profile.

[0789] The user inputs a question about the digital content being viewed into the terminal and transmits it to the server.

[0790] The server analyzes the profile and questions and generates personalized feedback using a generative AI model (e.g., GPT-4).

[0791] 4. A means to provide generated feedback to users and prompt them with follow-up questions to encourage their awareness and discovery.

[0792] The feedback generated by the server is sent to the terminal and displayed to the user.

[0793] The server then asks additional questions to help the user gain a deeper understanding.

[0794] 5. A means to continually track users' learning progress and provide progress-based assessments and necessary adjustments to their learning plans.

[0795] The server records the user's responses and progress in a database and evaluates them periodically.

[0796] Based on progress, we will adjust your next learning steps and plan.

[0797] 6. A means to generate encouragement and praise according to the user's achievement of learning goals and progress, and to help maintain motivation

[0798] The server generates words of encouragement and praise based on the user's progress in achieving the learning goal and sends them to the terminal.

[0799] 7. Means of providing interactive learning support based on specific digital content

[0800] The server provides learning support corresponding to the digital content that the user is currently viewing.

[0801] 8. A means for users to input questions or inquiries about the content they are viewing

[0802] The terminal provides an interface for users to input questions or inquiries about the content they are viewing.

[0803] Example prompt sentences

[0804] Below are some examples of prompts that can be passed to a generative AI model based on a user question:

[0805] Please answer the following question about the science documentary you are watching: "I would like to know more about how this experiment was done."

[0806] Based on the user's profile, they are interested in science and would like detailed instructions for experiments. Please explain them in an easy-to-understand manner for beginners.

[0807] The feedback generated based on this prompt sentence is provided to the user, realizing interactive learning support.

[0808] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0809] Step 1:

[0810] Users use their smartphones or tablets to enter basic information such as their name, age, areas of interest, and goals. This input form is provided by the device. The entered basic information is temporarily stored on the device.

[0811] Step 2:

[0812] The device sends the temporarily stored basic information to the server, which then stores the received basic information in a database. The input data is user information (name, age, areas of interest, and goals), and the output is the status of saving to the database.

[0813] Step 3:

[0814] The server retrieves the user's basic information and past learning history from the database and generates an individual profile based on that information. This profile includes the user's progress and areas of interest. The generated profile is then saved back into the database. The input is the user information and learning history data, and the output is the generated profile.

[0815] Step 4:

[0816] The user inputs a question about the digital content being viewed via the terminal. The question is sent from the terminal to the server. The input is the user's question, and the output is the status of transmission to the server.

[0817] Step 5:

[0818] The server analyzes the user's profile and question, sends prompts to a generative AI model (e.g., GPT-4) and generates personalized feedback. In this process, an appropriate response is generated based on the profile and question. The input is the prompt, and the output is the generated feedback.

[0819] Step 6:

[0820] The generated feedback is sent from the server to the terminal and displayed to the user. The user receives the feedback and deepens their understanding. The input is the generated feedback, and the output is the user's perception.

[0821] Step 7:

[0822] The server also generates questions for the user to further understand and sends them to the terminal. The questions are displayed to the user, and the user inputs answers accordingly. The input is the user's answer, and the output is the transmission status to the server.

[0823] Step 8:

[0824] The server records the user's answers and learning progress in a database and periodically evaluates the progress. The progress data is used to determine the next learning step and adjust the plan. The input is the user's answers and progress data, and the output is suggested adjustments to the learning plan.

[0825] Step 9:

[0826] Based on the user's achievement of learning goals and progress, the server generates encouragement and praise and sends them to the device. The user receives these and stays motivated. The input is progress data, and the output is encouragement and praise.

[0827] Step 10:

[0828] The server provides interactive learning support based on specific digital content. Users use their devices to input questions or inquiries about the content they are viewing. The server responds to these questions and generates appropriate feedback. The input is the user's question, and the output is the generated feedback.

[0829] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0830] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize the user's learning and provide emotional support. Below, we will explain the program and processing of this system in detail, using concrete examples.

[0831] System Overview

[0832] 1. User registration and profile creation

[0833] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[0834] Terminal: Receives user input and sends it to the server.

[0835] Server: Stores the received basic information in a database and generates an initial profile.

[0836] 2. Start the study session and take questions

[0837] User: Starts a learning session and enters the area they want to learn about and the question they have.

[0838] Terminal: Sends user input to the server.

[0839] Server: Analyzes the received questions and inquiries and simultaneously operates the emotion engine.

[0840] 3. Emotion Recognition and Feedback Generation

[0841] Server: The emotion engine analyzes the user's input and dialogue history to recognize the user's emotions. For example, the emotion engine detects emotions such as "excitement," "disappointment," and "anxiety."

[0842] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[0843] Server: Generates feedback and prompts or encouraging messages according to the user's emotions.

[0844] 4. Feedback and Questioning

[0845] Terminal: Displays feedback and prompts provided by the server to the user.

[0846] User: Review the feedback and enter answers based on the questions and emotions.

[0847] Terminal: Sends the user's answer to the server.

[0848] Server: Re-analyzes the user's response and provides appropriate additional feedback or explanation.

[0849] 5. Record and evaluate progress

[0850] Server: Records user progress in a database and keeps track of progress.

[0851] Server: Periodically evaluates progress, adjusts next learning steps and plans, and generates messages with suggestions.

[0852] Server: Provides special suggestions and support based on the user's emotions.

[0853] 6. Maintaining motivation for learning and emotional care

[0854] Server: Generates encouragement and praise based on emotion recognition according to the user's achievement of learning goals and progress.

[0855] Server: For example, if the user is feeling anxious, provide specific encouragement such as, "You're doing great! You're almost there."

[0856] On-device: Show encouragement and praise to users to increase their motivation.

[0857] User: Optionally enter a thank you message or next learning goal and submit.

[0858] Specific examples

[0859] For example, let's say User B wants to learn about the equations of motion in physics. User B inputs, "Please teach me about the equations of motion." The server performs an analysis based on User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concepts of the equations of motion. Which part do you find particularly difficult?"

[0860] If User B responds, "I don't understand the concept of acceleration," the server uses the emotion engine's recognition to provide a specific explanation, including an encouraging message such as, "Everyone has things they don't understand. I'll explain them in detail from the beginning." The server also records the progress and provides further emotional support, asking, "Your understanding of acceleration is great progress. What part would you like to learn next?"

[0861] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[0862] The processing flow will be explained below.

[0863] Step 1:

[0864] User: Accessing the system for the first time and opening the user registration page.

[0865] Device: Presents the user with a registration page, providing fields for input such as name, age, areas of interest, and learning goals.

[0866] Step 2:

[0867] User: Enter basic information and press the send button.

[0868] Terminal: Validates the entered information and sends it to the server.

[0869] Step 3:

[0870] Server: Stores the received basic information in a database and generates an initial profile.

[0871] Step 4:

[0872] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[0873] Terminal: Sends user input to the server.

[0874] Step 5:

[0875] Server: Analyzes the received questions and inquiries. At the same time, it starts the emotion engine and analyzes the user's input.

[0876] Step 6:

[0877] Server: Uses an emotion engine to recognize the user's emotions (e.g., "confused," "excited").

[0878] Step 7:

[0879] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[0880] Step 8:

[0881] Server: Generates messages including the generated feedback, questions, and even encouraging messages according to the user's emotions, and sends them to the device.

[0882] Step 9:

[0883] Terminal: Displays server-provided feedback, prompts, and encouraging messages to the user.

[0884] Step 10:

[0885] User: Check the feedback, enter an answer based on the question or emotion, and press the send button.

[0886] Step 11:

[0887] Terminal: Sends the user's answer to the server.

[0888] Step 12:

[0889] Server: Again analyzes the user's response and generates appropriate additional feedback and explanations, including special care and advice if needed, based on the emotions recognized by the emotion engine.

[0890] Step 13:

[0891] Server: Records user progress in a database and tracks progress continuously.

[0892] Step 14:

[0893] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[0894] Step 15:

[0895] Terminal: displays generated messages to the user.

[0896] Step 16:

[0897] Server: Based on the user's progress and achievement of learning goals, the server performs emotion recognition again and generates words of encouragement or praise. For example, "Your efforts are great. What would you like to learn next?"

[0898] Step 17:

[0899] On-device: Display encouragement and praise to the user to increase their motivation.

[0900] Step 18:

[0901] User: Enter a thank you message or next learning goal and submit.

[0902] By combining this conversational AI mentor system with an emotion engine, users can receive individually customized learning support and emotional care. As a concrete example, if a user is confused about the equations of motion in physics, the system will ask, "I'll start by explaining the basic concepts of the equations of motion. Which parts do you find particularly difficult?" After analyzing the user's confusion, the system will provide an encouraging message saying, "Everyone has parts they don't understand. I'll explain them in detail from the beginning." This will achieve high learning effectiveness and satisfaction.

[0903] Example 2

[0904] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0905] Conventional learning support systems often provide uniform feedback without considering the user's emotions, making it difficult to maintain the user's motivation and willingness to learn. Another issue is that feedback is not sufficiently personalized for each user, resulting in reduced learning efficiency. To address these issues, the present invention aims to provide an interactive AI mentor system combined with an emotion recognition engine, thereby realizing personalized learning and emotional care.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0907] In this invention, the server includes: means for receiving basic information about a user and saving it in a database; means for analyzing the user's learning history and progress and generating a profile based on the individual user; means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile; means for providing the generated feedback to the user and prompting the user with additional questions to encourage awareness and discovery; means including an emotion engine that recognizes the user's emotions and generating encouraging and support messages according to the user's emotions; means for continuously recording the user's learning progress and providing evaluations based on the progress and adjustments to the learning plan as needed; and means for generating encouraging and praise words according to the user's achievement of learning goals and progress to help maintain motivation. This makes it possible to individually personalize the user's learning and provide emotional support, thereby improving learning effectiveness and satisfaction.

[0908] "User" refers to an individual who uses this system to study.

[0909] "Basic Information" refers to information provided at the time of initial registration, such as the user's name, age, areas of interest, and learning goals.

[0910] "Database" refers to a system for recording and managing users' basic information, learning history, and progress information.

[0911] "Profile" refers to a set of learning information for each user that is generated based on the user's basic information and learning history.

[0912] "Feedback" refers to the response generated by the system in response to a user's question or inquiry.

[0913] A "natural language processing model" refers to an algorithm that uses a generative AI model to generate appropriate answers to users' questions and inquiries.

[0914] An "emotion engine" refers to a system that analyzes the user's input and dialogue history to recognize the user's emotions.

[0915] "Support messages" refer to messages containing encouragement or specific guidance that are generated by the system based on the user's emotions.

[0916] "Study progress" refers to information indicating how much content a user has studied and the level of understanding of the content.

[0917] "Study plan" refers to the combination of study schedules and tasks suggested by the system to help users study effectively.

[0918] "Maintaining motivation" refers to activities that support users so that they can maintain their motivation to learn during the learning process.

[0919] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize learning and provide emotional support. This system consists of a terminal used by the user to study, a server, and a network for communication between them.

[0920] 1. User registration and profile creation

[0921] First, the user enters basic information from their device, such as their name, age, areas of interest, and learning goals. This information is then sent from the device to the server, which then stores the information in a database (e.g., MySQL) and creates an initial profile.

[0922] 2. Start the study session and take questions

[0923] The user starts a learning session by inputting the subject they want to learn about and a question they have. For example, they can input a specific question such as, "Please teach me about the equation of motion." This input is sent to the server via the device. The server then analyzes the received question or inquiry using a natural language processing engine (e.g., OpenAI's GPT-4) and simultaneously runs an emotion engine (e.g., IBM Watson Tone Analyzer). This allows the system to recognize emotions from the user's input.

[0924] 3. Emotion Recognition and Feedback Generation

[0925] The server uses an emotion engine to analyze the user's input and dialogue history to recognize the user's emotions. For example, it detects "confusion" or "excitement." It then generates personalized feedback using a generative AI model (such as OpenAI's GPT-4) based on the user's profile, past learning history, and the recognized emotions. Specifically, it generates feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?"

[0926] 4. Feedback and Questioning

[0927] The generated feedback is displayed to the user via the device. The user checks the feedback and inputs a response in response to the question. For example, the user might reply, "I don't understand the concept of acceleration." This response is then sent back to the server from the device, and the server generates additional feedback and encouraging messages to support the user. For example, the server might say, "Everyone has parts they don't understand. I'll explain them in detail from the beginning."

[0928] 5. Record and evaluate progress

[0929] The server records the user's progress in a database and continuously tracks it. For example, it might record information such as "What the user learned yesterday: understanding the equation of motion" and "What the user learned today: the concept of acceleration." It also periodically evaluates the progress and adjusts the next learning step or plan. For example, it generates suggestions such as "Let's learn the concept of force next."

[0930] 6. Maintaining motivation for learning and emotional care

[0931] The system generates words of encouragement and praise based on emotion recognition according to the user's progress and achievement of learning goals. For example, if the user is feeling anxious, it will provide specific encouragement such as, "Your efforts are great. You're almost there." These messages are displayed to the user via their device, improving their motivation and helping them maintain their enthusiasm for learning.

[0932] Specific examples

[0933] For example, suppose User B wants to learn about the equation of motion in physics. User B enters, "Teach me about the equation of motion." The server analyzes User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concept of the equation of motion. Which part do you find particularly difficult?" If User B replies, "I don't understand the concept of acceleration," the server provides a specific explanation with an encouraging message, "Everyone has parts they don't understand. I'll explain in detail from the beginning." Progress is also recorded, and further questions and emotional support are provided, such as, "Your understanding of acceleration has improved, which is great. What part would you like to learn next?"

[0934] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[0935] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0936] Step 1:

[0937] Enter and submit basic information

[0938] When a user uses the system for the first time, they enter basic information such as their name, age, areas of interest, and goals. This input is entered into the terminal and then sent to the server. Specifically, when the user enters information into the form and presses the "Submit" button, the data is sent from the terminal to the server. Examples of input are "Name: Tanaka Taro," "Age: 20 years old," "Area of ​​interest: Chemistry," and "Goal: I want to gain a deep understanding from the basics." The output is the basic information data that reaches the server.

[0939] Step 2:

[0940] Generate a profile

[0941] The server stores the received basic information in a database (e.g., MySQL). If the storage is successful, the server generates an initial profile. Specifically, it generates a unique profile ID for each user based on the content stored in the database, associates it with related information, and stores it. The input is the basic information sent from the device, and the output is the user profile data recorded in the database.

[0942] Step 3:

[0943] Start a study session and enter your questions

[0944] A user starts a learning session and inputs the area they want to learn about and a specific question. For example, they might input, "Tell me about the equation of motion." This input is sent from the device to the server. An example input might be, "Could you tell me which part you don't understand?" The output is the specific question data sent to the server.

[0945] Step 4:

[0946] Question analysis and emotion recognition

[0947] The received question or consultation content is analyzed using a natural language processing engine (e.g., a generative AI model). An emotion engine runs in parallel to recognize emotions from the user's input. Specifically, when identifying the emotion "confused," the server calculates an emotion evaluation score based on the relevant keywords and context. The input is the user's question, and the output is the analysis result and emotion recognition result.

[0948] Step 5:

[0949] Feedback Generation

[0950] The server generates personalized feedback based on the user's recognized emotions, profile information, and past learning history. It uses a generative AI model to construct appropriate answers and include encouraging messages. For example, feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?" is generated. The input is emotions and analysis results, and the output is the generated feedback.

[0951] Step 6:

[0952] Displaying feedback and user responses

[0953] The generated feedback is displayed to the user through the device. The user checks the feedback and inputs a response to the question. For example, the user might reply, "I don't understand the concept of acceleration." The device receives this response and sends it to the server. The input is the feedback from the server, and the output is the user's response data.

[0954] Step 7:

[0955] Generate additional feedback and encouraging messages

[0956] The server analyzes the user's response again and generates appropriate additional feedback. It also includes data from the emotion engine and adds an encouraging message. For example, it provides feedback such as, "Everyone has some things they don't understand. I'll explain them in detail from the beginning." The input is the user's response data, and the output is the generated additional feedback.

[0957] Step 8:

[0958] Recording and evaluating progress

[0959] The server records the user's progress in a database. For example, it may record information such as "Today's learning: Equation of motion" or "Level of understanding: Medium." Furthermore, it periodically evaluates the user's progress and adjusts the next learning step and plan. The input is the user's learning data, and the output is the progress evaluation results and a new learning plan proposal.

[0960] Step 9:

[0961] Maintaining motivation for learning and emotional care

[0962] The server generates encouraging and praising messages based on emotion recognition according to the user's achievement of learning goals and progress. For example, it generates messages such as "Your efforts are great. You're almost there." This increases the user's motivation. The input is learning progress and evaluation data, and the output is encouraging and praising messages.

[0963] In this way, the system personalizes the user's learning and provides emotional support through a series of steps.

[0964] (Application example 2)

[0965] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0966] While conventional conversational AI systems can provide feedback to users' questions, they have difficulty providing appropriate responses and encouragement based on the user's emotional state. This can lead to users feeling stressed or losing motivation to learn without receiving appropriate support. Furthermore, in customer service at brick-and-mortar stores, there was a lack of support that took into account the customer's emotions, leading to concerns about a decline in customer satisfaction.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving basic information about the user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, and means for providing appropriate feedback and support messages in response to the user's emotions based on emotion recognition. This makes it possible to respond according to the user's emotional state, thereby reducing stress, increasing motivation to learn, and improving customer satisfaction.

[0968] "User" refers to a person who uses the system.

[0969] "Basic information" refers to the initial personal information a user provides to the system, such as name, age, areas of interest, and goals.

[0970] "Database" refers to the digital data structure that the system uses to store basic information about users, their learning history, and so on.

[0971] "Profile" refers to an individual learning model generated by analyzing a user's basic information, learning history, and progress data.

[0972] "Feedback" refers to the responses and advice the system provides to users in response to their questions or inquiries.

[0973] "Emotion recognition" refers to the process of analyzing a user's input and dialogue history to identify the user's emotional state at that time.

[0974] A "natural language processing model" refers to machine learning technology that understands users' questions and inquiries and generates appropriate feedback.

[0975] "Support messages" refer to messages generated based on emotion recognition to encourage and support users.

[0976] "Learning history" refers to the record of learning that a user has done through the system.

[0977] "Personalization" refers to responses that are customized to suit the characteristics and needs of each individual user.

[0978] "Progress assessment" refers to the process of periodically assessing a user's learning progress.

[0979] "Study Plan" means the specific steps or plan established to achieve a User's learning goals.

[0980] "Maintaining motivation" refers to supporting users to maintain their desire to learn.

[0981] "Customer" refers to a person who visits a physical store in search of a product or service.

[0982] This invention can be implemented as an interactive AI shopping assistant system that can be used in brick-and-mortar stores. A specific example of this system will now be described.

[0983] System configuration

[0984] The system consists of a smartphone, a server, an emotion engine, and a generative AI model. The smartphone is a device that the user can carry around and provides an interactive user interface. The server is the main processing unit that processes input from the user and generates the necessary feedback. The emotion engine is software that analyzes the user's emotional state from text input, and the generative AI model is a system that generates appropriate responses through natural language processing.

[0985] System Operation

[0986] 1. User registration and profile creation: The device (smartphone) receives the user's basic information (name, areas of interest, goals, etc.) and sends it to the server. The server stores this information in a database and creates a user profile.

[0987] 2. Question acceptance: A user uses a smartphone to input a question in a physical store, for example, "Where is this product?"

[0988] 3. Emotion recognition and feedback generation: The server receives the user's text input and analyzes their emotional state using an emotion engine. It then uses a generative AI model to generate appropriate feedback. For example, if the emotion engine detects that the user is confused, it can generate a response like, "I'm sorry, what are you looking for? I'll help you." It can also provide specific location information based on the user's input, such as, "Your item is at Aisle 5."

[0989] 4. Feedback presentation: The smartphone displays the feedback to the user, who can then head to the recommended location. If the user enters additional questions, the server again generates a response using the emotion engine and generative AI model.

[0990] Software and hardware used

[0991] Hardware: Smartphones, servers

[0992] Software: OpenAI generative AI models (e.g., GPT-3), EmotionEngine

[0993] The process involves the smartphone receiving the user's input and sending it to the server, which then analyzes the input and uses an emotion engine and generative AI model to generate appropriate feedback, which is then sent to the user's smartphone and displayed.

[0994] Specific examples

[0995] For example, if a user types "I don't know where this product is" into their smartphone, the emotion engine will detect "confusion," and the server will provide an encouraging message saying, "I'm sorry, what are you looking for? I'll help you," along with a specific recommended location, such as "That product is in Aisle 5."

[0996] Prompt Sentence Examples

[0997] An example of a prompt that a user might enter into the system is:

[0998] text

[0999] User Question: I don't know where this item is.

[1000] Provide appropriate assistance.

[1001] In this way, it is possible to increase user satisfaction by generating optimal feedback and support messages based on emotion recognition.

[1002] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1003] Step 1: The user uses their smartphone to enter basic information (such as their name, areas of interest, and goals). The device receives this input and sends it to the server.

[1004] Step 2: The server stores the received basic information in a database. It also generates a user profile and integrates it with other information stored in the database. The input is the user's basic information, and the output is the generated profile.

[1005] Step 3: The user uses their smartphone to input a specific question (e.g., "Where is this product?"). The device receives the user's input and sends it to the server.

[1006] Step 4: The server analyzes the received question using an emotion engine to detect the user's emotional state. The input is the user's text input, and the output is the detected emotional state. Specifically, the emotion engine analyzes the emotional nuances of the text and identifies emotions such as "confused" or "excited."

[1007] Step 5: The server uses the generative AI model to generate appropriate feedback for the user's question. The input is the user's question and the detected emotional state, and the output is personalized feedback. For example, if the user types, "I don't know where the product is," the generative AI model will generate the response, "The product is on Aisle 5."

[1008] Step 6: Based on the emotional state, the server further generates an appropriate support message for the user. The input is the detected emotional state, and the output is a support message. For example, if "confusion" is detected, the server generates a message saying "I'm sorry, what are you looking for? I'm here to help you."

[1009] Step 7: The server sends the generated feedback and support message to the terminal, which displays it to the user. The user confirms the feedback and enters further questions if necessary.

[1010] Step 8: The server continuously records the user's progress and updates their profile based on that data, making future interactions more personalized. The input is the user's feedback and emotion history, and the output is an updated profile.

[1011] These steps provide personalized feedback and support messages tailored to the user's emotional state.

[1012] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1013] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1014] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1015] [Third embodiment]

[1016] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1017] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1019] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1020] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1021] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1023] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1024] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1026] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1027] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1028] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support based on the user's characteristics and learning situation. This system enables users to study efficiently in line with their interests and goals, and develop essential problem-solving skills. Below, we will explain the system's program and its processing in natural language, including concrete examples.

[1029] System Overview

[1030] 1. User registration and profile creation

[1031] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[1032] Terminal: Receives user input and sends it to the server.

[1033] Server: Stores the received basic information in a database and generates an initial profile.

[1034] 2. Start the study session and take questions

[1035] User: Starts a learning session and enters questions and inquiries.

[1036] Terminal: Receives user input and sends it to the server.

[1037] Server: Analyzes the received question or inquiry and passes it to the appropriate dialogue model.

[1038] 3. Generate personalized feedback

[1039] Server: Based on the user's profile and learning history, the generative AI model analyzes the data and generates feedback and suggestions appropriate for the user.

[1040] Server: Generates a message containing a question along with feedback and sends it to the device.

[1041] 4. Feedback and Questioning

[1042] Terminal: Displays feedback and prompts provided by the server to the user.

[1043] User: Enter your understanding and opinion on the question and submit.

[1044] Terminal: Sends the user's answer to the server.

[1045] Server: Analyzes the user's response and provides additional feedback or explanation.

[1046] 5. Record and evaluate progress

[1047] Server: Records user progress in a database.

[1048] Server: Regularly evaluate progress and make next steps or adjustments to the plan.

[1049] Server: Based on the progress assessment, it provides feedback to the user and suggests any necessary details.

[1050] 6. Maintaining motivation to learn

[1051] Server: Generates and sends encouragement and praise to users based on their progress and achievement of learning goals.

[1052] On-device: Show encouragement and praise to users to increase their motivation.

[1053] Specific examples

[1054] For example, suppose User A wants to learn about vectors in mathematics. User A accesses the system and enters, "I'd like to learn about vector addition." The server uses User A's basic information and past learning history to provide a brief explanation of the basic concept of vectors and asks, "How much do you currently understand?" If User A then replies, "I understand the basics of addition, but I'm having trouble applying it," the server will provide specific application examples and additional materials. It will also record User A's progress and send an encouraging message saying, "You've achieved today's learning goal. Is there anything you'd like to advance to next?"

[1055] This system allows users to learn at their own pace and receives constant personalized feedback, resulting in high learning outcomes.

[1056] The processing flow will be explained below.

[1057] Step 1:

[1058] User: Accessing the system for the first time and opening the user registration page.

[1059] Terminal: Displays the registration page to the user and provides fields for entering the necessary information.

[1060] Step 2:

[1061] User: Enter basic information such as name, age, areas of interest, and learning goals, then press the submit button.

[1062] Terminal: Sends the entered information to the server.

[1063] Step 3:

[1064] Server: Stores the received basic information in a database.

[1065] Server: Generates an initial profile based on basic information.

[1066] Step 4:

[1067] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[1068] Terminal: Sends user input to the server.

[1069] Step 5:

[1070] Server: Analyzes the received questions and inquiries.

[1071] Server: Reads the user's profile and past learning history from the database and understands the user's characteristics.

[1072] Step 6:

[1073] Server: Calls the generative AI model based on the analysis results and generates personalized feedback.

[1074] Server: Generates a message containing a question to get the user to think along with the generated feedback.

[1075] Step 7:

[1076] Terminal: Displays feedback and prompts from the server to the user.

[1077] User: Check the feedback, enter their understanding and opinion of the question, and press the submit button.

[1078] Step 8:

[1079] Terminal: Sends the user's answer to the server.

[1080] Server: Parses the received answer and generates a message to provide additional feedback or clarification.

[1081] Step 9:

[1082] Server: Records user progress in a database and tracks progress continuously.

[1083] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[1084] Step 10:

[1085] Terminal: displays generated messages to the user.

[1086] Server: Generates encouragement and praise based on the user's achievement of learning goals and progress.

[1087] Server: Generates messages containing encouragement and praise and sends them to the device.

[1088] Step 11:

[1089] On-device: Display encouragement and praise to the user to increase their motivation.

[1090] User: Enter a thank you message or next learning goal if desired, and click the send button.

[1091] These are the specific processing steps of this interactive AI mentor system, which allows users to receive individually customized learning support and progress effectively at their own pace.

[1092] Example 1

[1093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1094] Conventional learning support systems lacked personalized support tailored to each user's characteristics and learning progress, making efficient learning difficult. Furthermore, there were limited ways to continuously maintain users' motivation to learn, which meant that learning effectiveness was not maximized. Furthermore, the quality and timing of feedback often did not meet users' needs, which reduced learning efficiency.

[1095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1096] In this invention, the server includes means for receiving basic information about a user and saving it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and asking additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing optimal feedback and suggestions to the user using a generative AI model, and means for generating prompt sentences, inputting the prompt sentences into the generative AI model, and providing the generated output to the user. This enables personalized support based on the user's characteristics and learning situation, maintaining motivation to learn, and providing effective feedback.

[1097] "User" refers to an individual who uses this system to carry out learning activities or ask questions.

[1098] "Basic information" refers to information entered during initial registration, such as the user's name, age, areas of interest, and learning goals.

[1099] "Database" refers to a digital information storage location for storing basic information, learning history, and progress data of users.

[1100] A "profile" refers to a collection of information specific to an individual user, generated based on the user's basic information, learning history, and progress.

[1101] "Feedback" refers to answers, suggestions, comments, etc. provided in response to users' questions or inquiries.

[1102] "Generative AI model" refers to an artificial intelligence model that generates optimal feedback and suggestions for users.

[1103] "Natural language processing model" refers to a software model used to analyze user input and generate an appropriate response.

[1104] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1105] "Progress" refers to the results of evaluating a user's learning situation and level of achievement.

[1106] "Study plan" refers to a plan or program designed to help users study efficiently.

[1107] "Encouragement and praise" refers to messages and comments provided to motivate users to learn.

[1108] "Continuously recording" means that the user's learning progress is periodically saved to a database.

[1109] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support according to the user's characteristics and learning situation. The program processing of this system is explained in detail below.

[1110] System configuration

[1111] This system mainly consists of three elements: the server, the terminal, and the user.

[1112] server

[1113] The server plays a central role in storing and analyzing data and generating feedback using generative AI models. The hardware used is a cloud-based server, and the corresponding software includes the following:

[1114] Database: MySQL

[1115] Natural language processing library: spaCy

[1116] Generative AI model: OpenAI's GPT-4

[1117] Programming language: Python

[1118] The server does the following:

[1119] 1. Store the basic information received from the user in a MySQL database.

[1120] 1. Use spaCy to parse user input and extract keywords.

[1121] 1. Use GPT-4 to generate feedback and suggestions.

[1122] 1. Evaluate your learning progress and adjust your next steps and learning plan.

[1123] Terminal

[1124] The terminal is responsible for providing an interface with the user, sending user input to the server, and displaying feedback from the server. The terminal's functions include:

[1125] 1. Receive user input in real time and send it to the server.

[1126] 1. Display feedback and questions received from the server to the user.

[1127] user

[1128] Users use the system to carry out learning activities and receive feedback to progress their learning. Users perform the following actions:

[1129] 1. Enter your basic information and learning goals.

[1130] 1. Enter any questions or concerns you may have while studying.

[1131] 1. Type your response to the feedback or questions provided.

[1132] Specific actions

[1133] The operation of this system will be explained using a specific example.

[1134] User Registration

[1135] When a user first accesses the system, they enter basic information such as their name, age, areas of interest, and learning goals. The device receives the information and sends it to the server in real time. The server stores the information in a MySQL database and creates an initial profile.

[1136] Learning sessions and questions

[1137] If a user wants to learn about "vector addition," they type that into their device. The device receives the question and sends it to the server, which uses spaCy to parse it and GPT-4 to generate appropriate feedback.

[1138] Providing feedback and asking questions

[1139] The server generates feedback and sends it to the device, which then displays it to the user. For example, if you type "Teach me about vector addition," the server will explain the basic concept of vectors and ask, "How well do you understand it now?"

[1140] Giving and receiving feedback

[1141] If the user responds, "I understand the basics of addition, but I'm having trouble applying it," the server will again display specific application examples and additional materials. By repeating this process, the server can provide optimal learning support to the user.

[1142] Prompt Sentence Examples

[1143] For example, the following prompt sentence is input to the generative AI model:

[1144] "If you're a 15-year-old student who's good at math, explain the basic concept of vector addition and ask if they'd like to see some applications. Also, ask follow-up questions to check their understanding."

[1145] By providing the user with feedback messages generated based on these prompts, more effective learning support is possible.

[1146] summary

[1147] This interactive AI mentoring system allows users to receive personalized support based on their individual characteristics and learning progress. The server provides feedback using a generative AI model, enabling efficient learning and maintaining motivation.

[1148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1149] Step 1:

[1150] User registration and profile creation

[1151] 1.1. Collecting User Input

[1152] Users enter basic information such as name, age, interests, and learning goals.

[1153] Input: A user enters information into a web form (e.g., "Taro Tanaka, 17 years old, mathematics, university exams").

[1154] Output: The basic information entered is sent to the terminal.

[1155] What happens: A user fills out a form and clicks the submit button.

[1156] 1.2. Transmission of User Information

[1157] The device receives the user's input and transmits it to the server in real time.

[1158] Input: Basic information submitted via the web form.

[1159] Output: Basic information is sent to the server.

[1160] Specific operation: The front-end script takes the input information and sends it to the server via the API.

[1161] 1.3. Server-based information storage and profile generation

[1162] The server stores the received basic information in a database and generates an initial profile.

[1163] Input: User basic information.

[1164] Output: The initial profile that will be saved in the database.

[1165] What it does: Executes an insert query in the MySQL database to save the information. Generates a profile ID.

[1166] Step 2:

[1167] Start a study session and take questions

[1168] 2.1. User Question Input

[1169] A user begins a learning session by typing in a question: "Teach me about vector addition."

[1170] Input: The user types a question into the terminal.

[1171] Output: The entered question is received by the terminal.

[1172] What it does: A user starts a learning session and uses a form field to enter a question.

[1173] 2.2. Submitting a Question

[1174] The device receives the user's question and sends it to the server.

[1175] Input: The question entered by the user.

[1176] Output: The question is sent to the server.

[1177] What happens: JavaScript or other front-end scripts submit questions to the server via an API.

[1178] 2.3. Query analysis by the server

[1179] The server parses the question and passes it to the appropriate dialogue model.

[1180] Input: Question sent from terminal.

[1181] Output: Feedback generated based on analysis results.

[1182] What it does: Uses spaCy to extract keywords from the question and prepares them for passing to the AI ​​model.

[1183] Step 3:

[1184] Generate personalized feedback

[1185] 3.1. Obtaining user information and learning history

[1186] The server retrieves the user's profile information and learning history from the database.

[1187] Input: User profile ID.

[1188] Output: Profile information and learning history.

[1189] Specific behavior: Executes a SELECT query based on the user ID in the MySQL database to retrieve information.

[1190] 3.2. Analysis and feedback generation using AI models

[1191] The server uses a generative AI model such as GPT-4 to generate appropriate feedback and suggestions.

[1192] Input: User information, learning history, question keywords.

[1193] Output: The generated feedback.

[1194] Specific operation: Generate a prompt sentence, input it into GPT-4, and generate feedback.

[1195] 3.3. Sending Feedback

[1196] The server transmits the generated feedback to the terminal.

[1197] Input: Generated feedback.

[1198] Output: Feedback is sent to the device.

[1199] Specific behavior: Sends feedback to the device using an API call.

[1200] Step 4:

[1201] Feedback and questions

[1202] 4.1. Display of Feedback

[1203] The device displays the feedback it receives from the server to the user.

[1204] Input: Feedback received from the server.

[1205] Output: Feedback that is displayed on the user's screen.

[1206] Specific behavior: Display a feedback message on the screen.

[1207] 4.2. User response input

[1208] Users enter responses to the feedback and questions provided.

[1209] Input: The user's response.

[1210] Output: The reply is received at the terminal.

[1211] Specific behavior: The user types a response and presses the send button.

[1212] 4.3. Sending a Reply

[1213] The device receives the user's response and sends it to the server.

[1214] Input: The user's response.

[1215] Output: The reply is sent to the server.

[1216] What happens: JavaScript or other front-end scripts send a response to the server via an API.

[1217] 4.4. Server Response Analysis and Additional Feedback Generation

[1218] The server analyzes the user's response and provides additional feedback or explanation.

[1219] Input: The user's response.

[1220] Output: Additional feedback.

[1221] What it does: We again use spaCy to parse the response and GPT-4 to generate additional feedback.

[1222] Step 5:

[1223] Recording and evaluating progress

[1224] 5.1. Recording progress

[1225] The server records the user's learning progress in a database.

[1226] Input: User's learning progress information.

[1227] Output: Progress information is saved to the database.

[1228] What it does: Executes an insert query into a MySQL database to store progress information.

[1229] 5.2. Periodic evaluation

[1230] The server periodically evaluates the user's progress and determines the next learning step.

[1231] Input: User's learning progress information.

[1232] Output: Progress assessment results.

[1233] Specific operation: Evaluation is performed using scikit-learn based on learning progress information.

[1234] 5.3. Adjusting the plan and proposing next steps

[1235] Based on the evaluation results, the server will suggest the user the best next learning steps and plan.

[1236] Input: Progress assessment results.

[1237] Output: Suggested next learning step.

[1238] Specific operation: Determine the next step based on the evaluation results, generate a prompt for GPT-4, and create a proposed sentence.

[1239] Step 6:

[1240] Maintaining motivation to learn

[1241] 6.1. Generating encouragement and praise

[1242] The server generates encouragement and praise based on the user's achievement of learning goals and progress.

[1243] Input: User's learning progress information.

[1244] Output: Encouragement and praise.

[1245] Specific behavior: Using GPT-4 to generate encouragement and praise.

[1246] 6.2. Showing encouragement and praise

[1247] The device displays encouragement and praise to the user, increasing motivation to learn.

[1248] Input: Encouragement or praise received from the server.

[1249] Output: Encouragement or praise that appears on the user's screen.

[1250] Specific behavior: Displaying encouragement and praise on the screen.

[1251] (Application example 1)

[1252] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1253] Conventional learning support systems lack the ability to provide personalized feedback based on user characteristics and progress, making it difficult to achieve efficient learning. Furthermore, they lack interactive learning support based on specific digital content, making it difficult for users to deepen their understanding of the content they are viewing. To address these issues, a more advanced interactive system based on generative AI models is needed.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1255] In this invention, the server includes means for receiving basic information of a user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on each individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and prompting the user with additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing interactive learning support based on specific digital content, means for the user to input questions and inquiries about the content being viewed, and means for generating personalized feedback about the content being viewed using a generative AI model. This allows the user to deepen their understanding of the content being viewed and to study more efficiently.

[1256] "Basic User Information" refers to personal data such as name, age, interests, and goals.

[1257] A "database" is a system for systematically storing and managing information.

[1258] "Study history" is a record of the content and progress of the user's studies to date.

[1259] A "profile" is individual information generated based on a user's characteristics and learning history.

[1260] "Personalized feedback" refers to individual answers and suggestions provided based on the user's characteristics and learning situation.

[1261] "Questions" are questions or confirmation messages sent to users.

[1262] "Realizations and discoveries" are new things that users understand or discover through learning.

[1263] "Progress" refers to the progress of learning or work.

[1264] "Evaluation" refers to judgments and assessments made based on progress and learning content.

[1265] "Adjusting your study plan" refers to modifying and optimizing your study plan based on your progress and feedback.

[1266] "Encouragement and praise" refers to positive messages provided to improve user motivation.

[1267] "Specific digital content" refers to specific video, audio, and text data intended for educational or informational purposes.

[1268] "Interactive learning support" refers to learning support that is provided through two-way interaction.

[1269] A "generative AI model" is a model of artificial intelligence used to conduct dialogue and analyze data.

[1270] "Currently viewed content" refers to digital content that a user is currently viewing.

[1271] This invention applies an interactive AI mentor system based on a generative AI model to a content distribution service. This system allows users to receive interactive learning support while viewing specific digital content. The following describes the program for this system and how to execute it.

[1272] Hardware and software used

[1273] Hardware: Smartphones, tablets

[1274] Software: Generative AI models (e.g., OpenAI's GPT-4), database management systems (e.g., MySQL), web servers (e.g., Apache)

[1275] Program Overview

[1276] The server executes a program that includes the following means:

[1277] 1. A means of receiving basic user information and storing it in a database

[1278] Users use their smartphone or tablet to enter basic information such as their name, age, areas of interest, and goals.

[1279] The terminal transmits the input information to the server.

[1280] The server stores this basic information in a database.

[1281] 2. A means to analyze a user's learning history and progress and generate a profile based on the individual user.

[1282] The server generates an individual profile based on the user's basic information and past learning history stored in a database.

[1283] 3. A means of generating personalized feedback in response to user questions or inquiries based on the generated profile.

[1284] The user inputs a question about the digital content being viewed into the terminal and transmits it to the server.

[1285] The server analyzes the profile and questions and generates personalized feedback using a generative AI model (e.g., GPT-4).

[1286] 4. A means to provide generated feedback to users and prompt them with follow-up questions to encourage their awareness and discovery.

[1287] The feedback generated by the server is sent to the terminal and displayed to the user.

[1288] The server then asks additional questions to help the user gain a deeper understanding.

[1289] 5. A means to continually track users' learning progress and provide progress-based assessments and necessary adjustments to their learning plans.

[1290] The server records the user's responses and progress in a database and evaluates them periodically.

[1291] Based on progress, we will adjust your next learning steps and plan.

[1292] 6. A means to generate encouragement and praise according to the user's achievement of learning goals and progress, and to help maintain motivation

[1293] The server generates words of encouragement and praise based on the user's progress in achieving the learning goal and sends them to the terminal.

[1294] 7. Means of providing interactive learning support based on specific digital content

[1295] The server provides learning support corresponding to the digital content that the user is currently viewing.

[1296] 8. A means for users to input questions or inquiries about the content they are viewing

[1297] The terminal provides an interface for users to input questions or inquiries about the content they are viewing.

[1298] Example prompt sentences

[1299] Below are some examples of prompts that can be passed to a generative AI model based on a user question:

[1300] Please answer the following question about the science documentary you are watching: "I would like to know more about how this experiment was done."

[1301] Based on the user's profile, they are interested in science and would like detailed instructions for experiments. Please explain them in an easy-to-understand manner for beginners.

[1302] The feedback generated based on this prompt sentence is provided to the user, realizing interactive learning support.

[1303] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1304] Step 1:

[1305] Users use their smartphones or tablets to enter basic information such as their name, age, areas of interest, and goals. This input form is provided by the device. The entered basic information is temporarily stored on the device.

[1306] Step 2:

[1307] The device sends the temporarily stored basic information to the server, which then stores the received basic information in a database. The input data is user information (name, age, areas of interest, and goals), and the output is the status of saving to the database.

[1308] Step 3:

[1309] The server retrieves the user's basic information and past learning history from the database and generates an individual profile based on that information. This profile includes the user's progress and areas of interest. The generated profile is then saved back into the database. The input is the user information and learning history data, and the output is the generated profile.

[1310] Step 4:

[1311] The user inputs a question about the digital content being viewed via the terminal. The question is sent from the terminal to the server. The input is the user's question, and the output is the status of transmission to the server.

[1312] Step 5:

[1313] The server analyzes the user's profile and question, sends prompts to a generative AI model (e.g., GPT-4) and generates personalized feedback. In this process, an appropriate response is generated based on the profile and question. The input is the prompt, and the output is the generated feedback.

[1314] Step 6:

[1315] The generated feedback is sent from the server to the terminal and displayed to the user. The user receives the feedback and deepens their understanding. The input is the generated feedback, and the output is the user's perception.

[1316] Step 7:

[1317] The server also generates questions for the user to further understand and sends them to the terminal. The questions are displayed to the user, and the user inputs answers accordingly. The input is the user's answer, and the output is the transmission status to the server.

[1318] Step 8:

[1319] The server records the user's answers and learning progress in a database and periodically evaluates the progress. The progress data is used to determine the next learning step and adjust the plan. The input is the user's answers and progress data, and the output is suggested adjustments to the learning plan.

[1320] Step 9:

[1321] Based on the user's achievement of learning goals and progress, the server generates encouragement and praise and sends them to the device. The user receives these and stays motivated. The input is progress data, and the output is encouragement and praise.

[1322] Step 10:

[1323] The server provides interactive learning support based on specific digital content. Users use their devices to input questions or inquiries about the content they are viewing. The server responds to these questions and generates appropriate feedback. The input is the user's question, and the output is the generated feedback.

[1324] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1325] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize the user's learning and provide emotional support. Below, we will explain the program and processing of this system in detail, using concrete examples.

[1326] System Overview

[1327] 1. User registration and profile creation

[1328] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[1329] Terminal: Receives user input and sends it to the server.

[1330] Server: Stores the received basic information in a database and generates an initial profile.

[1331] 2. Start the study session and take questions

[1332] User: Starts a learning session and enters the area they want to learn about and the question they have.

[1333] Terminal: Sends user input to the server.

[1334] Server: Analyzes the received questions and inquiries and simultaneously operates the emotion engine.

[1335] 3. Emotion Recognition and Feedback Generation

[1336] Server: The emotion engine analyzes the user's input and dialogue history to recognize the user's emotions. For example, the emotion engine detects emotions such as "excitement," "disappointment," and "anxiety."

[1337] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[1338] Server: Generates feedback and prompts or encouraging messages according to the user's emotions.

[1339] 4. Feedback and Questioning

[1340] Terminal: Displays feedback and prompts provided by the server to the user.

[1341] User: Review the feedback and enter answers based on the questions and emotions.

[1342] Terminal: Sends the user's answer to the server.

[1343] Server: Re-analyzes the user's response and provides appropriate additional feedback or explanation.

[1344] 5. Record and evaluate progress

[1345] Server: Records user progress in a database and keeps track of progress.

[1346] Server: Periodically evaluates progress, adjusts next learning steps and plans, and generates messages with suggestions.

[1347] Server: Provides special suggestions and support based on the user's emotions.

[1348] 6. Maintaining motivation for learning and emotional care

[1349] Server: Generates encouragement and praise based on emotion recognition according to the user's achievement of learning goals and progress.

[1350] Server: For example, if the user is feeling anxious, provide specific encouragement such as, "You're doing great! You're almost there."

[1351] On-device: Show encouragement and praise to users to increase their motivation.

[1352] User: Optionally enter a thank you message or next learning goal and submit.

[1353] Specific examples

[1354] For example, let's say User B wants to learn about the equations of motion in physics. User B inputs, "Please teach me about the equations of motion." The server performs an analysis based on User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concepts of the equations of motion. Which part do you find particularly difficult?"

[1355] If User B responds, "I don't understand the concept of acceleration," the server uses the emotion engine's recognition to provide a specific explanation, including an encouraging message such as, "Everyone has things they don't understand. I'll explain them in detail from the beginning." The server also records the progress and provides further emotional support, asking, "Your understanding of acceleration is great progress. What part would you like to learn next?"

[1356] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[1357] The processing flow will be explained below.

[1358] Step 1:

[1359] User: Accessing the system for the first time and opening the user registration page.

[1360] Device: Presents the user with a registration page, providing fields for input such as name, age, areas of interest, and learning goals.

[1361] Step 2:

[1362] User: Enter basic information and press the send button.

[1363] Terminal: Validates the entered information and sends it to the server.

[1364] Step 3:

[1365] Server: Stores the received basic information in a database and generates an initial profile.

[1366] Step 4:

[1367] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[1368] Terminal: Sends user input to the server.

[1369] Step 5:

[1370] Server: Analyzes the received questions and inquiries. At the same time, it starts the emotion engine and analyzes the user's input.

[1371] Step 6:

[1372] Server: Uses an emotion engine to recognize the user's emotions (e.g., "confused," "excited").

[1373] Step 7:

[1374] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[1375] Step 8:

[1376] Server: Generates messages including the generated feedback, questions, and even encouraging messages according to the user's emotions, and sends them to the device.

[1377] Step 9:

[1378] Terminal: Displays server-provided feedback, prompts, and encouraging messages to the user.

[1379] Step 10:

[1380] User: Check the feedback, enter an answer based on the question or emotion, and press the send button.

[1381] Step 11:

[1382] Terminal: Sends the user's answer to the server.

[1383] Step 12:

[1384] Server: Again analyzes the user's response and generates appropriate additional feedback and explanations, including special care and advice if needed, based on the emotions recognized by the emotion engine.

[1385] Step 13:

[1386] Server: Records user progress in a database and tracks progress continuously.

[1387] Step 14:

[1388] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[1389] Step 15:

[1390] Terminal: displays generated messages to the user.

[1391] Step 16:

[1392] Server: Based on the user's progress and achievement of learning goals, the server performs emotion recognition again and generates words of encouragement or praise. For example, "Your efforts are great. What would you like to learn next?"

[1393] Step 17:

[1394] On-device: Display encouragement and praise to the user to increase their motivation.

[1395] Step 18:

[1396] User: Enter a thank you message or next learning goal and submit.

[1397] By combining this conversational AI mentor system with an emotion engine, users can receive individually customized learning support and emotional care. As a concrete example, if a user is confused about the equations of motion in physics, the system will ask, "I'll start by explaining the basic concepts of the equations of motion. Which parts do you find particularly difficult?" After analyzing the user's confusion, the system will provide an encouraging message saying, "Everyone has parts they don't understand. I'll explain them in detail from the beginning." This will achieve high learning effectiveness and satisfaction.

[1398] Example 2

[1399] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1400] Conventional learning support systems often provide uniform feedback without considering the user's emotions, making it difficult to maintain the user's motivation and willingness to learn. Another issue is that feedback is not sufficiently personalized for each user, resulting in reduced learning efficiency. To address these issues, the present invention aims to provide an interactive AI mentor system combined with an emotion recognition engine, thereby realizing personalized learning and emotional care.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1402] In this invention, the server includes: means for receiving basic information about a user and saving it in a database; means for analyzing the user's learning history and progress and generating a profile based on the individual user; means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile; means for providing the generated feedback to the user and prompting the user with additional questions to encourage awareness and discovery; means including an emotion engine that recognizes the user's emotions and generating encouraging and support messages according to the user's emotions; means for continuously recording the user's learning progress and providing evaluations based on the progress and adjustments to the learning plan as needed; and means for generating encouraging and praise words according to the user's achievement of learning goals and progress to help maintain motivation. This makes it possible to individually personalize the user's learning and provide emotional support, thereby improving learning effectiveness and satisfaction.

[1403] "User" refers to an individual who uses this system to study.

[1404] "Basic Information" refers to information provided at the time of initial registration, such as the user's name, age, areas of interest, and learning goals.

[1405] "Database" refers to a system for recording and managing users' basic information, learning history, and progress information.

[1406] "Profile" refers to a set of learning information for each user that is generated based on the user's basic information and learning history.

[1407] "Feedback" refers to the response generated by the system in response to a user's question or inquiry.

[1408] A "natural language processing model" refers to an algorithm that uses a generative AI model to generate appropriate answers to users' questions and inquiries.

[1409] An "emotion engine" refers to a system that analyzes the user's input and dialogue history to recognize the user's emotions.

[1410] "Support messages" refer to messages containing encouragement or specific guidance that are generated by the system based on the user's emotions.

[1411] "Study progress" refers to information indicating how much content a user has studied and the level of understanding of the content.

[1412] "Study plan" refers to the combination of study schedules and tasks suggested by the system to help users study effectively.

[1413] "Maintaining motivation" refers to activities that support users so that they can maintain their motivation to learn during the learning process.

[1414] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize learning and provide emotional support. This system consists of a terminal used by the user to study, a server, and a network for communication between them.

[1415] 1. User registration and profile creation

[1416] First, the user enters basic information from their device, such as their name, age, areas of interest, and learning goals. This information is then sent from the device to the server, which then stores the information in a database (e.g., MySQL) and creates an initial profile.

[1417] 2. Start the study session and take questions

[1418] The user starts a learning session by inputting the subject they want to learn about and a question they have. For example, they can input a specific question such as, "Please teach me about the equation of motion." This input is sent to the server via the device. The server then analyzes the received question or inquiry using a natural language processing engine (e.g., OpenAI's GPT-4) and simultaneously runs an emotion engine (e.g., IBM Watson Tone Analyzer). This allows the system to recognize emotions from the user's input.

[1419] 3. Emotion Recognition and Feedback Generation

[1420] The server uses an emotion engine to analyze the user's input and dialogue history to recognize the user's emotions. For example, it detects "confusion" or "excitement." It then generates personalized feedback using a generative AI model (such as OpenAI's GPT-4) based on the user's profile, past learning history, and the recognized emotions. Specifically, it generates feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?"

[1421] 4. Feedback and Questioning

[1422] The generated feedback is displayed to the user via the device. The user checks the feedback and inputs a response in response to the question. For example, the user might reply, "I don't understand the concept of acceleration." This response is then sent back to the server from the device, and the server generates additional feedback and encouraging messages to support the user. For example, the server might say, "Everyone has parts they don't understand. I'll explain them in detail from the beginning."

[1423] 5. Record and evaluate progress

[1424] The server records the user's progress in a database and continuously tracks it. For example, it might record information such as "What the user learned yesterday: understanding the equation of motion" and "What the user learned today: the concept of acceleration." It also periodically evaluates the progress and adjusts the next learning step or plan. For example, it generates suggestions such as "Let's learn the concept of force next."

[1425] 6. Maintaining motivation for learning and emotional care

[1426] The system generates words of encouragement and praise based on emotion recognition according to the user's progress and achievement of learning goals. For example, if the user is feeling anxious, it will provide specific encouragement such as, "Your efforts are great. You're almost there." These messages are displayed to the user via their device, improving their motivation and helping them maintain their enthusiasm for learning.

[1427] Specific examples

[1428] For example, suppose User B wants to learn about the equation of motion in physics. User B enters, "Teach me about the equation of motion." The server analyzes User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concept of the equation of motion. Which part do you find particularly difficult?" If User B replies, "I don't understand the concept of acceleration," the server provides a specific explanation with an encouraging message, "Everyone has parts they don't understand. I'll explain in detail from the beginning." Progress is also recorded, and further questions and emotional support are provided, such as, "Your understanding of acceleration has improved, which is great. What part would you like to learn next?"

[1429] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[1430] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1431] Step 1:

[1432] Enter and submit basic information

[1433] When a user uses the system for the first time, they enter basic information such as their name, age, areas of interest, and goals. This input is entered into the terminal and then sent to the server. Specifically, when the user enters information into the form and presses the "Submit" button, the data is sent from the terminal to the server. Examples of input are "Name: Tanaka Taro," "Age: 20 years old," "Area of ​​interest: Chemistry," and "Goal: I want to gain a deep understanding from the basics." The output is the basic information data that reaches the server.

[1434] Step 2:

[1435] Generate a profile

[1436] The server stores the received basic information in a database (e.g., MySQL). If the storage is successful, the server generates an initial profile. Specifically, it generates a unique profile ID for each user based on the content stored in the database, associates it with related information, and stores it. The input is the basic information sent from the device, and the output is the user profile data recorded in the database.

[1437] Step 3:

[1438] Start a study session and enter your questions

[1439] A user starts a learning session and inputs the area they want to learn about and a specific question. For example, they might input, "Tell me about the equation of motion." This input is sent from the device to the server. An example input might be, "Could you tell me which part you don't understand?" The output is the specific question data sent to the server.

[1440] Step 4:

[1441] Question analysis and emotion recognition

[1442] The received question or consultation content is analyzed using a natural language processing engine (e.g., a generative AI model). An emotion engine runs in parallel to recognize emotions from the user's input. Specifically, when identifying the emotion "confused," the server calculates an emotion evaluation score based on the relevant keywords and context. The input is the user's question, and the output is the analysis result and emotion recognition result.

[1443] Step 5:

[1444] Feedback Generation

[1445] The server generates personalized feedback based on the user's recognized emotions, profile information, and past learning history. It uses a generative AI model to construct appropriate answers and include encouraging messages. For example, feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?" is generated. The input is emotions and analysis results, and the output is the generated feedback.

[1446] Step 6:

[1447] Displaying feedback and user responses

[1448] The generated feedback is displayed to the user through the device. The user checks the feedback and inputs a response to the question. For example, the user might reply, "I don't understand the concept of acceleration." The device receives this response and sends it to the server. The input is the feedback from the server, and the output is the user's response data.

[1449] Step 7:

[1450] Generate additional feedback and encouraging messages

[1451] The server analyzes the user's response again and generates appropriate additional feedback. It also includes data from the emotion engine and adds an encouraging message. For example, it provides feedback such as, "Everyone has some things they don't understand. I'll explain them in detail from the beginning." The input is the user's response data, and the output is the generated additional feedback.

[1452] Step 8:

[1453] Recording and evaluating progress

[1454] The server records the user's progress in a database. For example, it may record information such as "Today's learning: Equation of motion" or "Level of understanding: Medium." Furthermore, it periodically evaluates the user's progress and adjusts the next learning step and plan. The input is the user's learning data, and the output is the progress evaluation results and a new learning plan proposal.

[1455] Step 9:

[1456] Maintaining motivation for learning and emotional care

[1457] The server generates encouraging and praising messages based on emotion recognition according to the user's achievement of learning goals and progress. For example, it generates messages such as "Your efforts are great. You're almost there." This increases the user's motivation. The input is learning progress and evaluation data, and the output is encouraging and praising messages.

[1458] In this way, the system personalizes the user's learning and provides emotional support through a series of steps.

[1459] (Application example 2)

[1460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1461] While conventional conversational AI systems can provide feedback to users' questions, they have difficulty providing appropriate responses and encouragement based on the user's emotional state. This can lead to users feeling stressed or losing motivation to learn without receiving appropriate support. Furthermore, in customer service at brick-and-mortar stores, there was a lack of support that took into account the customer's emotions, leading to concerns about a decline in customer satisfaction.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving basic information about the user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, and means for providing appropriate feedback and support messages in response to the user's emotions based on emotion recognition. This makes it possible to respond according to the user's emotional state, thereby reducing stress, increasing motivation to learn, and improving customer satisfaction.

[1463] "User" refers to a person who uses the system.

[1464] "Basic information" refers to the initial personal information a user provides to the system, such as name, age, areas of interest, and goals.

[1465] "Database" refers to the digital data structure that the system uses to store basic information about users, their learning history, and so on.

[1466] "Profile" refers to an individual learning model generated by analyzing a user's basic information, learning history, and progress data.

[1467] "Feedback" refers to the responses and advice the system provides to users in response to their questions or inquiries.

[1468] "Emotion recognition" refers to the process of analyzing a user's input and dialogue history to identify the user's emotional state at that time.

[1469] A "natural language processing model" refers to machine learning technology that understands users' questions and inquiries and generates appropriate feedback.

[1470] "Support messages" refer to messages generated based on emotion recognition to encourage and support users.

[1471] "Learning history" refers to the record of learning that a user has done through the system.

[1472] "Personalization" refers to responses that are customized to suit the characteristics and needs of each individual user.

[1473] "Progress assessment" refers to the process of periodically assessing a user's learning progress.

[1474] "Study Plan" means the specific steps or plan established to achieve a User's learning goals.

[1475] "Maintaining motivation" refers to supporting users to maintain their desire to learn.

[1476] "Customer" refers to a person who visits a physical store in search of a product or service.

[1477] This invention can be implemented as an interactive AI shopping assistant system that can be used in brick-and-mortar stores. A specific example of this system will now be described.

[1478] System configuration

[1479] The system consists of a smartphone, a server, an emotion engine, and a generative AI model. The smartphone is a device that the user can carry around and provides an interactive user interface. The server is the main processing unit that processes input from the user and generates the necessary feedback. The emotion engine is software that analyzes the user's emotional state from text input, and the generative AI model is a system that generates appropriate responses through natural language processing.

[1480] System Operation

[1481] 1. User registration and profile creation: The device (smartphone) receives the user's basic information (name, areas of interest, goals, etc.) and sends it to the server. The server stores this information in a database and creates a user profile.

[1482] 2. Question acceptance: A user uses a smartphone to input a question in a physical store, for example, "Where is this product?"

[1483] 3. Emotion recognition and feedback generation: The server receives the user's text input and analyzes their emotional state using an emotion engine. It then uses a generative AI model to generate appropriate feedback. For example, if the emotion engine detects that the user is confused, it can generate a response like, "I'm sorry, what are you looking for? I'll help you." It can also provide specific location information based on the user's input, such as, "Your item is at Aisle 5."

[1484] 4. Feedback presentation: The smartphone displays the feedback to the user, who can then head to the recommended location. If the user enters additional questions, the server again generates a response using the emotion engine and generative AI model.

[1485] Software and hardware used

[1486] Hardware: Smartphones, servers

[1487] Software: OpenAI generative AI models (e.g., GPT-3), EmotionEngine

[1488] The process involves the smartphone receiving the user's input and sending it to the server, which then analyzes the input and uses an emotion engine and generative AI model to generate appropriate feedback, which is then sent to the user's smartphone and displayed.

[1489] Specific examples

[1490] For example, if a user types "I don't know where this product is" into their smartphone, the emotion engine will detect "confusion," and the server will provide an encouraging message saying, "I'm sorry, what are you looking for? I'll help you," along with a specific recommended location, such as "That product is in Aisle 5."

[1491] Prompt Sentence Examples

[1492] An example of a prompt that a user might enter into the system is:

[1493] text

[1494] User Question: I don't know where this item is.

[1495] Provide appropriate assistance.

[1496] In this way, it is possible to increase user satisfaction by generating optimal feedback and support messages based on emotion recognition.

[1497] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1498] Step 1: The user uses their smartphone to enter basic information (such as their name, areas of interest, and goals). The device receives this input and sends it to the server.

[1499] Step 2: The server stores the received basic information in a database. It also generates a user profile and integrates it with other information stored in the database. The input is the user's basic information, and the output is the generated profile.

[1500] Step 3: The user uses their smartphone to input a specific question (e.g., "Where is this product?"). The device receives the user's input and sends it to the server.

[1501] Step 4: The server analyzes the received question using an emotion engine to detect the user's emotional state. The input is the user's text input, and the output is the detected emotional state. Specifically, the emotion engine analyzes the emotional nuances of the text and identifies emotions such as "confused" or "excited."

[1502] Step 5: The server uses the generative AI model to generate appropriate feedback for the user's question. The input is the user's question and the detected emotional state, and the output is personalized feedback. For example, if the user types, "I don't know where the product is," the generative AI model will generate the response, "The product is on Aisle 5."

[1503] Step 6: Based on the emotional state, the server further generates an appropriate support message for the user. The input is the detected emotional state, and the output is a support message. For example, if "confusion" is detected, the server generates a message saying "I'm sorry, what are you looking for? I'm here to help you."

[1504] Step 7: The server sends the generated feedback and support message to the terminal, which displays it to the user. The user confirms the feedback and enters further questions if necessary.

[1505] Step 8: The server continuously records the user's progress and updates their profile based on that data, making future interactions more personalized. The input is the user's feedback and emotion history, and the output is an updated profile.

[1506] These steps provide personalized feedback and support messages tailored to the user's emotional state.

[1507] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1508] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1509] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1510] [Fourth embodiment]

[1511] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1512] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1513] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1514] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1515] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1516] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1517] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1518] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1519] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1520] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1521] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1522] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1523] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1524] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support based on the user's characteristics and learning situation. This system enables users to study efficiently in line with their interests and goals, and develop essential problem-solving skills. Below, we will explain the system's program and its processing in natural language, including concrete examples.

[1525] System Overview

[1526] 1. User registration and profile creation

[1527] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[1528] Terminal: Receives user input and sends it to the server.

[1529] Server: Stores the received basic information in a database and generates an initial profile.

[1530] 2. Start the study session and take questions

[1531] User: Starts a learning session and enters questions and inquiries.

[1532] Terminal: Receives user input and sends it to the server.

[1533] Server: Analyzes the received question or inquiry and passes it to the appropriate dialogue model.

[1534] 3. Generate personalized feedback

[1535] Server: Based on the user's profile and learning history, the generative AI model analyzes the data and generates feedback and suggestions appropriate for the user.

[1536] Server: Generates a message containing a question along with feedback and sends it to the device.

[1537] 4. Feedback and Questioning

[1538] Terminal: Displays feedback and prompts provided by the server to the user.

[1539] User: Enter your understanding and opinion on the question and submit.

[1540] Terminal: Sends the user's answer to the server.

[1541] Server: Analyzes the user's response and provides additional feedback or explanation.

[1542] 5. Record and evaluate progress

[1543] Server: Records user progress in a database.

[1544] Server: Regularly evaluate progress and make next steps or adjustments to the plan.

[1545] Server: Based on the progress assessment, it provides feedback to the user and suggests any necessary details.

[1546] 6. Maintaining motivation to learn

[1547] Server: Generates and sends encouragement and praise to users based on their progress and achievement of learning goals.

[1548] On-device: Show encouragement and praise to users to increase their motivation.

[1549] Specific examples

[1550] For example, suppose User A wants to learn about vectors in mathematics. User A accesses the system and enters, "I'd like to learn about vector addition." The server uses User A's basic information and past learning history to provide a brief explanation of the basic concept of vectors and asks, "How much do you currently understand?" If User A then replies, "I understand the basics of addition, but I'm having trouble applying it," the server will provide specific application examples and additional materials. It will also record User A's progress and send an encouraging message saying, "You've achieved today's learning goal. Is there anything you'd like to advance to next?"

[1551] This system allows users to learn at their own pace and receives constant personalized feedback, resulting in high learning outcomes.

[1552] The processing flow will be explained below.

[1553] Step 1:

[1554] User: Accessing the system for the first time and opening the user registration page.

[1555] Terminal: Displays the registration page to the user and provides fields for entering the necessary information.

[1556] Step 2:

[1557] User: Enter basic information such as name, age, areas of interest, and learning goals, then press the submit button.

[1558] Terminal: Sends the entered information to the server.

[1559] Step 3:

[1560] Server: Stores the received basic information in a database.

[1561] Server: Generates an initial profile based on basic information.

[1562] Step 4:

[1563] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[1564] Terminal: Sends user input to the server.

[1565] Step 5:

[1566] Server: Analyzes the received questions and inquiries.

[1567] Server: Reads the user's profile and past learning history from the database and understands the user's characteristics.

[1568] Step 6:

[1569] Server: Calls the generative AI model based on the analysis results and generates personalized feedback.

[1570] Server: Generates a message containing a question to get the user to think along with the generated feedback.

[1571] Step 7:

[1572] Terminal: Displays feedback and prompts from the server to the user.

[1573] User: Check the feedback, enter their understanding and opinion of the question, and press the submit button.

[1574] Step 8:

[1575] Terminal: Sends the user's answer to the server.

[1576] Server: Parses the received answer and generates a message to provide additional feedback or clarification.

[1577] Step 9:

[1578] Server: Records user progress in a database and tracks progress continuously.

[1579] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[1580] Step 10:

[1581] Terminal: displays generated messages to the user.

[1582] Server: Generates encouragement and praise based on the user's achievement of learning goals and progress.

[1583] Server: Generates messages containing encouragement and praise and sends them to the device.

[1584] Step 11:

[1585] On-device: Display encouragement and praise to the user to increase their motivation.

[1586] User: Enter a thank you message or next learning goal if desired, and click the send button.

[1587] These are the specific processing steps of this interactive AI mentor system, which allows users to receive individually customized learning support and progress effectively at their own pace.

[1588] Example 1

[1589] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] Conventional learning support systems lacked personalized support tailored to each user's characteristics and learning progress, making efficient learning difficult. Furthermore, there were limited ways to continuously maintain users' motivation to learn, which meant that learning effectiveness was not maximized. Furthermore, the quality and timing of feedback often did not meet users' needs, which reduced learning efficiency.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1592] In this invention, the server includes means for receiving basic information about a user and saving it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and asking additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing optimal feedback and suggestions to the user using a generative AI model, and means for generating prompt sentences, inputting the prompt sentences into the generative AI model, and providing the generated output to the user. This enables personalized support based on the user's characteristics and learning situation, maintaining motivation to learn, and providing effective feedback.

[1593] "User" refers to an individual who uses this system to carry out learning activities or ask questions.

[1594] "Basic information" refers to information entered during initial registration, such as the user's name, age, areas of interest, and learning goals.

[1595] "Database" refers to a digital information storage location for storing basic information, learning history, and progress data of users.

[1596] A "profile" refers to a collection of information specific to an individual user, generated based on the user's basic information, learning history, and progress.

[1597] "Feedback" refers to answers, suggestions, comments, etc. provided in response to users' questions or inquiries.

[1598] "Generative AI model" refers to an artificial intelligence model that generates optimal feedback and suggestions for users.

[1599] "Natural language processing model" refers to a software model used to analyze user input and generate an appropriate response.

[1600] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1601] "Progress" refers to the results of evaluating a user's learning situation and level of achievement.

[1602] "Study plan" refers to a plan or program designed to help users study efficiently.

[1603] "Encouragement and praise" refers to messages and comments provided to motivate users to learn.

[1604] "Continuously recording" means that the user's learning progress is periodically saved to a database.

[1605] This invention is an interactive AI mentor system based on a generative AI model that provides personalized support according to the user's characteristics and learning situation. The program processing of this system is explained in detail below.

[1606] System configuration

[1607] This system mainly consists of three elements: the server, the terminal, and the user.

[1608] server

[1609] The server plays a central role in storing and analyzing data and generating feedback using generative AI models. The hardware used is a cloud-based server, and the corresponding software includes the following:

[1610] Database: MySQL

[1611] Natural language processing library: spaCy

[1612] Generative AI model: OpenAI's GPT-4

[1613] Programming language: Python

[1614] The server does the following:

[1615] 1. Store the basic information received from the user in a MySQL database.

[1616] 1. Use spaCy to parse user input and extract keywords.

[1617] 1. Use GPT-4 to generate feedback and suggestions.

[1618] 1. Evaluate your learning progress and adjust your next steps and learning plan.

[1619] Terminal

[1620] The terminal is responsible for providing an interface with the user, sending user input to the server, and displaying feedback from the server. The terminal's functions include:

[1621] 1. Receive user input in real time and send it to the server.

[1622] 1. Display feedback and questions received from the server to the user.

[1623] user

[1624] Users use the system to carry out learning activities and receive feedback to progress their learning. Users perform the following actions:

[1625] 1. Enter your basic information and learning goals.

[1626] 1. Enter any questions or concerns you may have while studying.

[1627] 1. Type your response to the feedback or questions provided.

[1628] Specific actions

[1629] The operation of this system will be explained using a specific example.

[1630] User Registration

[1631] When a user first accesses the system, they enter basic information such as their name, age, areas of interest, and learning goals. The device receives the information and sends it to the server in real time. The server stores the information in a MySQL database and creates an initial profile.

[1632] Learning sessions and questions

[1633] If a user wants to learn about "vector addition," they type that into their device. The device receives the question and sends it to the server, which uses spaCy to parse it and GPT-4 to generate appropriate feedback.

[1634] Providing feedback and asking questions

[1635] The server generates feedback and sends it to the device, which then displays it to the user. For example, if you type "Teach me about vector addition," the server will explain the basic concept of vectors and ask, "How well do you understand it now?"

[1636] Giving and receiving feedback

[1637] If the user responds, "I understand the basics of addition, but I'm having trouble applying it," the server will again display specific application examples and additional materials. By repeating this process, the server can provide optimal learning support to the user.

[1638] Prompt Sentence Examples

[1639] For example, the following prompt sentence is input to the generative AI model:

[1640] "If you're a 15-year-old student who's good at math, explain the basic concept of vector addition and ask if they'd like to see some applications. Also, ask follow-up questions to check their understanding."

[1641] By providing the user with feedback messages generated based on these prompts, more effective learning support is possible.

[1642] summary

[1643] This interactive AI mentoring system allows users to receive personalized support based on their individual characteristics and learning progress. The server provides feedback using a generative AI model, enabling efficient learning and maintaining motivation.

[1644] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1645] Step 1:

[1646] User registration and profile creation

[1647] 1.1. Collecting User Input

[1648] Users enter basic information such as name, age, interests, and learning goals.

[1649] Input: A user enters information into a web form (e.g., "Taro Tanaka, 17 years old, mathematics, university exams").

[1650] Output: The basic information entered is sent to the terminal.

[1651] What happens: A user fills out a form and clicks the submit button.

[1652] 1.2. Transmission of User Information

[1653] The device receives the user's input and transmits it to the server in real time.

[1654] Input: Basic information submitted via the web form.

[1655] Output: Basic information is sent to the server.

[1656] Specific operation: The front-end script takes the input information and sends it to the server via the API.

[1657] 1.3. Server-based information storage and profile generation

[1658] The server stores the received basic information in a database and generates an initial profile.

[1659] Input: User basic information.

[1660] Output: The initial profile that will be saved in the database.

[1661] What it does: Executes an insert query in the MySQL database to save the information. Generates a profile ID.

[1662] Step 2:

[1663] Start a study session and take questions

[1664] 2.1. User Question Input

[1665] A user begins a learning session by typing in a question: "Teach me about vector addition."

[1666] Input: The user types a question into the terminal.

[1667] Output: The entered question is received by the terminal.

[1668] What it does: A user starts a learning session and uses a form field to enter a question.

[1669] 2.2. Submitting a Question

[1670] The device receives the user's question and sends it to the server.

[1671] Input: The question entered by the user.

[1672] Output: The question is sent to the server.

[1673] What happens: JavaScript or other front-end scripts submit questions to the server via an API.

[1674] 2.3. Query analysis by the server

[1675] The server parses the question and passes it to the appropriate dialogue model.

[1676] Input: Question sent from terminal.

[1677] Output: Feedback generated based on analysis results.

[1678] What it does: Uses spaCy to extract keywords from the question and prepares them for passing to the AI ​​model.

[1679] Step 3:

[1680] Generate personalized feedback

[1681] 3.1. Obtaining user information and learning history

[1682] The server retrieves the user's profile information and learning history from the database.

[1683] Input: User profile ID.

[1684] Output: Profile information and learning history.

[1685] Specific behavior: Executes a SELECT query based on the user ID in the MySQL database to retrieve information.

[1686] 3.2. Analysis and feedback generation using AI models

[1687] The server uses a generative AI model such as GPT-4 to generate appropriate feedback and suggestions.

[1688] Input: User information, learning history, question keywords.

[1689] Output: The generated feedback.

[1690] Specific operation: Generate a prompt sentence, input it into GPT-4, and generate feedback.

[1691] 3.3. Sending Feedback

[1692] The server transmits the generated feedback to the terminal.

[1693] Input: Generated feedback.

[1694] Output: Feedback is sent to the device.

[1695] Specific behavior: Sends feedback to the device using an API call.

[1696] Step 4:

[1697] Feedback and questions

[1698] 4.1. Display of Feedback

[1699] The device displays the feedback it receives from the server to the user.

[1700] Input: Feedback received from the server.

[1701] Output: Feedback that is displayed on the user's screen.

[1702] Specific behavior: Display a feedback message on the screen.

[1703] 4.2. User response input

[1704] Users enter responses to the feedback and questions provided.

[1705] Input: The user's response.

[1706] Output: The reply is received at the terminal.

[1707] Specific behavior: The user types a response and presses the send button.

[1708] 4.3. Sending a Reply

[1709] The device receives the user's response and sends it to the server.

[1710] Input: The user's response.

[1711] Output: The reply is sent to the server.

[1712] What happens: JavaScript or other front-end scripts send a response to the server via an API.

[1713] 4.4. Server Response Analysis and Additional Feedback Generation

[1714] The server analyzes the user's response and provides additional feedback or explanation.

[1715] Input: The user's response.

[1716] Output: Additional feedback.

[1717] What it does: We again use spaCy to parse the response and GPT-4 to generate additional feedback.

[1718] Step 5:

[1719] Recording and evaluating progress

[1720] 5.1. Recording progress

[1721] The server records the user's learning progress in a database.

[1722] Input: User's learning progress information.

[1723] Output: Progress information is saved to the database.

[1724] What it does: Executes an insert query into a MySQL database to store progress information.

[1725] 5.2. Periodic evaluation

[1726] The server periodically evaluates the user's progress and determines the next learning step.

[1727] Input: User's learning progress information.

[1728] Output: Progress assessment results.

[1729] Specific operation: Evaluation is performed using scikit-learn based on learning progress information.

[1730] 5.3. Adjusting the plan and proposing next steps

[1731] Based on the evaluation results, the server will suggest the user the best next learning steps and plan.

[1732] Input: Progress assessment results.

[1733] Output: Suggested next learning step.

[1734] Specific operation: Determine the next step based on the evaluation results, generate a prompt for GPT-4, and create a proposed sentence.

[1735] Step 6:

[1736] Maintaining motivation to learn

[1737] 6.1. Generating encouragement and praise

[1738] The server generates encouragement and praise based on the user's achievement of learning goals and progress.

[1739] Input: User's learning progress information.

[1740] Output: Encouragement and praise.

[1741] Specific behavior: Using GPT-4 to generate encouragement and praise.

[1742] 6.2. Showing encouragement and praise

[1743] The device displays encouragement and praise to the user, increasing motivation to learn.

[1744] Input: Encouragement or praise received from the server.

[1745] Output: Encouragement or praise that appears on the user's screen.

[1746] Specific behavior: Displaying encouragement and praise on the screen.

[1747] (Application example 1)

[1748] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1749] Conventional learning support systems lack the ability to provide personalized feedback based on user characteristics and progress, making it difficult to achieve efficient learning. Furthermore, they lack interactive learning support based on specific digital content, making it difficult for users to deepen their understanding of the content they are viewing. To address these issues, a more advanced interactive system based on generative AI models is needed.

[1750] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1751] In this invention, the server includes means for receiving basic information of a user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on each individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, means for providing the generated feedback to the user and prompting the user with additional questions to encourage the user's awareness and discovery, means for continuously recording the user's learning progress and providing progress-based evaluations and necessary adjustments to the learning plan, means for generating encouragement and praise according to the user's achievement of learning goals and progress to help maintain motivation, means for providing interactive learning support based on specific digital content, means for the user to input questions and inquiries about the content being viewed, and means for generating personalized feedback about the content being viewed using a generative AI model. This allows the user to deepen their understanding of the content being viewed and to study more efficiently.

[1752] "Basic User Information" refers to personal data such as name, age, interests, and goals.

[1753] A "database" is a system for systematically storing and managing information.

[1754] "Study history" is a record of the content and progress of the user's studies to date.

[1755] A "profile" is individual information generated based on a user's characteristics and learning history.

[1756] "Personalized feedback" refers to individual answers and suggestions provided based on the user's characteristics and learning situation.

[1757] "Questions" are questions or confirmation messages sent to users.

[1758] "Realizations and discoveries" are new things that users understand or discover through learning.

[1759] "Progress" refers to the progress of learning or work.

[1760] "Evaluation" refers to judgments and assessments made based on progress and learning content.

[1761] "Adjusting your study plan" refers to modifying and optimizing your study plan based on your progress and feedback.

[1762] "Encouragement and praise" refers to positive messages provided to improve user motivation.

[1763] "Specific digital content" refers to specific video, audio, and text data intended for educational or informational purposes.

[1764] "Interactive learning support" refers to learning support that is provided through two-way interaction.

[1765] A "generative AI model" is a model of artificial intelligence used to conduct dialogue and analyze data.

[1766] "Currently viewed content" refers to digital content that a user is currently viewing.

[1767] This invention applies an interactive AI mentor system based on a generative AI model to a content distribution service. This system allows users to receive interactive learning support while viewing specific digital content. The following describes the program for this system and how to execute it.

[1768] Hardware and software used

[1769] Hardware: Smartphones, tablets

[1770] Software: Generative AI models (e.g., OpenAI's GPT-4), database management systems (e.g., MySQL), web servers (e.g., Apache)

[1771] Program Overview

[1772] The server executes a program that includes the following means:

[1773] 1. A means of receiving basic user information and storing it in a database

[1774] Users use their smartphone or tablet to enter basic information such as their name, age, areas of interest, and goals.

[1775] The terminal transmits the input information to the server.

[1776] The server stores this basic information in a database.

[1777] 2. A means to analyze a user's learning history and progress and generate a profile based on the individual user.

[1778] The server generates an individual profile based on the user's basic information and past learning history stored in a database.

[1779] 3. A means of generating personalized feedback in response to user questions or inquiries based on the generated profile.

[1780] The user inputs a question about the digital content being viewed into the terminal and transmits it to the server.

[1781] The server analyzes the profile and questions and generates personalized feedback using a generative AI model (e.g., GPT-4).

[1782] 4. A means to provide generated feedback to users and prompt them with follow-up questions to encourage their awareness and discovery.

[1783] The feedback generated by the server is sent to the terminal and displayed to the user.

[1784] The server then asks additional questions to help the user gain a deeper understanding.

[1785] 5. A means to continually track users' learning progress and provide progress-based assessments and necessary adjustments to their learning plans.

[1786] The server records the user's responses and progress in a database and evaluates them periodically.

[1787] Based on progress, we will adjust your next learning steps and plan.

[1788] 6. A means to generate encouragement and praise according to the user's achievement of learning goals and progress, and to help maintain motivation

[1789] The server generates words of encouragement and praise based on the user's progress in achieving the learning goal and sends them to the terminal.

[1790] 7. Means of providing interactive learning support based on specific digital content

[1791] The server provides learning support corresponding to the digital content that the user is currently viewing.

[1792] 8. A means for users to input questions or inquiries about the content they are viewing

[1793] The terminal provides an interface for users to input questions or inquiries about the content they are viewing.

[1794] Example prompt sentences

[1795] Below are some examples of prompts that can be passed to a generative AI model based on a user question:

[1796] Please answer the following question about the science documentary you are watching: "I would like to know more about how this experiment was done."

[1797] Based on the user's profile, they are interested in science and would like detailed instructions for experiments. Please explain them in an easy-to-understand manner for beginners.

[1798] The feedback generated based on this prompt sentence is provided to the user, realizing interactive learning support.

[1799] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1800] Step 1:

[1801] Users use their smartphones or tablets to enter basic information such as their name, age, areas of interest, and goals. This input form is provided by the device. The entered basic information is temporarily stored on the device.

[1802] Step 2:

[1803] The device sends the temporarily stored basic information to the server, which then stores the received basic information in a database. The input data is user information (name, age, areas of interest, and goals), and the output is the status of saving to the database.

[1804] Step 3:

[1805] The server retrieves the user's basic information and past learning history from the database and generates an individual profile based on that information. This profile includes the user's progress and areas of interest. The generated profile is then saved back into the database. The input is the user information and learning history data, and the output is the generated profile.

[1806] Step 4:

[1807] The user inputs a question about the digital content being viewed via the terminal. The question is sent from the terminal to the server. The input is the user's question, and the output is the status of transmission to the server.

[1808] Step 5:

[1809] The server analyzes the user's profile and question, sends prompts to a generative AI model (e.g., GPT-4) and generates personalized feedback. In this process, an appropriate response is generated based on the profile and question. The input is the prompt, and the output is the generated feedback.

[1810] Step 6:

[1811] The generated feedback is sent from the server to the terminal and displayed to the user. The user receives the feedback and deepens their understanding. The input is the generated feedback, and the output is the user's perception.

[1812] Step 7:

[1813] The server also generates questions for the user to further understand and sends them to the terminal. The questions are displayed to the user, and the user inputs answers accordingly. The input is the user's answer, and the output is the transmission status to the server.

[1814] Step 8:

[1815] The server records the user's answers and learning progress in a database and periodically evaluates the progress. The progress data is used to determine the next learning step and adjust the plan. The input is the user's answers and progress data, and the output is suggested adjustments to the learning plan.

[1816] Step 9:

[1817] Based on the user's achievement of learning goals and progress, the server generates encouragement and praise and sends them to the device. The user receives these and stays motivated. The input is progress data, and the output is encouragement and praise.

[1818] Step 10:

[1819] The server provides interactive learning support based on specific digital content. Users use their devices to input questions or inquiries about the content they are viewing. The server responds to these questions and generates appropriate feedback. The input is the user's question, and the output is the generated feedback.

[1820] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1821] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize the user's learning and provide emotional support. Below, we will explain the program and processing of this system in detail, using concrete examples.

[1822] System Overview

[1823] 1. User registration and profile creation

[1824] Users: When using the system for the first time, they enter basic information such as their name, age, areas of interest, and goals.

[1825] Terminal: Receives user input and sends it to the server.

[1826] Server: Stores the received basic information in a database and generates an initial profile.

[1827] 2. Start the study session and take questions

[1828] User: Starts a learning session and enters the area they want to learn about and the question they have.

[1829] Terminal: Sends user input to the server.

[1830] Server: Analyzes the received questions and inquiries and simultaneously operates the emotion engine.

[1831] 3. Emotion Recognition and Feedback Generation

[1832] Server: The emotion engine analyzes the user's input and dialogue history to recognize the user's emotions. For example, the emotion engine detects emotions such as "excitement," "disappointment," and "anxiety."

[1833] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[1834] Server: Generates feedback and prompts or encouraging messages according to the user's emotions.

[1835] 4. Feedback and Questioning

[1836] Terminal: Displays feedback and prompts provided by the server to the user.

[1837] User: Review the feedback and enter answers based on the questions and emotions.

[1838] Terminal: Sends the user's answer to the server.

[1839] Server: Re-analyzes the user's response and provides appropriate additional feedback or explanation.

[1840] 5. Record and evaluate progress

[1841] Server: Records user progress in a database and keeps track of progress.

[1842] Server: Periodically evaluates progress, adjusts next learning steps and plans, and generates messages with suggestions.

[1843] Server: Provides special suggestions and support based on the user's emotions.

[1844] 6. Maintaining motivation for learning and emotional care

[1845] Server: Generates encouragement and praise based on emotion recognition according to the user's achievement of learning goals and progress.

[1846] Server: For example, if the user is feeling anxious, provide specific encouragement such as, "You're doing great! You're almost there."

[1847] On-device: Show encouragement and praise to users to increase their motivation.

[1848] User: Optionally enter a thank you message or next learning goal and submit.

[1849] Specific examples

[1850] For example, let's say User B wants to learn about the equations of motion in physics. User B inputs, "Please teach me about the equations of motion." The server performs an analysis based on User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concepts of the equations of motion. Which part do you find particularly difficult?"

[1851] If User B responds, "I don't understand the concept of acceleration," the server uses the emotion engine's recognition to provide a specific explanation, including an encouraging message such as, "Everyone has things they don't understand. I'll explain them in detail from the beginning." The server also records the progress and provides further emotional support, asking, "Your understanding of acceleration is great progress. What part would you like to learn next?"

[1852] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[1853] The processing flow will be explained below.

[1854] Step 1:

[1855] User: Accessing the system for the first time and opening the user registration page.

[1856] Device: Presents the user with a registration page, providing fields for input such as name, age, areas of interest, and learning goals.

[1857] Step 2:

[1858] User: Enter basic information and press the send button.

[1859] Terminal: Validates the entered information and sends it to the server.

[1860] Step 3:

[1861] Server: Stores the received basic information in a database and generates an initial profile.

[1862] Step 4:

[1863] User: Starts a study session, enters the topic they want to learn about, their question, and hits submit.

[1864] Terminal: Sends user input to the server.

[1865] Step 5:

[1866] Server: Analyzes the received questions and inquiries. At the same time, it starts the emotion engine and analyzes the user's input.

[1867] Step 6:

[1868] Server: Uses an emotion engine to recognize the user's emotions (e.g., "confused," "excited").

[1869] Step 7:

[1870] Server: Generates personalized feedback using a generative AI model based on the user's profile, past learning history, and recognized emotions.

[1871] Step 8:

[1872] Server: Generates messages including the generated feedback, questions, and even encouraging messages according to the user's emotions, and sends them to the device.

[1873] Step 9:

[1874] Terminal: Displays server-provided feedback, prompts, and encouraging messages to the user.

[1875] Step 10:

[1876] User: Check the feedback, enter an answer based on the question or emotion, and press the send button.

[1877] Step 11:

[1878] Terminal: Sends the user's answer to the server.

[1879] Step 12:

[1880] Server: Again analyzes the user's response and generates appropriate additional feedback and explanations, including special care and advice if needed, based on the emotions recognized by the emotion engine.

[1881] Step 13:

[1882] Server: Records user progress in a database and tracks progress continuously.

[1883] Step 14:

[1884] Server: Periodically evaluates progress, adjusts the learning plan, and generates messages with suggestions.

[1885] Step 15:

[1886] Terminal: displays generated messages to the user.

[1887] Step 16:

[1888] Server: Based on the user's progress and achievement of learning goals, the server performs emotion recognition again and generates words of encouragement or praise. For example, "Your efforts are great. What would you like to learn next?"

[1889] Step 17:

[1890] On-device: Display encouragement and praise to the user to increase their motivation.

[1891] Step 18:

[1892] User: Enter a thank you message or next learning goal and submit.

[1893] By combining this conversational AI mentor system with an emotion engine, users can receive individually customized learning support and emotional care. As a concrete example, if a user is confused about the equations of motion in physics, the system will ask, "I'll start by explaining the basic concepts of the equations of motion. Which parts do you find particularly difficult?" After analyzing the user's confusion, the system will provide an encouraging message saying, "Everyone has parts they don't understand. I'll explain them in detail from the beginning." This will achieve high learning effectiveness and satisfaction.

[1894] Example 2

[1895] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1896] Conventional learning support systems often provide uniform feedback without considering the user's emotions, making it difficult to maintain the user's motivation and willingness to learn. Another issue is that feedback is not sufficiently personalized for each user, resulting in reduced learning efficiency. To address these issues, the present invention aims to provide an interactive AI mentor system combined with an emotion recognition engine, thereby realizing personalized learning and emotional care.

[1897] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1898] In this invention, the server includes: means for receiving basic information about a user and saving it in a database; means for analyzing the user's learning history and progress and generating a profile based on the individual user; means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile; means for providing the generated feedback to the user and prompting the user with additional questions to encourage awareness and discovery; means including an emotion engine that recognizes the user's emotions and generating encouraging and support messages according to the user's emotions; means for continuously recording the user's learning progress and providing evaluations based on the progress and adjustments to the learning plan as needed; and means for generating encouraging and praise words according to the user's achievement of learning goals and progress to help maintain motivation. This makes it possible to individually personalize the user's learning and provide emotional support, thereby improving learning effectiveness and satisfaction.

[1899] "User" refers to an individual who uses this system to study.

[1900] "Basic Information" refers to information provided at the time of initial registration, such as the user's name, age, areas of interest, and learning goals.

[1901] "Database" refers to a system for recording and managing users' basic information, learning history, and progress information.

[1902] "Profile" refers to a set of learning information for each user that is generated based on the user's basic information and learning history.

[1903] "Feedback" refers to the response generated by the system in response to a user's question or inquiry.

[1904] A "natural language processing model" refers to an algorithm that uses a generative AI model to generate appropriate answers to users' questions and inquiries.

[1905] An "emotion engine" refers to a system that analyzes the user's input and dialogue history to recognize the user's emotions.

[1906] "Support messages" refer to messages containing encouragement or specific guidance that are generated by the system based on the user's emotions.

[1907] "Study progress" refers to information indicating how much content a user has studied and the level of understanding of the content.

[1908] "Study plan" refers to the combination of study schedules and tasks suggested by the system to help users study effectively.

[1909] "Maintaining motivation" refers to activities that support users so that they can maintain their motivation to learn during the learning process.

[1910] This invention combines an emotion engine with a conversational AI mentor system based on a generative AI model to further personalize learning and provide emotional support. This system consists of a terminal used by the user to study, a server, and a network for communication between them.

[1911] 1. User registration and profile creation

[1912] First, the user enters basic information from their device, such as their name, age, areas of interest, and learning goals. This information is then sent from the device to the server, which then stores the information in a database (e.g., MySQL) and creates an initial profile.

[1913] 2. Start the study session and take questions

[1914] The user starts a learning session by inputting the subject they want to learn about and a question they have. For example, they can input a specific question such as, "Please teach me about the equation of motion." This input is sent to the server via the device. The server then analyzes the received question or inquiry using a natural language processing engine (e.g., OpenAI's GPT-4) and simultaneously runs an emotion engine (e.g., IBM Watson Tone Analyzer). This allows the system to recognize emotions from the user's input.

[1915] 3. Emotion Recognition and Feedback Generation

[1916] The server uses an emotion engine to analyze the user's input and dialogue history to recognize the user's emotions. For example, it detects "confusion" or "excitement." It then generates personalized feedback using a generative AI model (such as OpenAI's GPT-4) based on the user's profile, past learning history, and the recognized emotions. Specifically, it generates feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?"

[1917] 4. Feedback and Questioning

[1918] The generated feedback is displayed to the user via the device. The user checks the feedback and inputs a response in response to the question. For example, the user might reply, "I don't understand the concept of acceleration." This response is then sent back to the server from the device, and the server generates additional feedback and encouraging messages to support the user. For example, the server might say, "Everyone has parts they don't understand. I'll explain them in detail from the beginning."

[1919] 5. Record and evaluate progress

[1920] The server records the user's progress in a database and continuously tracks it. For example, it might record information such as "What the user learned yesterday: understanding the equation of motion" and "What the user learned today: the concept of acceleration." It also periodically evaluates the progress and adjusts the next learning step or plan. For example, it generates suggestions such as "Let's learn the concept of force next."

[1921] 6. Maintaining motivation for learning and emotional care

[1922] The system generates words of encouragement and praise based on emotion recognition according to the user's progress and achievement of learning goals. For example, if the user is feeling anxious, it will provide specific encouragement such as, "Your efforts are great. You're almost there." These messages are displayed to the user via their device, improving their motivation and helping them maintain their enthusiasm for learning.

[1923] Specific examples

[1924] For example, suppose User B wants to learn about the equation of motion in physics. User B enters, "Teach me about the equation of motion." The server analyzes User B's basic information, past learning history, and the input. The emotion engine recognizes that User B is feeling "confused." Based on this, the server generates feedback such as, "I'll start by explaining the basic concept of the equation of motion. Which part do you find particularly difficult?" If User B replies, "I don't understand the concept of acceleration," the server provides a specific explanation with an encouraging message, "Everyone has parts they don't understand. I'll explain in detail from the beginning." Progress is also recorded, and further questions and emotional support are provided, such as, "Your understanding of acceleration has improved, which is great. What part would you like to learn next?"

[1925] In this way, the present invention uses a system that combines an emotion engine to personalize the user's learning and provide emotional support, thereby achieving high learning effectiveness and satisfaction.

[1926] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1927] Step 1:

[1928] Enter and submit basic information

[1929] When a user uses the system for the first time, they enter basic information such as their name, age, areas of interest, and goals. This input is entered into the terminal and then sent to the server. Specifically, when the user enters information into the form and presses the "Submit" button, the data is sent from the terminal to the server. Examples of input are "Name: Tanaka Taro," "Age: 20 years old," "Area of ​​interest: Chemistry," and "Goal: I want to gain a deep understanding from the basics." The output is the basic information data that reaches the server.

[1930] Step 2:

[1931] Generate a profile

[1932] The server stores the received basic information in a database (e.g., MySQL). If the storage is successful, the server generates an initial profile. Specifically, it generates a unique profile ID for each user based on the content stored in the database, associates it with related information, and stores it. The input is the basic information sent from the device, and the output is the user profile data recorded in the database.

[1933] Step 3:

[1934] Start a study session and enter your questions

[1935] A user starts a learning session and inputs the area they want to learn about and a specific question. For example, they might input, "Tell me about the equation of motion." This input is sent from the device to the server. An example input might be, "Could you tell me which part you don't understand?" The output is the specific question data sent to the server.

[1936] Step 4:

[1937] Question analysis and emotion recognition

[1938] The received question or consultation content is analyzed using a natural language processing engine (e.g., a generative AI model). An emotion engine runs in parallel to recognize emotions from the user's input. Specifically, when identifying the emotion "confused," the server calculates an emotion evaluation score based on the relevant keywords and context. The input is the user's question, and the output is the analysis result and emotion recognition result.

[1939] Step 5:

[1940] Feedback Generation

[1941] The server generates personalized feedback based on the user's recognized emotions, profile information, and past learning history. It uses a generative AI model to construct appropriate answers and include encouraging messages. For example, feedback such as, "Let's start by explaining the basic concepts of the equation of motion. Which part do you find particularly difficult?" is generated. The input is emotions and analysis results, and the output is the generated feedback.

[1942] Step 6:

[1943] Displaying feedback and user responses

[1944] The generated feedback is displayed to the user through the device. The user checks the feedback and inputs a response to the question. For example, the user might reply, "I don't understand the concept of acceleration." The device receives this response and sends it to the server. The input is the feedback from the server, and the output is the user's response data.

[1945] Step 7:

[1946] Generate additional feedback and encouraging messages

[1947] The server analyzes the user's response again and generates appropriate additional feedback. It also includes data from the emotion engine and adds an encouraging message. For example, it provides feedback such as, "Everyone has some things they don't understand. I'll explain them in detail from the beginning." The input is the user's response data, and the output is the generated additional feedback.

[1948] Step 8:

[1949] Recording and evaluating progress

[1950] The server records the user's progress in a database. For example, it may record information such as "Today's learning: Equation of motion" or "Level of understanding: Medium." Furthermore, it periodically evaluates the user's progress and adjusts the next learning step and plan. The input is the user's learning data, and the output is the progress evaluation results and a new learning plan proposal.

[1951] Step 9:

[1952] Maintaining motivation for learning and emotional care

[1953] The server generates encouraging and praising messages based on emotion recognition according to the user's achievement of learning goals and progress. For example, it generates messages such as "Your efforts are great. You're almost there." This increases the user's motivation. The input is learning progress and evaluation data, and the output is encouraging and praising messages.

[1954] In this way, the system personalizes the user's learning and provides emotional support through a series of steps.

[1955] (Application example 2)

[1956] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1957] While conventional conversational AI systems can provide feedback to users' questions, they have difficulty providing appropriate responses and encouragement based on the user's emotional state. This can lead to users feeling stressed or losing motivation to learn without receiving appropriate support. Furthermore, in customer service at brick-and-mortar stores, there was a lack of support that took into account the customer's emotions, leading to concerns about a decline in customer satisfaction.

[1958] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving basic information about the user and storing it in a database, means for analyzing the user's learning history and progress and generating a profile based on the individual user, means for generating personalized feedback in response to the user's questions and inquiries based on the generated profile, and means for providing appropriate feedback and support messages in response to the user's emotions based on emotion recognition. This makes it possible to respond according to the user's emotional state, thereby reducing stress, increasing motivation to learn, and improving customer satisfaction.

[1959] "User" refers to a person who uses the system.

[1960] "Basic information" refers to the initial personal information a user provides to the system, such as name, age, areas of interest, and goals.

[1961] "Database" refers to the digital data structure that the system uses to store basic information about users, their learning history, and so on.

[1962] "Profile" refers to an individual learning model generated by analyzing a user's basic information, learning history, and progress data.

[1963] "Feedback" refers to the responses and advice the system provides to users in response to their questions or inquiries.

[1964] "Emotion recognition" refers to the process of analyzing a user's input and dialogue history to identify the user's emotional state at that time.

[1965] A "natural language processing model" refers to machine learning technology that understands users' questions and inquiries and generates appropriate feedback.

[1966] "Support messages" refer to messages generated based on emotion recognition to encourage and support users.

[1967] "Learning history" refers to the record of learning that a user has done through the system.

[1968] "Personalization" refers to responses that are customized to suit the characteristics and needs of each individual user.

[1969] "Progress assessment" refers to the process of periodically assessing a user's learning progress.

[1970] "Study Plan" means the specific steps or plan established to achieve a User's learning goals.

[1971] "Maintaining motivation" refers to supporting users to maintain their desire to learn.

[1972] "Customer" refers to a person who visits a physical store in search of a product or service.

[1973] This invention can be implemented as an interactive AI shopping assistant system that can be used in brick-and-mortar stores. A specific example of this system will now be described.

[1974] System configuration

[1975] The system consists of a smartphone, a server, an emotion engine, and a generative AI model. The smartphone is a device that the user can carry around and provides an interactive user interface. The server is the main processing unit that processes input from the user and generates the necessary feedback. The emotion engine is software that analyzes the user's emotional state from text input, and the generative AI model is a system that generates appropriate responses through natural language processing.

[1976] System Operation

[1977] 1. User registration and profile creation: The device (smartphone) receives the user's basic information (name, areas of interest, goals, etc.) and sends it to the server. The server stores this information in a database and creates a user profile.

[1978] 2. Question acceptance: A user uses a smartphone to input a question in a physical store, for example, "Where is this product?"

[1979] 3. Emotion recognition and feedback generation: The server receives the user's text input and analyzes their emotional state using an emotion engine. It then uses a generative AI model to generate appropriate feedback. For example, if the emotion engine detects that the user is confused, it can generate a response like, "I'm sorry, what are you looking for? I'll help you." It can also provide specific location information based on the user's input, such as, "Your item is at Aisle 5."

[1980] 4. Feedback presentation: The smartphone displays the feedback to the user, who can then head to the recommended location. If the user enters additional questions, the server again generates a response using the emotion engine and generative AI model.

[1981] Software and hardware used

[1982] Hardware: Smartphones, servers

[1983] Software: OpenAI generative AI models (e.g., GPT-3), EmotionEngine

[1984] The process involves the smartphone receiving the user's input and sending it to the server, which then analyzes the input and uses an emotion engine and generative AI model to generate appropriate feedback, which is then sent to the user's smartphone and displayed.

[1985] Specific examples

[1986] For example, if a user types "I don't know where this product is" into their smartphone, the emotion engine will detect "confusion," and the server will provide an encouraging message saying, "I'm sorry, what are you looking for? I'll help you," along with a specific recommended location, such as "That product is in Aisle 5."

[1987] Prompt Sentence Examples

[1988] An example of a prompt that a user might enter into the system is:

[1989] text

[1990] User Question: I don't know where this item is.

[1991] Provide appropriate assistance.

[1992] In this way, it is possible to increase user satisfaction by generating optimal feedback and support messages based on emotion recognition.

[1993] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1994] Step 1: The user uses their smartphone to enter basic information (such as their name, areas of interest, and goals). The device receives this input and sends it to the server.

[1995] Step 2: The server stores the received basic information in a database. It also generates a user profile and integrates it with other information stored in the database. The input is the user's basic information, and the output is the generated profile.

[1996] Step 3: The user uses their smartphone to input a specific question (e.g., "Where is this product?"). The device receives the user's input and sends it to the server.

[1997] Step 4: The server analyzes the received question using an emotion engine to detect the user's emotional state. The input is the user's text input, and the output is the detected emotional state. Specifically, the emotion engine analyzes the emotional nuances of the text and identifies emotions such as "confused" or "excited."

[1998] Step 5: The server uses the generative AI model to generate appropriate feedback for the user's question. The input is the user's question and the detected emotional state, and the output is personalized feedback. For example, if the user types, "I don't know where the product is," the generative AI model will generate the response, "The product is on Aisle 5."

[1999] Step 6: Based on the emotional state, the server further generates an appropriate support message for the user. The input is the detected emotional state, and the output is a support message. For example, if "confusion" is detected, the server generates a message saying "I'm sorry, what are you looking for? I'm here to help you."

[2000] Step 7: The server sends the generated feedback and support message to the terminal, which displays it to the user. The user confirms the feedback and enters further questions if necessary.

[2001] Step 8: The server continuously records the user's progress and updates their profile based on that data, making future interactions more personalized. The input is the user's feedback and emotion history, and the output is an updated profile.

[2002] These steps provide personalized feedback and support messages tailored to the user's emotional state.

[2003] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2005] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2006] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2007] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2008] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2009] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2010] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2011] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2012] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2013] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2014] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2015] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2016] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2017] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2018] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2019] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2020] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2021] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2022] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2023] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2024] The following is further disclosed regarding the above embodiment.

[2025] (Claim 1)

[2026] means for receiving and storing basic user information in a database;

[2027] means for analyzing a user's learning history and progress to generate a profile based on the individual user;

[2028] A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile;

[2029] A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery;

[2030] a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan;

[2031] A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation;

[2032] A system including:

[2033] (Claim 2)

[2034] 10. The system of claim 1, further comprising means for using a natural language processing model to provide optimal information based on the user's characteristics and learning level in generating the feedback.

[2035] (Claim 3)

[2036] 10. The system of claim 1, further comprising means for periodically assessing a user's progress and adjusting next steps or learning plans based on learning history.

[2037] "Example 1"

[2038] (Claim 1)

[2039] means for receiving and storing basic user information in a database;

[2040] means for analyzing a user's learning history and progress to generate a profile based on the individual user;

[2041] A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile;

[2042] A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery;

[2043] a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan;

[2044] A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation;

[2045] A means to provide optimal feedback and suggestions to users using generative AI models;

[2046] A means for generating prompt sentences, inputting the prompt sentences to the generative AI model, and providing the generated output to the user;

[2047] A system including:

[2048] (Claim 2)

[2049] 10. The system of claim 1, further comprising means for using a natural language processing model to provide optimal information based on the user's characteristics and learning level in generating the feedback.

[2050] (Claim 3)

[2051] 10. The system of claim 1, further comprising means for periodically assessing a user's progress and adjusting next steps or learning plans based on learning history.

[2052] "Application Example 1"

[2053] (Claim 1)

[2054] means for receiving and storing basic user information in a database;

[2055] means for analyzing a user's learning history and progress to generate a profile based on the individual user;

[2056] A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile;

[2057] A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery;

[2058] a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan;

[2059] A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation;

[2060] means for providing interactive learning support based on specific digital content;

[2061] A means for users to input questions or inquiries about the content they are viewing;

[2062] a means for generating personalized feedback about the content being viewed using a generative AI model; and

[2063] A system including:

[2064] (Claim 2)

[2065] The system of claim 1, wherein the feedback generation uses a natural language processing model that provides optimal information based on the user's characteristics and learning level.

[2066] (Claim 3)

[2067] 10. The system of claim 1, wherein the system periodically evaluates the user's progress and adjusts next steps and learning plans based on the user's learning history.

[2068] "Example 2: Combining Emotion Engines"

[2069] (Claim 1)

[2070] means for receiving and storing basic user information in a database;

[2071] means for analyzing a user's learning history and progress to generate a profile based on the individual user;

[2072] A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile;

[2073] A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery;

[2074] a means for generating encouraging or supportive messages according to the user's emotions, the means including an emotion engine for recognizing the user's emotions;

[2075] a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan;

[2076] A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation;

[2077] A system including:

[2078] (Claim 2)

[2079] 10. The system of claim 1, further comprising means for using a natural language processing model to provide optimal information based on the user's characteristics and learning level i...

Claims

1. means for receiving and storing basic user information in a database; means for analyzing a user's learning history and progress to generate a profile based on the individual user; A means for generating personalized feedback in response to a user's questions or inquiries based on the generated profile; A means of providing the generated feedback to the user and prompting additional questions to encourage the user's awareness and discovery; a means of continually recording the user's learning progress and providing progress-based assessment and necessary adjustments to the learning plan; A means for generating encouragement and praise according to the user's achievement of learning goals and progress, and helping the user maintain motivation; A system including:

2. 10. The system of claim 1, further comprising means for using a natural language processing model to provide optimal information based on the characteristics and learning level of the user in generating the feedback.

3. 10. The system of claim 1, further comprising means for periodically assessing the user's progress and adjusting the next steps or learning plan based on the user's learning history.

Images