system

Abstract

Provide a system. A database means for storing lesson content; an input means for receiving a user request; a server means for searching for and providing lesson content based on the request; a display means for displaying the provided lesson content; an interaction means for the user to ask questions or make additional requests; A generating means for generating additional explanations and supplementary materials using generative artificial intelligence; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The current education system faces the problem of disparities based on region and economic situation, and does not provide an environment where all students can receive an equal, high-quality education. Furthermore, limited teacher resources make it difficult to provide prompt and appropriate feedback to individual students. It is necessary to solve these problems and provide a system where all students can receive an equal, high-quality education. [Means for solving the problem]

[0005] The present invention provides a system that includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for users to ask questions or make additional requests, and a generation means for generating additional explanations and supplemental materials using generative artificial intelligence. This allows users to receive optimal lesson content in response to their requests, and allows for prompt responses to additional questions. Furthermore, because supplemental materials and explanations are automatically generated by generative artificial intelligence, high-quality education can be provided while efficiently utilizing teacher resources.

[0006] "Class content" is a general term for content used for educational purposes to aid learning, such as teaching materials, lecture content, videos, documents, and interactive assignments.

[0007] The term "database means" refers to a structured digital system in which information is efficiently stored, searched, and managed, and in the present invention refers to the entire system that stores lesson content.

[0008] "Input means" refers to a device or method for providing a user's request or information to the system, and in the present invention includes a keyboard, a mouse, a voice recognition device, and the like.

[0009] "Server means" refers to a computer system that has the function of receiving requests from users and searching for and providing the required lesson content.

[0010] "Display means" refers to a device or method for visually presenting lesson content to a user, and in the present invention includes a monitor, a screen, a projector, and the like.

[0011] "Interaction means" refers to a device or method that allows a user to interact with a system in a two-way manner, and in the present invention includes chatbots, feedback forms, voice dialogue systems, etc.

[0012] "Generation means" refers to a device or method that has the function of automatically generating additional explanations or supplementary materials based on requests from users using generative artificial intelligence.

[0013] "Generative artificial intelligence" refers to systems that use deep learning models and natural language processing technology to generate text and speech, answer questions, generate content, and more. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system for effectively providing lesson content in the field of education and for assisting users in answering questions and learning. Specific embodiments of the system will be described below.

[0036] System Overview

[0037] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[0038] server

[0039] The server has the following functions:

[0040] Database Management:

[0041] The server stores lesson content from teachers all over the world, from elementary school teachers to university professors, in a database that includes videos, text materials, interactive assignments, and more.

[0042] The server structures the lesson content and tags each one, including the target age group, subject, and lesson format.

[0043] Content search and delivery:

[0044] The server receives requests from users and searches the database for appropriate lesson content to provide, including requests in text and audio formats.

[0045] The server transmits the lesson content to the user's terminal so that it can be displayed.

[0046] Use of generative AI:

[0047] The server uses generative AI (e.g., GPT) to generate additional explanations and supporting materials in response to the user's question.

[0048] For example, if a user requests a detailed explanation of adding fractions, the generative AI will write out concrete examples and step-by-step instructions.

[0049] Terminal

[0050] The terminal has the following functions:

[0051] User Interface:

[0052] The terminal provides an interface for users to input requests using keyboard input, voice input, touch operation, and other methods.

[0053] The terminal has an interface for displaying lesson content and generated materials.

[0054] Viewing and Interaction:

[0055] The device plays the lesson content received from the server and allows users to view it. It is possible to play videos, display text, and complete interactive assignments.

[0056] The device provides interaction capabilities for users to ask questions or make additional requests, for example, through a chatbot or voice assistant.

[0057] User

[0058] The user performs the following operations:

[0059] Request content:

[0060] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0061] After making a question or request, the user views the lesson content provided by the server.

[0062] Learning and feedback:

[0063] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[0064] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[0065] Specific examples

[0066] 1. Learning basic mathematics

[0067] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[0068] Step 2: The device sends a request to the server.

[0069] Step 3: The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[0070] Step 4: The device displays the lesson content to the user, who plays the video and views the materials.

[0071] Step 5: The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[0072] Step 6: The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[0073] Step 7: The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[0074] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[0078] Step 2:

[0079] The device sends the user's request to the server as an HTTP request, which also includes the user's profile information and learning history.

[0080] Step 3:

[0081] The server analyzes the received HTTP request and identifies the lesson content (in this case, the basics of fractions) that the user is looking for from the request content.

[0082] Step 4:

[0083] The server queries the database to find the relevant lesson content, including the results of text analysis, target age group, learning history, etc.

[0084] Step 5:

[0085] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[0086] Step 6:

[0087] The server embeds the acquired lesson content in an HTTP response and returns it to the terminal.

[0088] Step 7:

[0089] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[0090] Step 8:

[0091] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[0092] Step 9:

[0093] The terminal receives the user's additional questions and sends them back to the server.

[0094] Step 10:

[0095] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials (e.g., "Explain the difference between 1 / 2 and 1 / 3").

[0096] Step 11:

[0097] The server returns the generated answer to the terminal as an HTTP response.

[0098] Step 12:

[0099] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[0100] Step 13:

[0101] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[0102] Step 14:

[0103] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[0104] Through this series of steps, users can effectively advance their learning and contribute to improving the quality of the entire system.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional educational systems have had problems with the inability to provide lesson content or respond to users' follow-up questions quickly and effectively, resulting in reduced learning efficiency. Furthermore, it has been difficult to provide appropriate responses to user requests and generate supplementary materials, resulting in uneven quality of education.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using a generative artificial intelligence model, a communication means for sending the user's request to the server in the form of an HTTP request, and a communication means for sending the generated explanations and supplementary materials to the terminal. This enables effective learning support by responding quickly and appropriately to the user's learning requests and providing advanced supplementary explanations using the generative artificial intelligence model.

[0110] "Class Content" means information or materials used for educational purposes, including videos, text, interactive assignments, etc.

[0111] "Database means" refers to a system or software for storing and managing lesson content in digital form.

[0112] "Input means" refers to a device or interface that allows a user to input requests or questions to the system, and includes a keyboard, microphone, touch screen, etc.

[0113] The "server means" is a computer system that searches a database for lesson content based on a user request and provides appropriate content.

[0114] "Display means" refers to a device or interface for visually displaying the provided lesson content to the user, and includes a display, monitor, etc.

[0115] "Interaction means" refers to devices or interfaces through which users can ask questions or make additional requests, including chatbots, voice assistants, form filling, etc.

[0116] A "generative artificial intelligence model" is a machine learning model that uses natural language processing technology to automatically generate additional explanations and supplementary materials in response to user questions.

[0117] "Generation means" refers to a device or software that uses a generative artificial intelligence model to generate detailed answers and supplementary materials to a user's question.

[0118] "Communication means" refers to the devices and protocols used to transmit and receive user requests and generated responses between the server and the device, including internet connection, Wi-Fi, mobile data, etc.

[0119] The present invention relates to a system for effectively providing lesson content and assisting users in answering questions and learning. Specific embodiments of the present invention will be described below.

[0120] System Overview

[0121] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[0122] server

[0123] The server has the following functions:

[0124] Database Management:

[0125] The server manages lesson content using a database. Specifically, it uses an SQL database (e.g., MySQL (registered trademark), PostgreSQL) to store videos, text materials, interactive assignments, etc. Lesson content is tagged with information such as target age, subject, and lesson format.

[0126] Content search and delivery:

[0127] The server receives a request from a user and searches the database for appropriate lesson content using a search algorithm, and the search results are sent to the terminal via an HTTP request.

[0128] Use of generative AI:

[0129] The server uses a generative artificial intelligence model (e.g., OpenAI's GPT-3®) to generate additional explanations and supplementary materials for the user's questions. For example, if a user requests a detailed explanation of adding fractions, the generative AI automatically generates concrete examples and step-by-step explanations.

[0130] Terminal

[0131] The terminal has the following functions:

[0132] User Interface:

[0133] The device provides an interface for users to input requests using keyboard input, voice input, touch input, a microphone with voice recognition capabilities, or a touchscreen display.

[0134] The terminal is equipped with a user interface for displaying lesson content and generated materials. The display includes video playback functions and a PDF viewer.

[0135] Viewing and Interaction:

[0136] The device plays the lesson content received from the server and allows users to view it. Specifically, it has a video playback application, a PDF viewer, and an interactive assignment display function.

[0137] The terminal provides an interaction function for the user to ask questions or make additional requests. For example, if the user types "What is the difference between 1 / 2 and 1 / 3?", the terminal sends the question to the server.

[0138] User

[0139] The user performs the following operations:

[0140] Request content:

[0141] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0142] After making a question or request, the user views the lesson content provided by the server.

[0143] Learning and feedback:

[0144] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[0145] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[0146] Specific examples

[0147] Learning basic mathematics

[0148] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[0149] 2. The device sends a request to the server.

[0150] 3. The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[0151] 4. The device displays the lesson content to the user, who plays the video and views the materials.

[0152] 5. The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[0153] 6. The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[0154] 7. The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[0155] Prompt Sentence Examples

[0156] If the user requests a detailed explanation of adding fractions, the server sends the following prompt to the generative AI:

[0157] "Please explain fraction addition step by step with concrete examples."

[0158] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] The user inputs what they want to learn on the device. For example, the user might input "I want to learn about the basics of fractions." The input is displayed in an input field on the device. Input can be done using keyboard input or voice input. The device converts the user's input into JSON format data.

[0162] Step 2:

[0163] The device sends the user's input to the server as an HTTP POST request. The input data is in JSON format and is sent to the server's receiving endpoint. At this time, the device uses a network communication module (e.g., Wi-Fi, mobile data). The server analyzes the received request data and extracts the request content.

[0164] Step 3:

[0165] The server uses a database management system (e.g., MySQL) to search the database for lesson content based on the request. Specifically, it executes an SQL query to retrieve lesson content that matches tags or keywords. The search results include the lesson content's metadata and URL.

[0166] Step 4:

[0167] The server converts the acquired lesson content into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received JSON data and extracts the URL and metadata of the lesson content.

[0168] Step 5:

[0169] To display the lesson content to the user, the device launches a video playback application or PDF viewer. Specifically, for video content, a play button is provided, and the video starts when the user clicks it. For text materials or interactive assignments, the appropriate viewer is launched to display the content.

[0170] Step 6:

[0171] The user views the lesson content and, if they have any additional questions, they enter them again in the input field on their device. For example, they might enter, "I don't understand the difference between 1 / 2 and 1 / 3." The input data is again converted to JSON format.

[0172] Step 7:

[0173] The device then sends the additional question to the server as an HTTP POST request, which then analyzes the received question, extracts the specific question, and prepares to send a prompt to the generative AI model based on the question.

[0174] Step 8:

[0175] The server uses a generative AI model (e.g., GPT-3) to generate a detailed answer to the user's question. Specifically, it sends the following prompt to the generative AI model: "Please explain the difference between 1 / 2 and 1 / 3 step by step with concrete examples." The generative AI model then generates an answer text based on this prompt.

[0176] Step 9:

[0177] The server converts the generated response text into JSON format and sends it back to the terminal as an HTTP response. The terminal then analyzes the received JSON data and extracts the generated response text.

[0178] Step 10:

[0179] The terminal displays the generated answer text to the user. Specifically, the text is displayed in a chat window or text viewer. The user can deepen their understanding by viewing the answer. In this way, a quick and appropriate answer to the user's question is provided.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] Conventional virtual store systems often lack detailed product information and usage instructions that users require, and do not offer real-time question-answering or interactive experiences, which can lead to reduced user satisfaction. Furthermore, it is difficult to incorporate effective information provision and question-answering services that utilize generative artificial intelligence. Our goal is to solve these issues and improve the user experience in virtual stores.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, and a server means for searching and providing lesson content based on the requests. This enables a virtual store means for providing product information and offering users an interactive product experience, and a means for answering questions about products using generative artificial intelligence.

[0185] "Database means" refers to a system for storing lesson content and product information, and enabling efficient search and management.

[0186] An "input means" is a device or interface that allows a user to input a request, and can accept voice or text input.

[0187] The "server means" is a server that searches for and provides appropriate content and information from a database based on a request from a user.

[0188] "Display means" refers to a device or interface for displaying the provided lesson content and product information to the user.

[0189] An "interaction means" is a system that provides a dialogue facility for users to ask questions or make additional requests.

[0190] "Generation means" refers to a system for generating additional explanations and supplementary materials using generative artificial intelligence.

[0191] A "virtual storefront" is a system for providing product information and offering users an interactive product experience.

[0192] "Generative AI" is an AI technology that uses natural language processing technology to analyze user input and generate appropriate responses and information.

[0193] "Product Information" refers to information such as detailed product descriptions, instructions for use, and reviews.

[0194] "User" refers to an individual or organization that uses this system.

[0195] MODE FOR CARRYING OUT THE INVENTION

[0196] The present invention relates to a system for effectively providing product information and responding to user product-related questions in real time. The system is comprised of three parties: a server, a terminal, and a user. Specific embodiments are described below.

[0197] server

[0198] The server has the following functions:

[0199] 1. Database Management:

[0200] It stores a wide range of product information from around the world in a database that includes text, images, videos, interactive reviews, and more.

[0201] Product information is tagged for efficient searching and management, including product categories, features, and use cases.

[0202] 2. Content Search and Delivery:

[0203] It receives a request from a user, searches for appropriate product information from a database, and provides it to the terminal.

[0204] Requests can come in voice or text form.

[0205] 3. Use of generative AI:

[0206] It has the ability to generate answers to user questions using natural language processing technology.

[0207] For example, in response to a question such as "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[0208] Terminal

[0209] The terminal has the following functions:

[0210] 1. User Interface:

[0211] It provides an interface for users to input requests using keyboard, voice, touch, etc.

[0212] It provides an interface for displaying the provided product information and generated answers.

[0213] 2. Viewing and Interaction:

[0214] The product information received from the server is reproduced and made available for viewing by the user.

[0215] Provide interaction features for users to ask questions or make additional requests, for example, by using a chatbot or voice assistant.

[0216] User

[0217] The user does the following:

[0218] 1. Product Information Request:

[0219] Use your device to request product information, for example, "Tell me about the features of this phone."

[0220] After making a question or request, view the product information provided by the server.

[0221] 2. Questions and Feedback:

[0222] Browse product information and learn about uses and features.

[0223] If they don't understand something or need further clarification, they ask the generative AI a question, using prompts such as "Tell me how to clean this coffee maker."

[0224] After the learning is complete, feedback is provided to the system and used to improve the system and generate new content.

[0225] Hardware and software used

[0226] Hardware:

[0227] Smartphone (iOS or ANDROID (registered trademark))

[0228] software:

[0229] Framework: Flask (build server API with Python)

[0230] Database: PostgreSQL (manages product information and user queries)

[0231] Generative AI: OpenAI GPT-3 API (detailed product question and answering)

[0232] This allows the system to provide optimal product information in response to user requests and provide detailed question and answer responses using a generative AI model, providing a high-quality user experience.

[0233] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0234] Step 1:

[0235] The user uses the input means of the smartphone to input product information or a question.

[0236] Input methods include keyboard input, voice input, touch operation, etc. The input request is sent to the device.

[0237] Step 2:

[0238] The terminal sends a request to the server, and the server searches the database based on the request.

[0239] Requests include product specific information and questions, and the database stores tagged product information for efficient searching.

[0240] Step 3:

[0241] The server searches the database for the appropriate product information and returns the results to the terminal.

[0242] This process retrieves relevant text, images, videos, etc. from the database, combines them as needed, and sends them to the device, providing the detailed product information requested by the user.

[0243] Step 4:

[0244] The terminal receives the information from the server and displays it on the user interface.

[0245] The received information may include text materials, images, videos, etc., and the device will display them in an appropriate format for the user to view.

[0246] Step 5:

[0247] The user enters a follow-up question based on the product information.

[0248] For example, you can type "Tell me how to use this product." This request is sent again from the device to the server.

[0249] Step 6:

[0250] The server uses a generative AI model (using natural language processing technology) to generate answers to the user's questions.

[0251] Specifically, it analyzes user input, creates prompts to generate appropriate answers, and then makes API requests to the generative AI. For example, in response to a question like, "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[0252] Step 7:

[0253] The generated answer is sent from the server to the terminal and displayed on the user interface.

[0254] Users can view detailed answers and explanations to deepen their understanding. The generated answers include specific steps and precautions.

[0255] These steps allow users to access detailed product information in a virtual store and get real-time answers to their product-related questions.

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] The present invention relates to a system for providing lesson content, answering users' questions, and assisting learning in the field of education. In particular, the system aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. Specific embodiments of the system are described below.

[0258] System Overview

[0259] This system consists of three main elements: a server, a terminal, and a user. The server stores and manages lesson content and provides appropriate content in response to user requests. The terminal is a device through which users interact with the system, and has input and display functions. Users are the entities that acquire educational content through the system and engage in learning. The system also uses an emotion engine to recognize user emotions and dynamically change the way interactions and content are provided based on those emotions.

[0260] server

[0261] The server has the following functions:

[0262] Database Management:

[0263] The server stores a database of various lesson content provided by educators from elementary school to university, including videos, texts, and interactive assignments.

[0264] Lesson content is tagged based on target age, subject, and lesson format.

[0265] Content search and delivery:

[0266] The server receives a user request, searches the database for the most suitable lesson content, and provides it.

[0267] The lesson content is transmitted to the user's terminal and displayed.

[0268] Use of generative AI:

[0269] The server uses generative AI (e.g., GPT) to generate supporting materials and explanations based on the user's questions and additional requests.

[0270] For example, if a user requests, "Teach me more about adding fractions," the generative AI will generate specific explanations and examples.

[0271] Emotion Engine

[0272] The emotion engine has the following functions:

[0273] Emotion recognition:

[0274] Analyzes emotions from the user's voice, facial expressions, text input, etc.

[0275] Based on the emotion data, the user's current emotional state is recognized.

[0276] Emotion-based interactions:

[0277] The generative AI adjusts the explanations and writing style provided based on the user's emotions.

[0278] For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[0279] Terminal

[0280] The terminal has the following functions:

[0281] User Interface:

[0282] The device provides an interface for users to input requests, which can be keyboard input, voice input, or touch operation.

[0283] It has an interface for displaying lesson content and generated materials.

[0284] Using the Emotion Engine:

[0285] The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time.

[0286] The emotion engine adjusts the content provided by the server based on this.

[0287] User

[0288] The user performs the following operations:

[0289] Request content:

[0290] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0291] View the lesson content provided by the server in response to the request.

[0292] Emotional feedback:

[0293] Users can input their emotions into the device or express them spontaneously while studying. The emotion engine analyzes these emotions, and the server optimizes the content provided.

[0294] Specific examples

[0295] 1. Emotion-based learning regulation

[0296] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[0297] Step 2: The device sends this request to the server. At the same time, it analyzes the user's emotional state from their facial expressions and voice and sends the emotional data to the server.

[0298] Step 3: The server searches the database for the most suitable lesson content based on the received request and emotion data, and sends it to the terminal.

[0299] Step 4: The device displays the lesson content to the user.

[0300] Step 5: The user asks a further question while browsing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3." At the same time, the emotion engine detects anxiety in the user's voice.

[0301] Step 6: The server uses generative AI to generate a friendly explanation for the user's follow-up questions and sends it to the device.

[0302] Step 7: The device displays the generated explanation to the user, who can study it and deepen their understanding.

[0303] In this way, the system can provide a personalized learning experience that takes into account the user's emotional state, enabling a user-centered educational environment.

[0304] The processing flow will be explained below.

[0305] Step 1:

[0306] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[0307] Step 2:

[0308] The device analyzes the user's emotional state from their facial expressions and voice, along with their request. The emotion engine determines the user's current emotional state and stores the result as data.

[0309] Step 3:

[0310] The device sends the user's request and emotional data to the server as an HTTP request, which also includes the user's profile information and learning history.

[0311] Step 4:

[0312] The server analyzes the received HTTP request and identifies the lesson content the user is looking for based on the request content and emotional data.

[0313] Step 5:

[0314] The server queries the database to find relevant lesson content, including text analysis results, target age group, learning history, and emotional data.

[0315] Step 6:

[0316] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[0317] Step 7:

[0318] The server uses generative AI to adjust the lesson content based on the emotional data. For example, if the emotional data indicates that the user is tired, it generates a short, concise explanation.

[0319] Step 8:

[0320] The server embeds the adjusted lesson content into an HTTP response and sends it to the terminal.

[0321] Step 9:

[0322] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[0323] Step 10:

[0324] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[0325] Step 11:

[0326] The device receives the user's additional question and sends it back to the server, along with the additional emotion data.

[0327] Step 12:

[0328] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials. The emotion engine then analyzes the user's emotional state again and adjusts the content accordingly.

[0329] Step 13:

[0330] The server returns the generated answer to the terminal as an HTTP response.

[0331] Step 14:

[0332] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[0333] Step 15:

[0334] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[0335] Step 16:

[0336] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[0337] Through this series of steps, users can learn effectively and receive support appropriate to their individual emotional state, improving the quality of the system as a whole.

[0338] Example 2

[0339] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional educational systems struggle to provide content that is tailored to each individual user because they lack the ability to consider the user's level of understanding and emotional state. Furthermore, conventional systems have limited ability to respond immediately to follow-up questions or requests from users. This can lead to problems such as confusion and insufficient understanding during learning. Therefore, there is a need for a system that can analyze the user's emotional state in real time, provide a personalized learning experience based on that information, and quickly provide appropriate learning support.

[0341] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a data storage means for saving lesson content, an input means for receiving a user request, a server means for searching for and providing lesson content based on the request, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations or supplementary materials using generative artificial intelligence, and an emotion recognition means for analyzing emotional data from the user's facial expressions and voice and adjusting the content to be provided based on the emotion. This makes it possible to analyze the user's emotional state and provide appropriate content based on that, thereby effectively supporting the user's learning.

[0342] "Class content" refers to videos, texts, interactive assignments, etc. used for educational purposes.

[0343] "Data storage means" refers to a database or storage system for saving and managing lesson content.

[0344] "Input means" refers to an interface that allows users to input what they want to learn or questions into the system, and includes keyboard input and voice input.

[0345] The "server means" refers to a computer system that receives a user's request, searches for appropriate lesson content from a database, and provides it.

[0346] "Display means" refers to a device for displaying lesson content provided by the server to the user, such as a monitor or screen.

[0347] "Interaction means" refers to the interface or method by which a user can ask questions or make additional requests.

[0348] "Generation means" is a function that uses generative artificial intelligence to generate supplementary materials and explanations in response to user questions and additional requests.

[0349] "Emotion recognition means" refers to a system or function that analyzes emotional data from the user's facial expressions and voice, and dynamically adjusts the content provided based on that data.

[0350] "Generative AI" refers to an AI model that generates sentences and explanations using natural language processing technology.

[0351] This invention is a system for providing lesson content, answering users' questions, and assisting learning in the field of education. Specifically, it aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. This system is composed of the following elements.

[0352] server

[0353] The server has a data storage means for saving and managing lesson content. This data storage means stores various lesson content provided by educators from elementary school to university. This includes videos, texts, and interactive assignments. The lesson content is tagged based on target age, subject, and lesson format.

[0354] The server then receives the user's request using a content search means, searches the database for the most appropriate lesson content, and provides it.Furthermore, the server has a generation means that uses generative artificial intelligence (e.g., GPT-3) to generate supplementary materials and explanations based on the user's questions and additional requests.

[0355] Terminal

[0356] The terminal provides an input means for the user to input a request, which includes keyboard input, voice input, and touch operation. The terminal is equipped with a display means for displaying lesson content received from the server, thereby providing educational content to the user.

[0357] The device also uses an emotion engine to analyze the user's facial expressions and voice, and transmits emotional data to the server in real time, which then becomes the basis for dynamically adjusting the content provided by the server.

[0358] User

[0359] Users can access educational content through this system and study. Specifically, they use their devices to request the subject they want to learn, such as "Teach me how to solve quadratic equations." The server responds to this request and provides the most appropriate lesson content.

[0360] Users can also display or spontaneously express their emotions while learning. The emotion engine analyzes these and the server uses them to optimize content delivery. For example, if the user is recognized as "frustrated because they don't understand," the generative AI will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[0361] Specific examples

[0362] For example, the following shows the sequence of events when a user inputs "I want to learn about the basics of fractions." The user inputs this request into the device, and the device sends this request along with the user's facial expression data to the server. Based on the received request and emotional data, the server searches its database for the most suitable lesson content and provides it to the device. The device displays this lesson content to the user, who then studies it.

[0363] Furthermore, if the user enters a follow-up question such as "I don't know the difference between 1 / 2 and 1 / 3," the emotion engine detects the user's anxiety. The server uses generative AI to generate a specific explanation in a gentle tone and provides it to the device. The device then displays this generated explanation to the user, deepening their understanding.

[0364] Prompt Sentence Examples

[0365] "If a user types in 'I want to learn about the basics of fractions,' please provide appropriate lesson content that takes their feelings into consideration. Also, explain how to respond if they have additional questions."

[0366] With the above-described configuration, the present invention can provide a personalized learning experience that takes into account the user's emotional state, thereby improving learning efficiency.

[0367] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0368] Step 1:

[0369] The user inputs a content request. For example, the user sends a request to the device using keyboard or voice input, such as "I want to learn about the basics of fractions." The input request is sent to the server by the device in real time.

[0370] Step 2:

[0371] The device collects and analyzes emotion data along with the user's request. The device captures the user's facial expressions and voice and analyzes them using emotion recognition means. The analyzed emotion data includes information on whether the user is relaxed or anxious, and transmits this data to the server.

[0372] Step 3:

[0373] The server searches for lesson content based on the received request and emotional data. The server extracts lesson content appropriate for the target age, subject, and emotional state from the data storage means. For example, video content tagged with "fractions," "basic," and "relaxation" is extracted. The server then transmits this to the terminal.

[0374] Step 4:

[0375] The terminal displays the lesson content received from the server. The display means of the terminal plays the video and provides it to the user. The user studies while watching this video.

[0376] Step 5:

[0377] While viewing lesson content, the user can input an additional question. For example, "I don't know the difference between 1 / 2 and 1 / 3." In this case, the device's emotion engine detects anxiety from the user's facial expression and voice, and sends this emotion data to the server.

[0378] Step 6:

[0379] The server uses generative AI to generate an answer based on the additional questions and emotional data. The server inputs the prompt into a generative AI model (e.g., GPT-3) to generate a specific and gentle answer. For example, an explanation such as "1 / 2 is one of two parts, and 1 / 3 is one of three parts" is generated.

[0380] Step 7:

[0381] The device displays the generated answer to the user. The explanation generated by the generative AI is displayed to the user via the display means, allowing the user to gain additional knowledge.

[0382] The above processing steps enable real-time analysis of the user's emotional state and the provision of individually optimized educational content and support based on that analysis.

[0383] (Application example 2)

[0384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] Conventional educational systems provide uniform lesson content without considering the user's emotional state, which can make it difficult for users to fully understand the learning content. Furthermore, when users ask questions or make additional requests, appropriate responses based on their emotions are not provided, resulting in lower user satisfaction. In particular, there is a need for systems that can respond to situations where face-to-face customer support is difficult, such as in real-world environments such as brick-and-mortar stores.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0387] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using generative artificial intelligence, an emotion recognition means for recognizing the user's emotions, and an adjustment means for dynamically adjusting the lesson content provided and the generated explanations based on the recognized emotions, thereby making it possible to provide more personalized educational support based on the user's emotions.

[0388] Creating definition statements

[0389] "Course content" is a collective term for digital materials such as text, videos, images, and interactive assignments used for educational purposes.

[0390] "Database means" is a system for organizing, storing, and managing lesson content.

[0391] "Input means" refers to a device or function that provides an interface for users to input requests to the system, and includes keyboards, voice input, touch operations, etc.

[0392] The "server means" is a server or computer system having the function of searching a database for and providing the most suitable lesson content based on a user request received from an input means.

[0393] "Display means" refers to a device or interface for visually presenting lesson content to users, and includes monitors, smartphones, tablets, etc.

[0394] An "interaction means" is a device or function that allows two-way communication for users to ask questions or make additional requests.

[0395] The "generation means" is a computer system that has the function of using generative artificial intelligence to generate explanations and supplementary materials in response to user questions and additional requests.

[0396] "Emotion recognition means" refers to a device or software that analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes their state.

[0397] The "adjustment means" is a system that has the functionality to dynamically adjust the lesson content provided and the explanations generated based on the recognized emotions.

[0398] MODE FOR CARRYING OUT THE INVENTION

[0399] System configuration

[0400] The system of the present invention consists of three main elements: a server device, a terminal device, and a user. The server device stores and manages lesson content and provides appropriate content in response to user requests. The terminal device is a device through which the user interacts with the system and has input and display functions. The user is the entity that acquires educational content through the system and engages in learning. The system is equipped with an emotion engine that can recognize the user's emotions and dynamically change the interaction and content delivery method based on those emotions.

[0401] Server device functions

[0402] The server device includes the following main functions:

[0403] Database management: The server stores lesson contents provided by various educators in a database and tags them based on target age, subject, and lesson format.

[0404] Content search and provision: The server receives a user request, searches the database for the most appropriate lesson content, and provides it. The provided content is sent to the terminal and displayed to the user.

[0405] Use of generative AI: Generative AI (e.g., GPT) is used to generate supplementary material and explanations based on user questions and additional requests. For example, in response to a request such as "Tell me more about adding fractions," generative AI can generate specific explanations and examples.

[0406] Emotion Engine Functions

[0407] The emotion engine has the following features:

[0408] Emotion recognition: Analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes the user's current emotional state based on the emotional data.

[0409] Emotion-based interaction: The generative AI will adjust the explanations and writing style it provides depending on the user's emotions. For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific and easy-to-understand examples and explanations in a gentler tone.

[0410] Terminal device functions

[0411] The terminal device has the following functions:

[0412] User interface: The terminal provides an interface for users to input requests. Keyboard input, voice input, and touch operation are possible. It also has an interface for displaying lesson content and generated materials.

[0413] Use of emotion engine: The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time. The emotion engine adjusts the content provided by the server based on this information.

[0414] User operations

[0415] The user does the following:

[0416] Requesting content: The user uses the device to request what they want to learn and browses the lesson content provided by the server in response to the request. For example, they might type, "Teach me how to solve quadratic equations."

[0417] Emotional feedback: Users can input their emotions into the device or express them spontaneously while learning. The emotion engine analyzes this and the server optimizes the content provided.

[0418] Specific examples

[0419] Example 1: Emotion-based learning adjustment

[0420] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[0421] 2. The device sends this request to the server, analyzes the user's emotional state from their facial expressions and voice, and sends the emotional data to the server.

[0422] 3. The server searches the database for the most appropriate lesson content based on the request and emotion data and sends it to the device.

[0423] 4. The device displays the lesson content to the user.

[0424] 5. The user asks a further question while viewing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3," and the emotion engine detects the sense of anxiety.

[0425] 6. The server uses generative AI to generate gentle explanations in response to the user's follow-up questions and sends them to the device.

[0426] 7. The terminal displays the generated explanation to the user, who then studies it.

[0427] Prompt Sentence Examples

[0428] Q: Does this restaurant have a menu for peanut allergies?

[0429] Emotion: Confused

[0430] answer:

[0431] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0432] Program processing flow

[0433] Step 1:

[0434] The user inputs a request to the terminal. The input method is keyboard input or voice input. This input includes a request such as "I want to learn the basics of fractions." The input data is sent to the server by the terminal.

[0435] Step 2:

[0436] The device captures the user's facial expressions and voice using a camera and microphone and sends them to the emotion recognition means. The emotion recognition means analyzes the voice and facial expression data to recognize the user's emotional state. The data used here is voice data and image data, and the emotional state (e.g., anxiety, joy, confusion, etc.) is output as the analysis result.

[0437] Step 3:

[0438] The server uses a database management means to search for the most suitable lesson content based on the received request and emotional data. The input data includes the request and the emotional state. The server extracts the target content from the database and selects it as the data to be sent to the terminal.

[0439] Step 4:

[0440] The server uses generative AI as needed to generate explanations and supplemental materials based on the user's request and emotional state. The server inputs the request and emotional state as prompts and uses a generative AI model (e.g., GPT-3) to generate appropriate explanations. The generated content is output in text format and sent to the device.

[0441] Step 5:

[0442] The terminal displays the lesson content received from the server and the generated explanations. The display means is a monitor or a smartphone screen. The content is presented to the user visually.

[0443] Step 6:

[0444] The user browses the lesson content and enters any additional questions or requests they may have. At this time, the device sends the input data to the server again, and at the same time sends the user's new emotional state to the emotion recognition means. The new input data and emotional data are analyzed again, and the server takes appropriate action.

[0445] Step 7:

[0446] In response to new questions or requests, the server again uses generative AI to generate answers and explanations and send them to the device, and this process is repeated until the user is fully trained.

[0447] Specific operation example

[0448] Step 1:

[0449] User: Says, "Tell me more about this book."

[0450] Step 2:

[0451] The device transmits the voice data to the server, and at the same time captures the user's facial expression with a camera and transmits it to the emotion recognition means, which then detects "interest" from the user's facial expression.

[0452] Step 3:

[0453] The server receives the request and the emotional state "interest," searches the database for detailed information about the related book, and sends it to the terminal.

[0454] Step 4:

[0455] Use generative AI to generate text like "This book has the following features:..."

[0456] Step 5:

[0457] The terminal displays the received information on the screen so that the user can view it.

[0458] Step 6:

[0459] The user enters a follow-up question: "Please tell me more specifically about the contents of each chapter."

[0460] Step 7:

[0461] The server sends additional questions and the emotional state "interest" as prompts to the generative AI, which then generates a detailed explanation and sends it back to the device. This process is repeated.

[0462] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0464] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0465] [Second embodiment]

[0466] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0467] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0469] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0473] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0474] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0475] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0476] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0477] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0478] The present invention relates to a system for effectively providing lesson content in the field of education and for assisting users in answering questions and learning. Specific embodiments of the system will be described below.

[0479] System Overview

[0480] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[0481] server

[0482] The server has the following functions:

[0483] Database Management:

[0484] The server stores lesson content from teachers all over the world, from elementary school teachers to university professors, in a database that includes videos, text materials, interactive assignments, and more.

[0485] The server structures the lesson content and tags each one, including the target age group, subject, and lesson format.

[0486] Content search and delivery:

[0487] The server receives requests from users and searches the database for appropriate lesson content to provide, including requests in text and audio formats.

[0488] The server transmits the lesson content to the user's terminal so that it can be displayed.

[0489] Use of generative AI:

[0490] The server uses generative AI (e.g., GPT) to generate additional explanations and supporting materials in response to the user's question.

[0491] For example, if a user requests a detailed explanation of adding fractions, the generative AI will write out concrete examples and step-by-step instructions.

[0492] Terminal

[0493] The terminal has the following functions:

[0494] User Interface:

[0495] The terminal provides an interface for users to input requests using keyboard input, voice input, touch operation, and other methods.

[0496] The terminal has an interface for displaying lesson content and generated materials.

[0497] Viewing and Interaction:

[0498] The device plays the lesson content received from the server and allows users to view it. It is possible to play videos, display text, and complete interactive assignments.

[0499] The device provides interaction capabilities for users to ask questions or make additional requests, for example, through a chatbot or voice assistant.

[0500] User

[0501] The user performs the following operations:

[0502] Request content:

[0503] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0504] After making a question or request, the user views the lesson content provided by the server.

[0505] Learning and feedback:

[0506] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[0507] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[0508] Specific examples

[0509] 1. Learning basic mathematics

[0510] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[0511] Step 2: The device sends a request to the server.

[0512] Step 3: The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[0513] Step 4: The device displays the lesson content to the user, who plays the video and views the materials.

[0514] Step 5: The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[0515] Step 6: The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[0516] Step 7: The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[0517] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[0518] The processing flow will be explained below.

[0519] Step 1:

[0520] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[0521] Step 2:

[0522] The device sends the user's request to the server as an HTTP request, which also includes the user's profile information and learning history.

[0523] Step 3:

[0524] The server analyzes the received HTTP request and identifies the lesson content (in this case, the basics of fractions) that the user is looking for from the request content.

[0525] Step 4:

[0526] The server queries the database to find the relevant lesson content, including the results of text analysis, target age group, learning history, etc.

[0527] Step 5:

[0528] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[0529] Step 6:

[0530] The server embeds the acquired lesson content in an HTTP response and returns it to the terminal.

[0531] Step 7:

[0532] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[0533] Step 8:

[0534] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[0535] Step 9:

[0536] The terminal receives the user's additional questions and sends them back to the server.

[0537] Step 10:

[0538] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials (e.g., "Explain the difference between 1 / 2 and 1 / 3").

[0539] Step 11:

[0540] The server returns the generated answer to the terminal as an HTTP response.

[0541] Step 12:

[0542] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[0543] Step 13:

[0544] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[0545] Step 14:

[0546] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[0547] Through this series of steps, users can effectively advance their learning and contribute to improving the quality of the entire system.

[0548] Example 1

[0549] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0550] Conventional educational systems have had problems with the inability to provide lesson content or respond to users' follow-up questions quickly and effectively, resulting in reduced learning efficiency. Furthermore, it has been difficult to provide appropriate responses to user requests and generate supplementary materials, resulting in uneven quality of education.

[0551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0552] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using a generative artificial intelligence model, a communication means for sending the user's request to the server in the form of an HTTP request, and a communication means for sending the generated explanations and supplementary materials to the terminal. This enables effective learning support by responding quickly and appropriately to the user's learning requests and providing advanced supplementary explanations using the generative artificial intelligence model.

[0553] "Class Content" means information or materials used for educational purposes, including videos, text, interactive assignments, etc.

[0554] "Database means" refers to a system or software for storing and managing lesson content in digital form.

[0555] "Input means" refers to a device or interface that allows a user to input requests or questions to the system, and includes a keyboard, microphone, touch screen, etc.

[0556] The "server means" is a computer system that searches a database for lesson content based on a user request and provides appropriate content.

[0557] "Display means" refers to a device or interface for visually displaying the provided lesson content to the user, and includes a display, monitor, etc.

[0558] "Interaction means" refers to devices or interfaces through which users can ask questions or make additional requests, including chatbots, voice assistants, form filling, etc.

[0559] A "generative artificial intelligence model" is a machine learning model that uses natural language processing technology to automatically generate additional explanations and supplementary materials in response to user questions.

[0560] "Generation means" refers to a device or software that uses a generative artificial intelligence model to generate detailed answers and supplementary materials to a user's question.

[0561] "Communication means" refers to the devices and protocols used to transmit and receive user requests and generated responses between the server and the device, including internet connection, Wi-Fi, mobile data, etc.

[0562] The present invention relates to a system for effectively providing lesson content and assisting users in answering questions and learning. Specific embodiments of the present invention will be described below.

[0563] System Overview

[0564] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[0565] server

[0566] The server has the following functions:

[0567] Database Management:

[0568] The server manages the lesson content using a database. Specifically, it uses an SQL database (e.g., MySQL or PostgreSQL) to store videos, text materials, interactive assignments, etc. The lesson content is tagged with information such as target age, subject, and lesson format.

[0569] Content search and delivery:

[0570] The server receives a request from a user and searches the database for appropriate lesson content using a search algorithm, and the search results are sent to the terminal via an HTTP request.

[0571] Use of generative AI:

[0572] The server uses a generative artificial intelligence model (e.g., OpenAI's GPT-3) to generate additional explanations and supplementary materials for the user's question. For example, if a user requests a detailed explanation of adding fractions, the generative AI automatically generates concrete examples and step-by-step explanations.

[0573] Terminal

[0574] The terminal has the following functions:

[0575] User Interface:

[0576] The device provides an interface for users to input requests using keyboard input, voice input, touch input, a microphone with voice recognition capabilities, or a touchscreen display.

[0577] The terminal is equipped with a user interface for displaying lesson content and generated materials. The display includes video playback functions and a PDF viewer.

[0578] Viewing and Interaction:

[0579] The device plays the lesson content received from the server and allows users to view it. Specifically, it has a video playback application, a PDF viewer, and an interactive assignment display function.

[0580] The terminal provides an interaction function for the user to ask questions or make additional requests. For example, if the user types "What is the difference between 1 / 2 and 1 / 3?", the terminal sends the question to the server.

[0581] User

[0582] The user performs the following operations:

[0583] Request content:

[0584] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0585] After making a question or request, the user views the lesson content provided by the server.

[0586] Learning and feedback:

[0587] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[0588] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[0589] Specific examples

[0590] Learning basic mathematics

[0591] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[0592] 2. The device sends a request to the server.

[0593] 3. The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[0594] 4. The device displays the lesson content to the user, who plays the video and views the materials.

[0595] 5. The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[0596] 6. The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[0597] 7. The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[0598] Prompt Sentence Examples

[0599] If the user requests a detailed explanation of adding fractions, the server sends the following prompt to the generative AI:

[0600] "Please explain fraction addition step by step with concrete examples."

[0601] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[0602] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0603] Step 1:

[0604] The user inputs what they want to learn on the device. For example, the user might input "I want to learn about the basics of fractions." The input is displayed in an input field on the device. Input can be done using keyboard input or voice input. The device converts the user's input into JSON format data.

[0605] Step 2:

[0606] The device sends the user's input to the server as an HTTP POST request. The input data is in JSON format and is sent to the server's receiving endpoint. At this time, the device uses a network communication module (e.g., Wi-Fi, mobile data). The server analyzes the received request data and extracts the request content.

[0607] Step 3:

[0608] The server uses a database management system (e.g., MySQL) to search the database for lesson content based on the request. Specifically, it executes an SQL query to retrieve lesson content that matches tags or keywords. The search results include the lesson content's metadata and URL.

[0609] Step 4:

[0610] The server converts the acquired lesson content into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received JSON data and extracts the URL and metadata of the lesson content.

[0611] Step 5:

[0612] To display the lesson content to the user, the device launches a video playback application or PDF viewer. Specifically, for video content, a play button is provided, and the video starts when the user clicks it. For text materials or interactive assignments, the appropriate viewer is launched to display the content.

[0613] Step 6:

[0614] The user views the lesson content and, if they have any additional questions, they enter them again in the input field on their device. For example, they might enter, "I don't understand the difference between 1 / 2 and 1 / 3." The input data is again converted to JSON format.

[0615] Step 7:

[0616] The device then sends the additional question to the server as an HTTP POST request, which then analyzes the received question, extracts the specific question, and prepares to send a prompt to the generative AI model based on the question.

[0617] Step 8:

[0618] The server uses a generative AI model (e.g., GPT-3) to generate a detailed answer to the user's question. Specifically, it sends the following prompt to the generative AI model: "Please explain the difference between 1 / 2 and 1 / 3 step by step with concrete examples." The generative AI model then generates an answer text based on this prompt.

[0619] Step 9:

[0620] The server converts the generated response text into JSON format and sends it back to the terminal as an HTTP response. The terminal then analyzes the received JSON data and extracts the generated response text.

[0621] Step 10:

[0622] The terminal displays the generated answer text to the user. Specifically, the text is displayed in a chat window or text viewer. The user can deepen their understanding by viewing the answer. In this way, a quick and appropriate answer to the user's question is provided.

[0623] (Application example 1)

[0624] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0625] Conventional virtual store systems often lack detailed product information and usage instructions that users require, and do not offer real-time question-answering or interactive experiences, which can lead to reduced user satisfaction. Furthermore, it is difficult to incorporate effective information provision and question-answering services that utilize generative artificial intelligence. Our goal is to solve these issues and improve the user experience in virtual stores.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0627] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, and a server means for searching and providing lesson content based on the requests. This enables a virtual store means for providing product information and offering users an interactive product experience, and a means for answering questions about products using generative artificial intelligence.

[0628] "Database means" refers to a system for storing lesson content and product information, and enabling efficient search and management.

[0629] An "input means" is a device or interface that allows a user to input a request, and can accept voice or text input.

[0630] The "server means" is a server that searches for and provides appropriate content and information from a database based on a request from a user.

[0631] "Display means" refers to a device or interface for displaying the provided lesson content and product information to the user.

[0632] An "interaction means" is a system that provides a dialogue facility for users to ask questions or make additional requests.

[0633] "Generation means" refers to a system for generating additional explanations and supplementary materials using generative artificial intelligence.

[0634] A "virtual storefront" is a system for providing product information and offering users an interactive product experience.

[0635] "Generative AI" is an AI technology that uses natural language processing technology to analyze user input and generate appropriate responses and information.

[0636] "Product Information" refers to information such as detailed product descriptions, instructions for use, and reviews.

[0637] "User" refers to an individual or organization that uses this system.

[0638] MODE FOR CARRYING OUT THE INVENTION

[0639] The present invention relates to a system for effectively providing product information and responding to user product-related questions in real time. The system is comprised of three parties: a server, a terminal, and a user. Specific embodiments are described below.

[0640] server

[0641] The server has the following functions:

[0642] 1. Database Management:

[0643] It stores a wide range of product information from around the world in a database that includes text, images, videos, interactive reviews, and more.

[0644] Product information is tagged for efficient searching and management, including product categories, features, and use cases.

[0645] 2. Content Search and Delivery:

[0646] It receives a request from a user, searches for appropriate product information from a database, and provides it to the terminal.

[0647] Requests can come in voice or text form.

[0648] 3. Use of generative AI:

[0649] It has the ability to generate answers to user questions using natural language processing technology.

[0650] For example, in response to a question such as "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[0651] Terminal

[0652] The terminal has the following functions:

[0653] 1. User Interface:

[0654] It provides an interface for users to input requests using keyboard, voice, touch, etc.

[0655] It provides an interface for displaying the provided product information and generated answers.

[0656] 2. Viewing and Interaction:

[0657] The product information received from the server is reproduced and made available for viewing by the user.

[0658] Provide interaction features for users to ask questions or make additional requests, for example, by using a chatbot or voice assistant.

[0659] User

[0660] The user does the following:

[0661] 1. Product Information Request:

[0662] Use your device to request product information, for example, "Tell me about the features of this phone."

[0663] After making a question or request, view the product information provided by the server.

[0664] 2. Questions and Feedback:

[0665] Browse product information and learn about uses and features.

[0666] If they don't understand something or need further clarification, they ask the generative AI a question, using prompts such as "Tell me how to clean this coffee maker."

[0667] After the learning is complete, feedback is provided to the system and used to improve the system and generate new content.

[0668] Hardware and software used

[0669] Hardware:

[0670] Smartphone (iOS or Android)

[0671] software:

[0672] Framework: Flask (build server API with Python)

[0673] Database: PostgreSQL (manages product information and user queries)

[0674] Generative AI: OpenAI GPT-3 API (detailed product question and answering)

[0675] This allows the system to provide optimal product information in response to user requests and provide detailed question and answer responses using a generative AI model, providing a high-quality user experience.

[0676] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0677] Step 1:

[0678] The user uses the input means of the smartphone to input product information or a question.

[0679] Input methods include keyboard input, voice input, touch operation, etc. The input request is sent to the device.

[0680] Step 2:

[0681] The terminal sends a request to the server, and the server searches the database based on the request.

[0682] Requests include product specific information and questions, and the database stores tagged product information for efficient searching.

[0683] Step 3:

[0684] The server searches the database for the appropriate product information and returns the results to the terminal.

[0685] This process retrieves relevant text, images, videos, etc. from the database, combines them as needed, and sends them to the device, providing the detailed product information requested by the user.

[0686] Step 4:

[0687] The terminal receives the information from the server and displays it on the user interface.

[0688] The received information may include text materials, images, videos, etc., and the device will display them in an appropriate format for the user to view.

[0689] Step 5:

[0690] The user enters a follow-up question based on the product information.

[0691] For example, you can type "Tell me how to use this product." This request is sent again from the device to the server.

[0692] Step 6:

[0693] The server uses a generative AI model (using natural language processing technology) to generate answers to the user's questions.

[0694] Specifically, it analyzes user input, creates prompts to generate appropriate answers, and then makes API requests to the generative AI. For example, in response to a question like, "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[0695] Step 7:

[0696] The generated answer is sent from the server to the terminal and displayed on the user interface.

[0697] Users can view detailed answers and explanations to deepen their understanding. The generated answers include specific steps and precautions.

[0698] These steps allow users to access detailed product information in a virtual store and get real-time answers to their product-related questions.

[0699] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0700] The present invention relates to a system for providing lesson content, answering users' questions, and assisting learning in the field of education. In particular, the system aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. Specific embodiments of the system are described below.

[0701] System Overview

[0702] This system consists of three main elements: a server, a terminal, and a user. The server stores and manages lesson content and provides appropriate content in response to user requests. The terminal is a device through which users interact with the system, and has input and display functions. Users are the entities that acquire educational content through the system and engage in learning. The system also uses an emotion engine to recognize user emotions and dynamically change the way interactions and content are provided based on those emotions.

[0703] server

[0704] The server has the following functions:

[0705] Database Management:

[0706] The server stores a database of various lesson content provided by educators from elementary school to university, including videos, texts, and interactive assignments.

[0707] Lesson content is tagged based on target age, subject, and lesson format.

[0708] Content search and delivery:

[0709] The server receives a user request, searches the database for the most suitable lesson content, and provides it.

[0710] The lesson content is transmitted to the user's terminal and displayed.

[0711] Use of generative AI:

[0712] The server uses generative AI (e.g., GPT) to generate supporting materials and explanations based on the user's questions and additional requests.

[0713] For example, if a user requests, "Teach me more about adding fractions," the generative AI will generate specific explanations and examples.

[0714] Emotion Engine

[0715] The emotion engine has the following functions:

[0716] Emotion recognition:

[0717] Analyzes emotions from the user's voice, facial expressions, text input, etc.

[0718] Based on the emotion data, the user's current emotional state is recognized.

[0719] Emotion-based interactions:

[0720] The generative AI adjusts the explanations and writing style provided based on the user's emotions.

[0721] For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[0722] Terminal

[0723] The terminal has the following functions:

[0724] User Interface:

[0725] The device provides an interface for users to input requests, which can be keyboard input, voice input, or touch operation.

[0726] It has an interface for displaying lesson content and generated materials.

[0727] Using the Emotion Engine:

[0728] The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time.

[0729] The emotion engine adjusts the content provided by the server based on this.

[0730] User

[0731] The user performs the following operations:

[0732] Request content:

[0733] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0734] View the lesson content provided by the server in response to the request.

[0735] Emotional feedback:

[0736] Users can input their emotions into the device or express them spontaneously while studying. The emotion engine analyzes these emotions, and the server optimizes the content provided.

[0737] Specific examples

[0738] 1. Emotion-based learning regulation

[0739] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[0740] Step 2: The device sends this request to the server. At the same time, it analyzes the user's emotional state from their facial expressions and voice and sends the emotional data to the server.

[0741] Step 3: The server searches the database for the most suitable lesson content based on the received request and emotion data, and sends it to the terminal.

[0742] Step 4: The device displays the lesson content to the user.

[0743] Step 5: The user asks a further question while browsing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3." At the same time, the emotion engine detects anxiety in the user's voice.

[0744] Step 6: The server uses generative AI to generate a friendly explanation for the user's follow-up questions and sends it to the device.

[0745] Step 7: The device displays the generated explanation to the user, who can study it and deepen their understanding.

[0746] In this way, the system can provide a personalized learning experience that takes into account the user's emotional state, enabling a user-centered educational environment.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[0750] Step 2:

[0751] The device analyzes the user's emotional state from their facial expressions and voice, along with their request. The emotion engine determines the user's current emotional state and stores the result as data.

[0752] Step 3:

[0753] The device sends the user's request and emotional data to the server as an HTTP request, which also includes the user's profile information and learning history.

[0754] Step 4:

[0755] The server analyzes the received HTTP request and identifies the lesson content the user is looking for based on the request content and emotional data.

[0756] Step 5:

[0757] The server queries the database to find relevant lesson content, including text analysis results, target age group, learning history, and emotional data.

[0758] Step 6:

[0759] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[0760] Step 7:

[0761] The server uses generative AI to adjust the lesson content based on the emotional data. For example, if the emotional data indicates that the user is tired, it generates a short, concise explanation.

[0762] Step 8:

[0763] The server embeds the adjusted lesson content into an HTTP response and sends it to the terminal.

[0764] Step 9:

[0765] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[0766] Step 10:

[0767] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[0768] Step 11:

[0769] The device receives the user's additional question and sends it back to the server, along with the additional emotion data.

[0770] Step 12:

[0771] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials. The emotion engine then analyzes the user's emotional state again and adjusts the content accordingly.

[0772] Step 13:

[0773] The server returns the generated answer to the terminal as an HTTP response.

[0774] Step 14:

[0775] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[0776] Step 15:

[0777] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[0778] Step 16:

[0779] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[0780] Through this series of steps, users can learn effectively and receive support appropriate to their individual emotional state, improving the quality of the system as a whole.

[0781] Example 2

[0782] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0783] Conventional educational systems struggle to provide content that is tailored to each individual user because they lack the ability to consider the user's level of understanding and emotional state. Furthermore, conventional systems have limited ability to respond immediately to follow-up questions or requests from users. This can lead to problems such as confusion and insufficient understanding during learning. Therefore, there is a need for a system that can analyze the user's emotional state in real time, provide a personalized learning experience based on that information, and quickly provide appropriate learning support.

[0784] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a data storage means for saving lesson content, an input means for receiving a user request, a server means for searching for and providing lesson content based on the request, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations or supplementary materials using generative artificial intelligence, and an emotion recognition means for analyzing emotional data from the user's facial expressions and voice and adjusting the content to be provided based on the emotion. This makes it possible to analyze the user's emotional state and provide appropriate content based on that, thereby effectively supporting the user's learning.

[0785] "Class content" refers to videos, texts, interactive assignments, etc. used for educational purposes.

[0786] "Data storage means" refers to a database or storage system for saving and managing lesson content.

[0787] "Input means" refers to an interface that allows users to input what they want to learn or questions into the system, and includes keyboard input and voice input.

[0788] The "server means" refers to a computer system that receives a user's request, searches for appropriate lesson content from a database, and provides it.

[0789] "Display means" refers to a device for displaying lesson content provided by the server to the user, such as a monitor or screen.

[0790] "Interaction means" refers to the interface or method by which a user can ask questions or make additional requests.

[0791] "Generation means" is a function that uses generative artificial intelligence to generate supplementary materials and explanations in response to user questions and additional requests.

[0792] "Emotion recognition means" refers to a system or function that analyzes emotional data from the user's facial expressions and voice, and dynamically adjusts the content provided based on that data.

[0793] "Generative AI" refers to an AI model that generates sentences and explanations using natural language processing technology.

[0794] This invention is a system for providing lesson content, answering users' questions, and assisting learning in the field of education. Specifically, it aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. This system is composed of the following elements.

[0795] server

[0796] The server has a data storage means for saving and managing lesson content. This data storage means stores various lesson content provided by educators from elementary school to university. This includes videos, texts, and interactive assignments. The lesson content is tagged based on target age, subject, and lesson format.

[0797] The server then receives the user's request using a content search means, searches the database for the most appropriate lesson content, and provides it.Furthermore, the server has a generation means that uses generative artificial intelligence (e.g., GPT-3) to generate supplementary materials and explanations based on the user's questions and additional requests.

[0798] Terminal

[0799] The terminal provides an input means for the user to input a request, which includes keyboard input, voice input, and touch operation. The terminal is equipped with a display means for displaying lesson content received from the server, thereby providing educational content to the user.

[0800] The device also uses an emotion engine to analyze the user's facial expressions and voice, and transmits emotional data to the server in real time, which then becomes the basis for dynamically adjusting the content provided by the server.

[0801] User

[0802] Users can access educational content through this system and study. Specifically, they use their devices to request the subject they want to learn, such as "Teach me how to solve quadratic equations." The server responds to this request and provides the most appropriate lesson content.

[0803] Users can also display or spontaneously express their emotions while learning. The emotion engine analyzes these and the server uses them to optimize content delivery. For example, if the user is recognized as "frustrated because they don't understand," the generative AI will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[0804] Specific examples

[0805] For example, the following shows the sequence of events when a user inputs "I want to learn about the basics of fractions." The user inputs this request into the device, and the device sends this request along with the user's facial expression data to the server. Based on the received request and emotional data, the server searches its database for the most suitable lesson content and provides it to the device. The device displays this lesson content to the user, who then studies it.

[0806] Furthermore, if the user enters a follow-up question such as "I don't know the difference between 1 / 2 and 1 / 3," the emotion engine detects the user's anxiety. The server uses generative AI to generate a specific explanation in a gentle tone and provides it to the device. The device then displays this generated explanation to the user, deepening their understanding.

[0807] Prompt Sentence Examples

[0808] "If a user types in 'I want to learn about the basics of fractions,' please provide appropriate lesson content that takes their feelings into consideration. Also, explain how to respond if they have additional questions."

[0809] With the above-described configuration, the present invention can provide a personalized learning experience that takes into account the user's emotional state, thereby improving learning efficiency.

[0810] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0811] Step 1:

[0812] The user inputs a content request. For example, the user sends a request to the device using keyboard or voice input, such as "I want to learn about the basics of fractions." The input request is sent to the server by the device in real time.

[0813] Step 2:

[0814] The device collects and analyzes emotion data along with the user's request. The device captures the user's facial expressions and voice and analyzes them using emotion recognition means. The analyzed emotion data includes information on whether the user is relaxed or anxious, and transmits this data to the server.

[0815] Step 3:

[0816] The server searches for lesson content based on the received request and emotional data. The server extracts lesson content appropriate for the target age, subject, and emotional state from the data storage means. For example, video content tagged with "fractions," "basic," and "relaxation" is extracted. The server then transmits this to the terminal.

[0817] Step 4:

[0818] The terminal displays the lesson content received from the server. The display means of the terminal plays the video and provides it to the user. The user studies while watching this video.

[0819] Step 5:

[0820] While viewing lesson content, the user can input an additional question. For example, "I don't know the difference between 1 / 2 and 1 / 3." In this case, the device's emotion engine detects anxiety from the user's facial expression and voice, and sends this emotion data to the server.

[0821] Step 6:

[0822] The server uses generative AI to generate an answer based on the additional questions and emotional data. The server inputs the prompt into a generative AI model (e.g., GPT-3) to generate a specific and gentle answer. For example, an explanation such as "1 / 2 is one of two parts, and 1 / 3 is one of three parts" is generated.

[0823] Step 7:

[0824] The device displays the generated answer to the user. The explanation generated by the generative AI is displayed to the user via the display means, allowing the user to gain additional knowledge.

[0825] The above processing steps enable real-time analysis of the user's emotional state and the provision of individually optimized educational content and support based on that analysis.

[0826] (Application example 2)

[0827] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0828] Conventional educational systems provide uniform lesson content without considering the user's emotional state, which can make it difficult for users to fully understand the learning content. Furthermore, when users ask questions or make additional requests, appropriate responses based on their emotions are not provided, resulting in lower user satisfaction. In particular, there is a need for systems that can respond to situations where face-to-face customer support is difficult, such as in real-world environments such as brick-and-mortar stores.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0830] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using generative artificial intelligence, an emotion recognition means for recognizing the user's emotions, and an adjustment means for dynamically adjusting the lesson content provided and the generated explanations based on the recognized emotions, thereby making it possible to provide more personalized educational support based on the user's emotions.

[0831] Creating definition statements

[0832] "Course content" is a collective term for digital materials such as text, videos, images, and interactive assignments used for educational purposes.

[0833] "Database means" is a system for organizing, storing, and managing lesson content.

[0834] "Input means" refers to a device or function that provides an interface for users to input requests to the system, and includes keyboards, voice input, touch operations, etc.

[0835] The "server means" is a server or computer system having the function of searching a database for and providing the most suitable lesson content based on a user request received from an input means.

[0836] "Display means" refers to a device or interface for visually presenting lesson content to users, and includes monitors, smartphones, tablets, etc.

[0837] An "interaction means" is a device or function that allows two-way communication for users to ask questions or make additional requests.

[0838] The "generation means" is a computer system that has the function of using generative artificial intelligence to generate explanations and supplementary materials in response to user questions and additional requests.

[0839] "Emotion recognition means" refers to a device or software that analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes their state.

[0840] The "adjustment means" is a system that has the functionality to dynamically adjust the lesson content provided and the explanations generated based on the recognized emotions.

[0841] MODE FOR CARRYING OUT THE INVENTION

[0842] System configuration

[0843] The system of the present invention consists of three main elements: a server device, a terminal device, and a user. The server device stores and manages lesson content and provides appropriate content in response to user requests. The terminal device is a device through which the user interacts with the system and has input and display functions. The user is the entity that acquires educational content through the system and engages in learning. The system is equipped with an emotion engine that can recognize the user's emotions and dynamically change the interaction and content delivery method based on those emotions.

[0844] Server device functions

[0845] The server device includes the following main functions:

[0846] Database management: The server stores lesson contents provided by various educators in a database and tags them based on target age, subject, and lesson format.

[0847] Content search and provision: The server receives a user request, searches the database for the most appropriate lesson content, and provides it. The provided content is sent to the terminal and displayed to the user.

[0848] Use of generative AI: Generative AI (e.g., GPT) is used to generate supplementary material and explanations based on user questions and additional requests. For example, in response to a request such as "Tell me more about adding fractions," generative AI can generate specific explanations and examples.

[0849] Emotion Engine Functions

[0850] The emotion engine has the following features:

[0851] Emotion recognition: Analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes the user's current emotional state based on the emotional data.

[0852] Emotion-based interaction: The generative AI will adjust the explanations and writing style it provides depending on the user's emotions. For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific and easy-to-understand examples and explanations in a gentler tone.

[0853] Terminal device functions

[0854] The terminal device has the following functions:

[0855] User interface: The terminal provides an interface for users to input requests. Keyboard input, voice input, and touch operation are possible. It also has an interface for displaying lesson content and generated materials.

[0856] Use of emotion engine: The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time. The emotion engine adjusts the content provided by the server based on this information.

[0857] User operations

[0858] The user does the following:

[0859] Requesting content: The user uses the device to request what they want to learn and browses the lesson content provided by the server in response to the request. For example, they might type, "Teach me how to solve quadratic equations."

[0860] Emotional feedback: Users can input their emotions into the device or express them spontaneously while learning. The emotion engine analyzes this and the server optimizes the content provided.

[0861] Specific examples

[0862] Example 1: Emotion-based learning adjustment

[0863] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[0864] 2. The device sends this request to the server, analyzes the user's emotional state from their facial expressions and voice, and sends the emotional data to the server.

[0865] 3. The server searches the database for the most appropriate lesson content based on the request and emotion data and sends it to the device.

[0866] 4. The device displays the lesson content to the user.

[0867] 5. The user asks a further question while viewing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3," and the emotion engine detects the sense of anxiety.

[0868] 6. The server uses generative AI to generate gentle explanations in response to the user's follow-up questions and sends them to the device.

[0869] 7. The terminal displays the generated explanation to the user, who then studies it.

[0870] Prompt Sentence Examples

[0871] Q: Does this restaurant have a menu for peanut allergies?

[0872] Emotion: Confused

[0873] answer:

[0874] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0875] Program processing flow

[0876] Step 1:

[0877] The user inputs a request to the terminal. The input method is keyboard input or voice input. This input includes a request such as "I want to learn the basics of fractions." The input data is sent to the server by the terminal.

[0878] Step 2:

[0879] The device captures the user's facial expressions and voice using a camera and microphone and sends them to the emotion recognition means. The emotion recognition means analyzes the voice and facial expression data to recognize the user's emotional state. The data used here is voice data and image data, and the emotional state (e.g., anxiety, joy, confusion, etc.) is output as the analysis result.

[0880] Step 3:

[0881] The server uses a database management means to search for the most suitable lesson content based on the received request and emotional data. The input data includes the request and the emotional state. The server extracts the target content from the database and selects it as the data to be sent to the terminal.

[0882] Step 4:

[0883] The server uses generative AI as needed to generate explanations and supplemental materials based on the user's request and emotional state. The server inputs the request and emotional state as prompts and uses a generative AI model (e.g., GPT-3) to generate appropriate explanations. The generated content is output in text format and sent to the device.

[0884] Step 5:

[0885] The terminal displays the lesson content received from the server and the generated explanations. The display means is a monitor or a smartphone screen. The content is presented to the user visually.

[0886] Step 6:

[0887] The user browses the lesson content and enters any additional questions or requests they may have. At this time, the device sends the input data to the server again, and at the same time sends the user's new emotional state to the emotion recognition means. The new input data and emotional data are analyzed again, and the server takes appropriate action.

[0888] Step 7:

[0889] In response to new questions or requests, the server again uses generative AI to generate answers and explanations and send them to the device, and this process is repeated until the user is fully trained.

[0890] Specific operation example

[0891] Step 1:

[0892] User: Says, "Tell me more about this book."

[0893] Step 2:

[0894] The device transmits the voice data to the server, and at the same time captures the user's facial expression with a camera and transmits it to the emotion recognition means, which then detects "interest" from the user's facial expression.

[0895] Step 3:

[0896] The server receives the request and the emotional state "interest," searches the database for detailed information about the related book, and sends it to the terminal.

[0897] Step 4:

[0898] Use generative AI to generate text like "This book has the following features:..."

[0899] Step 5:

[0900] The terminal displays the received information on the screen so that the user can view it.

[0901] Step 6:

[0902] The user enters a follow-up question: "Please tell me more specifically about the contents of each chapter."

[0903] Step 7:

[0904] The server sends additional questions and the emotional state "interest" as prompts to the generative AI, which then generates a detailed explanation and sends it back to the device. This process is repeated.

[0905] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0906] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0907] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0908] [Third embodiment]

[0909] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0910] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0911] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0912] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0913] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0914] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0915] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0916] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0917] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0918] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0919] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0920] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0921] The present invention relates to a system for effectively providing lesson content in the field of education and for assisting users in answering questions and learning. Specific embodiments of the system will be described below.

[0922] System Overview

[0923] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[0924] server

[0925] The server has the following functions:

[0926] Database Management:

[0927] The server stores lesson content from teachers all over the world, from elementary school teachers to university professors, in a database that includes videos, text materials, interactive assignments, and more.

[0928] The server structures the lesson content and tags each one, including the target age group, subject, and lesson format.

[0929] Content search and delivery:

[0930] The server receives requests from users and searches the database for appropriate lesson content to provide, including requests in text and audio formats.

[0931] The server transmits the lesson content to the user's terminal so that it can be displayed.

[0932] Use of generative AI:

[0933] The server uses generative AI (e.g., GPT) to generate additional explanations and supporting materials in response to the user's question.

[0934] For example, if a user requests a detailed explanation of adding fractions, the generative AI will write out concrete examples and step-by-step instructions.

[0935] Terminal

[0936] The terminal has the following functions:

[0937] User Interface:

[0938] The terminal provides an interface for users to input requests using keyboard input, voice input, touch operation, and other methods.

[0939] The terminal has an interface for displaying lesson content and generated materials.

[0940] Viewing and Interaction:

[0941] The device plays the lesson content received from the server and allows users to view it. It is possible to play videos, display text, and complete interactive assignments.

[0942] The device provides interaction capabilities for users to ask questions or make additional requests, for example, through a chatbot or voice assistant.

[0943] User

[0944] The user performs the following operations:

[0945] Request content:

[0946] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[0947] After making a question or request, the user views the lesson content provided by the server.

[0948] Learning and feedback:

[0949] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[0950] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[0951] Specific examples

[0952] 1. Learning basic mathematics

[0953] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[0954] Step 2: The device sends a request to the server.

[0955] Step 3: The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[0956] Step 4: The device displays the lesson content to the user, who plays the video and views the materials.

[0957] Step 5: The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[0958] Step 6: The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[0959] Step 7: The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[0960] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[0961] The processing flow will be explained below.

[0962] Step 1:

[0963] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[0964] Step 2:

[0965] The device sends the user's request to the server as an HTTP request, which also includes the user's profile information and learning history.

[0966] Step 3:

[0967] The server analyzes the received HTTP request and identifies the lesson content (in this case, the basics of fractions) that the user is looking for from the request content.

[0968] Step 4:

[0969] The server queries the database to find the relevant lesson content, including the results of text analysis, target age group, learning history, etc.

[0970] Step 5:

[0971] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[0972] Step 6:

[0973] The server embeds the acquired lesson content in an HTTP response and returns it to the terminal.

[0974] Step 7:

[0975] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[0976] Step 8:

[0977] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[0978] Step 9:

[0979] The terminal receives the user's additional questions and sends them back to the server.

[0980] Step 10:

[0981] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials (e.g., "Explain the difference between 1 / 2 and 1 / 3").

[0982] Step 11:

[0983] The server returns the generated answer to the terminal as an HTTP response.

[0984] Step 12:

[0985] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[0986] Step 13:

[0987] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[0988] Step 14:

[0989] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[0990] Through this series of steps, users can effectively advance their learning and contribute to improving the quality of the entire system.

[0991] Example 1

[0992] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0993] Conventional educational systems have had problems with the inability to provide lesson content or respond to users' follow-up questions quickly and effectively, resulting in reduced learning efficiency. Furthermore, it has been difficult to provide appropriate responses to user requests and generate supplementary materials, resulting in uneven quality of education.

[0994] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0995] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using a generative artificial intelligence model, a communication means for sending the user's request to the server in the form of an HTTP request, and a communication means for sending the generated explanations and supplementary materials to the terminal. This enables effective learning support by responding quickly and appropriately to the user's learning requests and providing advanced supplementary explanations using the generative artificial intelligence model.

[0996] "Class Content" means information or materials used for educational purposes, including videos, text, interactive assignments, etc.

[0997] "Database means" refers to a system or software for storing and managing lesson content in digital form.

[0998] "Input means" refers to a device or interface that allows a user to input requests or questions to the system, and includes a keyboard, microphone, touch screen, etc.

[0999] The "server means" is a computer system that searches a database for lesson content based on a user request and provides appropriate content.

[1000] "Display means" refers to a device or interface for visually displaying the provided lesson content to the user, and includes a display, monitor, etc.

[1001] "Interaction means" refers to devices or interfaces through which users can ask questions or make additional requests, including chatbots, voice assistants, form filling, etc.

[1002] A "generative artificial intelligence model" is a machine learning model that uses natural language processing technology to automatically generate additional explanations and supplementary materials in response to user questions.

[1003] "Generation means" refers to a device or software that uses a generative artificial intelligence model to generate detailed answers and supplementary materials to a user's question.

[1004] "Communication means" refers to the devices and protocols used to transmit and receive user requests and generated responses between the server and the device, including internet connection, Wi-Fi, mobile data, etc.

[1005] The present invention relates to a system for effectively providing lesson content and assisting users in answering questions and learning. Specific embodiments of the present invention will be described below.

[1006] System Overview

[1007] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[1008] server

[1009] The server has the following functions:

[1010] Database Management:

[1011] The server manages the lesson content using a database. Specifically, it uses an SQL database (e.g., MySQL or PostgreSQL) to store videos, text materials, interactive assignments, etc. The lesson content is tagged with information such as target age, subject, and lesson format.

[1012] Content search and delivery:

[1013] The server receives a request from a user and searches the database for appropriate lesson content using a search algorithm, and the search results are sent to the terminal via an HTTP request.

[1014] Use of generative AI:

[1015] The server uses a generative artificial intelligence model (e.g., OpenAI's GPT-3) to generate additional explanations and supplementary materials for the user's question. For example, if a user requests a detailed explanation of adding fractions, the generative AI automatically generates concrete examples and step-by-step explanations.

[1016] Terminal

[1017] The terminal has the following functions:

[1018] User Interface:

[1019] The device provides an interface for users to input requests using keyboard input, voice input, touch input, a microphone with voice recognition capabilities, or a touchscreen display.

[1020] The terminal is equipped with a user interface for displaying lesson content and generated materials. The display includes video playback functions and a PDF viewer.

[1021] Viewing and Interaction:

[1022] The device plays the lesson content received from the server and allows users to view it. Specifically, it has a video playback application, a PDF viewer, and an interactive assignment display function.

[1023] The terminal provides an interaction function for the user to ask questions or make additional requests. For example, if the user types "What is the difference between 1 / 2 and 1 / 3?", the terminal sends the question to the server.

[1024] User

[1025] The user performs the following operations:

[1026] Request content:

[1027] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[1028] After making a question or request, the user views the lesson content provided by the server.

[1029] Learning and feedback:

[1030] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[1031] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[1032] Specific examples

[1033] Learning basic mathematics

[1034] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[1035] 2. The device sends a request to the server.

[1036] 3. The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[1037] 4. The device displays the lesson content to the user, who plays the video and views the materials.

[1038] 5. The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[1039] 6. The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[1040] 7. The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[1041] Prompt Sentence Examples

[1042] If the user requests a detailed explanation of adding fractions, the server sends the following prompt to the generative AI:

[1043] "Please explain fraction addition step by step with concrete examples."

[1044] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[1045] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1046] Step 1:

[1047] The user inputs what they want to learn on the device. For example, the user might input "I want to learn about the basics of fractions." The input is displayed in an input field on the device. Input can be done using keyboard input or voice input. The device converts the user's input into JSON format data.

[1048] Step 2:

[1049] The device sends the user's input to the server as an HTTP POST request. The input data is in JSON format and is sent to the server's receiving endpoint. At this time, the device uses a network communication module (e.g., Wi-Fi, mobile data). The server analyzes the received request data and extracts the request content.

[1050] Step 3:

[1051] The server uses a database management system (e.g., MySQL) to search the database for lesson content based on the request. Specifically, it executes an SQL query to retrieve lesson content that matches tags or keywords. The search results include the lesson content's metadata and URL.

[1052] Step 4:

[1053] The server converts the acquired lesson content into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received JSON data and extracts the URL and metadata of the lesson content.

[1054] Step 5:

[1055] To display the lesson content to the user, the device launches a video playback application or PDF viewer. Specifically, for video content, a play button is provided, and the video starts when the user clicks it. For text materials or interactive assignments, the appropriate viewer is launched to display the content.

[1056] Step 6:

[1057] The user views the lesson content and, if they have any additional questions, they enter them again in the input field on their device. For example, they might enter, "I don't understand the difference between 1 / 2 and 1 / 3." The input data is again converted to JSON format.

[1058] Step 7:

[1059] The device then sends the additional question to the server as an HTTP POST request, which then analyzes the received question, extracts the specific question, and prepares to send a prompt to the generative AI model based on the question.

[1060] Step 8:

[1061] The server uses a generative AI model (e.g., GPT-3) to generate a detailed answer to the user's question. Specifically, it sends the following prompt to the generative AI model: "Please explain the difference between 1 / 2 and 1 / 3 step by step with concrete examples." The generative AI model then generates an answer text based on this prompt.

[1062] Step 9:

[1063] The server converts the generated response text into JSON format and sends it back to the terminal as an HTTP response. The terminal then analyzes the received JSON data and extracts the generated response text.

[1064] Step 10:

[1065] The terminal displays the generated answer text to the user. Specifically, the text is displayed in a chat window or text viewer. The user can deepen their understanding by viewing the answer. In this way, a quick and appropriate answer to the user's question is provided.

[1066] (Application example 1)

[1067] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1068] Conventional virtual store systems often lack detailed product information and usage instructions that users require, and do not offer real-time question-answering or interactive experiences, which can lead to reduced user satisfaction. Furthermore, it is difficult to incorporate effective information provision and question-answering services that utilize generative artificial intelligence. Our goal is to solve these issues and improve the user experience in virtual stores.

[1069] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1070] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, and a server means for searching and providing lesson content based on the requests. This enables a virtual store means for providing product information and offering users an interactive product experience, and a means for answering questions about products using generative artificial intelligence.

[1071] "Database means" refers to a system for storing lesson content and product information, and enabling efficient search and management.

[1072] An "input means" is a device or interface that allows a user to input a request, and can accept voice or text input.

[1073] The "server means" is a server that searches for and provides appropriate content and information from a database based on a request from a user.

[1074] "Display means" refers to a device or interface for displaying the provided lesson content and product information to the user.

[1075] An "interaction means" is a system that provides a dialogue facility for users to ask questions or make additional requests.

[1076] "Generation means" refers to a system for generating additional explanations and supplementary materials using generative artificial intelligence.

[1077] A "virtual storefront" is a system for providing product information and offering users an interactive product experience.

[1078] "Generative AI" is an AI technology that uses natural language processing technology to analyze user input and generate appropriate responses and information.

[1079] "Product Information" refers to information such as detailed product descriptions, instructions for use, and reviews.

[1080] "User" refers to an individual or organization that uses this system.

[1081] MODE FOR CARRYING OUT THE INVENTION

[1082] The present invention relates to a system for effectively providing product information and responding to user product-related questions in real time. The system is comprised of three parties: a server, a terminal, and a user. Specific embodiments are described below.

[1083] server

[1084] The server has the following functions:

[1085] 1. Database Management:

[1086] It stores a wide range of product information from around the world in a database that includes text, images, videos, interactive reviews, and more.

[1087] Product information is tagged for efficient searching and management, including product categories, features, and use cases.

[1088] 2. Content Search and Delivery:

[1089] It receives a request from a user, searches for appropriate product information from a database, and provides it to the terminal.

[1090] Requests can come in voice or text form.

[1091] 3. Use of generative AI:

[1092] It has the ability to generate answers to user questions using natural language processing technology.

[1093] For example, in response to a question such as "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[1094] Terminal

[1095] The terminal has the following functions:

[1096] 1. User Interface:

[1097] It provides an interface for users to input requests using keyboard, voice, touch, etc.

[1098] It provides an interface for displaying the provided product information and generated answers.

[1099] 2. Viewing and Interaction:

[1100] The product information received from the server is reproduced and made available for viewing by the user.

[1101] Provide interaction features for users to ask questions or make additional requests, for example, by using a chatbot or voice assistant.

[1102] User

[1103] The user does the following:

[1104] 1. Product Information Request:

[1105] Use your device to request product information, for example, "Tell me about the features of this phone."

[1106] After making a question or request, view the product information provided by the server.

[1107] 2. Questions and Feedback:

[1108] Browse product information and learn about uses and features.

[1109] If they don't understand something or need further clarification, they ask the generative AI a question, using prompts such as "Tell me how to clean this coffee maker."

[1110] After the learning is complete, feedback is provided to the system and used to improve the system and generate new content.

[1111] Hardware and software used

[1112] Hardware:

[1113] Smartphone (iOS or Android)

[1114] software:

[1115] Framework: Flask (build server API with Python)

[1116] Database: PostgreSQL (manages product information and user queries)

[1117] Generative AI: OpenAI GPT-3 API (detailed product question and answering)

[1118] This allows the system to provide optimal product information in response to user requests and provide detailed question and answer responses using a generative AI model, providing a high-quality user experience.

[1119] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1120] Step 1:

[1121] The user uses the input means of the smartphone to input product information or a question.

[1122] Input methods include keyboard input, voice input, touch operation, etc. The input request is sent to the device.

[1123] Step 2:

[1124] The terminal sends a request to the server, and the server searches the database based on the request.

[1125] Requests include product specific information and questions, and the database stores tagged product information for efficient searching.

[1126] Step 3:

[1127] The server searches the database for the appropriate product information and returns the results to the terminal.

[1128] This process retrieves relevant text, images, videos, etc. from the database, combines them as needed, and sends them to the device, providing the detailed product information requested by the user.

[1129] Step 4:

[1130] The terminal receives the information from the server and displays it on the user interface.

[1131] The received information may include text materials, images, videos, etc., and the device will display them in an appropriate format for the user to view.

[1132] Step 5:

[1133] The user enters a follow-up question based on the product information.

[1134] For example, you can type "Tell me how to use this product." This request is sent again from the device to the server.

[1135] Step 6:

[1136] The server uses a generative AI model (using natural language processing technology) to generate answers to the user's questions.

[1137] Specifically, it analyzes user input, creates prompts to generate appropriate answers, and then makes API requests to the generative AI. For example, in response to a question like, "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[1138] Step 7:

[1139] The generated answer is sent from the server to the terminal and displayed on the user interface.

[1140] Users can view detailed answers and explanations to deepen their understanding. The generated answers include specific steps and precautions.

[1141] These steps allow users to access detailed product information in a virtual store and get real-time answers to their product-related questions.

[1142] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1143] The present invention relates to a system for providing lesson content, answering users' questions, and assisting learning in the field of education. In particular, the system aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. Specific embodiments of the system are described below.

[1144] System Overview

[1145] This system consists of three main elements: a server, a terminal, and a user. The server stores and manages lesson content and provides appropriate content in response to user requests. The terminal is a device through which users interact with the system, and has input and display functions. Users are the entities that acquire educational content through the system and engage in learning. The system also uses an emotion engine to recognize user emotions and dynamically change the way interactions and content are provided based on those emotions.

[1146] server

[1147] The server has the following functions:

[1148] Database Management:

[1149] The server stores a database of various lesson content provided by educators from elementary school to university, including videos, texts, and interactive assignments.

[1150] Lesson content is tagged based on target age, subject, and lesson format.

[1151] Content search and delivery:

[1152] The server receives a user request, searches the database for the most suitable lesson content, and provides it.

[1153] The lesson content is transmitted to the user's terminal and displayed.

[1154] Use of generative AI:

[1155] The server uses generative AI (e.g., GPT) to generate supporting materials and explanations based on the user's questions and additional requests.

[1156] For example, if a user requests, "Teach me more about adding fractions," the generative AI will generate specific explanations and examples.

[1157] Emotion Engine

[1158] The emotion engine has the following functions:

[1159] Emotion recognition:

[1160] Analyzes emotions from the user's voice, facial expressions, text input, etc.

[1161] Based on the emotion data, the user's current emotional state is recognized.

[1162] Emotion-based interactions:

[1163] The generative AI adjusts the explanations and writing style provided based on the user's emotions.

[1164] For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[1165] Terminal

[1166] The terminal has the following functions:

[1167] User Interface:

[1168] The device provides an interface for users to input requests, which can be keyboard input, voice input, or touch operation.

[1169] It has an interface for displaying lesson content and generated materials.

[1170] Using the Emotion Engine:

[1171] The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time.

[1172] The emotion engine adjusts the content provided by the server based on this.

[1173] User

[1174] The user performs the following operations:

[1175] Request content:

[1176] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[1177] View the lesson content provided by the server in response to the request.

[1178] Emotional feedback:

[1179] Users can input their emotions into the device or express them spontaneously while studying. The emotion engine analyzes these emotions, and the server optimizes the content provided.

[1180] Specific examples

[1181] 1. Emotion-based learning regulation

[1182] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[1183] Step 2: The device sends this request to the server. At the same time, it analyzes the user's emotional state from their facial expressions and voice and sends the emotional data to the server.

[1184] Step 3: The server searches the database for the most suitable lesson content based on the received request and emotion data, and sends it to the terminal.

[1185] Step 4: The device displays the lesson content to the user.

[1186] Step 5: The user asks a further question while browsing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3." At the same time, the emotion engine detects anxiety in the user's voice.

[1187] Step 6: The server uses generative AI to generate a friendly explanation for the user's follow-up questions and sends it to the device.

[1188] Step 7: The device displays the generated explanation to the user, who can study it and deepen their understanding.

[1189] In this way, the system can provide a personalized learning experience that takes into account the user's emotional state, enabling a user-centered educational environment.

[1190] The processing flow will be explained below.

[1191] Step 1:

[1192] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[1193] Step 2:

[1194] The device analyzes the user's emotional state from their facial expressions and voice, along with their request. The emotion engine determines the user's current emotional state and stores the result as data.

[1195] Step 3:

[1196] The device sends the user's request and emotional data to the server as an HTTP request, which also includes the user's profile information and learning history.

[1197] Step 4:

[1198] The server analyzes the received HTTP request and identifies the lesson content the user is looking for based on the request content and emotional data.

[1199] Step 5:

[1200] The server queries the database to find relevant lesson content, including text analysis results, target age group, learning history, and emotional data.

[1201] Step 6:

[1202] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[1203] Step 7:

[1204] The server uses generative AI to adjust the lesson content based on the emotional data. For example, if the emotional data indicates that the user is tired, it generates a short, concise explanation.

[1205] Step 8:

[1206] The server embeds the adjusted lesson content into an HTTP response and sends it to the terminal.

[1207] Step 9:

[1208] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[1209] Step 10:

[1210] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[1211] Step 11:

[1212] The device receives the user's additional question and sends it back to the server, along with the additional emotion data.

[1213] Step 12:

[1214] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials. The emotion engine then analyzes the user's emotional state again and adjusts the content accordingly.

[1215] Step 13:

[1216] The server returns the generated answer to the terminal as an HTTP response.

[1217] Step 14:

[1218] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[1219] Step 15:

[1220] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[1221] Step 16:

[1222] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[1223] Through this series of steps, users can learn effectively and receive support appropriate to their individual emotional state, improving the quality of the system as a whole.

[1224] Example 2

[1225] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1226] Conventional educational systems struggle to provide content that is tailored to each individual user because they lack the ability to consider the user's level of understanding and emotional state. Furthermore, conventional systems have limited ability to respond immediately to follow-up questions or requests from users. This can lead to problems such as confusion and insufficient understanding during learning. Therefore, there is a need for a system that can analyze the user's emotional state in real time, provide a personalized learning experience based on that information, and quickly provide appropriate learning support.

[1227] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a data storage means for saving lesson content, an input means for receiving a user request, a server means for searching for and providing lesson content based on the request, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations or supplementary materials using generative artificial intelligence, and an emotion recognition means for analyzing emotional data from the user's facial expressions and voice and adjusting the content to be provided based on the emotion. This makes it possible to analyze the user's emotional state and provide appropriate content based on that, thereby effectively supporting the user's learning.

[1228] "Class content" refers to videos, texts, interactive assignments, etc. used for educational purposes.

[1229] "Data storage means" refers to a database or storage system for saving and managing lesson content.

[1230] "Input means" refers to an interface that allows users to input what they want to learn or questions into the system, and includes keyboard input and voice input.

[1231] The "server means" refers to a computer system that receives a user's request, searches for appropriate lesson content from a database, and provides it.

[1232] "Display means" refers to a device for displaying lesson content provided by the server to the user, such as a monitor or screen.

[1233] "Interaction means" refers to the interface or method by which a user can ask questions or make additional requests.

[1234] "Generation means" is a function that uses generative artificial intelligence to generate supplementary materials and explanations in response to user questions and additional requests.

[1235] "Emotion recognition means" refers to a system or function that analyzes emotional data from the user's facial expressions and voice, and dynamically adjusts the content provided based on that data.

[1236] "Generative AI" refers to an AI model that generates sentences and explanations using natural language processing technology.

[1237] This invention is a system for providing lesson content, answering users' questions, and assisting learning in the field of education. Specifically, it aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. This system is composed of the following elements.

[1238] server

[1239] The server has a data storage means for saving and managing lesson content. This data storage means stores various lesson content provided by educators from elementary school to university. This includes videos, texts, and interactive assignments. The lesson content is tagged based on target age, subject, and lesson format.

[1240] The server then receives the user's request using a content search means, searches the database for the most appropriate lesson content, and provides it.Furthermore, the server has a generation means that uses generative artificial intelligence (e.g., GPT-3) to generate supplementary materials and explanations based on the user's questions and additional requests.

[1241] Terminal

[1242] The terminal provides an input means for the user to input a request, which includes keyboard input, voice input, and touch operation. The terminal is equipped with a display means for displaying lesson content received from the server, thereby providing educational content to the user.

[1243] The device also uses an emotion engine to analyze the user's facial expressions and voice, and transmits emotional data to the server in real time, which then becomes the basis for dynamically adjusting the content provided by the server.

[1244] User

[1245] Users can access educational content through this system and study. Specifically, they use their devices to request the subject they want to learn, such as "Teach me how to solve quadratic equations." The server responds to this request and provides the most appropriate lesson content.

[1246] Users can also display or spontaneously express their emotions while learning. The emotion engine analyzes these and the server uses them to optimize content delivery. For example, if the user is recognized as "frustrated because they don't understand," the generative AI will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[1247] Specific examples

[1248] For example, the following shows the sequence of events when a user inputs "I want to learn about the basics of fractions." The user inputs this request into the device, and the device sends this request along with the user's facial expression data to the server. Based on the received request and emotional data, the server searches its database for the most suitable lesson content and provides it to the device. The device displays this lesson content to the user, who then studies it.

[1249] Furthermore, if the user enters a follow-up question such as "I don't know the difference between 1 / 2 and 1 / 3," the emotion engine detects the user's anxiety. The server uses generative AI to generate a specific explanation in a gentle tone and provides it to the device. The device then displays this generated explanation to the user, deepening their understanding.

[1250] Prompt Sentence Examples

[1251] "If a user types in 'I want to learn about the basics of fractions,' please provide appropriate lesson content that takes their feelings into consideration. Also, explain how to respond if they have additional questions."

[1252] With the above-described configuration, the present invention can provide a personalized learning experience that takes into account the user's emotional state, thereby improving learning efficiency.

[1253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1254] Step 1:

[1255] The user inputs a content request. For example, the user sends a request to the device using keyboard or voice input, such as "I want to learn about the basics of fractions." The input request is sent to the server by the device in real time.

[1256] Step 2:

[1257] The device collects and analyzes emotion data along with the user's request. The device captures the user's facial expressions and voice and analyzes them using emotion recognition means. The analyzed emotion data includes information on whether the user is relaxed or anxious, and transmits this data to the server.

[1258] Step 3:

[1259] The server searches for lesson content based on the received request and emotional data. The server extracts lesson content appropriate for the target age, subject, and emotional state from the data storage means. For example, video content tagged with "fractions," "basic," and "relaxation" is extracted. The server then transmits this to the terminal.

[1260] Step 4:

[1261] The terminal displays the lesson content received from the server. The display means of the terminal plays the video and provides it to the user. The user studies while watching this video.

[1262] Step 5:

[1263] While viewing lesson content, the user can input an additional question. For example, "I don't know the difference between 1 / 2 and 1 / 3." In this case, the device's emotion engine detects anxiety from the user's facial expression and voice, and sends this emotion data to the server.

[1264] Step 6:

[1265] The server uses generative AI to generate an answer based on the additional questions and emotional data. The server inputs the prompt into a generative AI model (e.g., GPT-3) to generate a specific and gentle answer. For example, an explanation such as "1 / 2 is one of two parts, and 1 / 3 is one of three parts" is generated.

[1266] Step 7:

[1267] The device displays the generated answer to the user. The explanation generated by the generative AI is displayed to the user via the display means, allowing the user to gain additional knowledge.

[1268] The above processing steps enable real-time analysis of the user's emotional state and the provision of individually optimized educational content and support based on that analysis.

[1269] (Application example 2)

[1270] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1271] Conventional educational systems provide uniform lesson content without considering the user's emotional state, which can make it difficult for users to fully understand the learning content. Furthermore, when users ask questions or make additional requests, appropriate responses based on their emotions are not provided, resulting in lower user satisfaction. In particular, there is a need for systems that can respond to situations where face-to-face customer support is difficult, such as in real-world environments such as brick-and-mortar stores.

[1272] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1273] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using generative artificial intelligence, an emotion recognition means for recognizing the user's emotions, and an adjustment means for dynamically adjusting the lesson content provided and the generated explanations based on the recognized emotions, thereby making it possible to provide more personalized educational support based on the user's emotions.

[1274] Creating definition statements

[1275] "Course content" is a collective term for digital materials such as text, videos, images, and interactive assignments used for educational purposes.

[1276] "Database means" is a system for organizing, storing, and managing lesson content.

[1277] "Input means" refers to a device or function that provides an interface for users to input requests to the system, and includes keyboards, voice input, touch operations, etc.

[1278] The "server means" is a server or computer system having the function of searching a database for and providing the most suitable lesson content based on a user request received from an input means.

[1279] "Display means" refers to a device or interface for visually presenting lesson content to users, and includes monitors, smartphones, tablets, etc.

[1280] An "interaction means" is a device or function that allows two-way communication for users to ask questions or make additional requests.

[1281] The "generation means" is a computer system that has the function of using generative artificial intelligence to generate explanations and supplementary materials in response to user questions and additional requests.

[1282] "Emotion recognition means" refers to a device or software that analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes their state.

[1283] The "adjustment means" is a system that has the functionality to dynamically adjust the lesson content provided and the explanations generated based on the recognized emotions.

[1284] MODE FOR CARRYING OUT THE INVENTION

[1285] System configuration

[1286] The system of the present invention consists of three main elements: a server device, a terminal device, and a user. The server device stores and manages lesson content and provides appropriate content in response to user requests. The terminal device is a device through which the user interacts with the system and has input and display functions. The user is the entity that acquires educational content through the system and engages in learning. The system is equipped with an emotion engine that can recognize the user's emotions and dynamically change the interaction and content delivery method based on those emotions.

[1287] Server device functions

[1288] The server device includes the following main functions:

[1289] Database management: The server stores lesson contents provided by various educators in a database and tags them based on target age, subject, and lesson format.

[1290] Content search and provision: The server receives a user request, searches the database for the most appropriate lesson content, and provides it. The provided content is sent to the terminal and displayed to the user.

[1291] Use of generative AI: Generative AI (e.g., GPT) is used to generate supplementary material and explanations based on user questions and additional requests. For example, in response to a request such as "Tell me more about adding fractions," generative AI can generate specific explanations and examples.

[1292] Emotion Engine Functions

[1293] The emotion engine has the following features:

[1294] Emotion recognition: Analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes the user's current emotional state based on the emotional data.

[1295] Emotion-based interaction: The generative AI will adjust the explanations and writing style it provides depending on the user's emotions. For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific and easy-to-understand examples and explanations in a gentler tone.

[1296] Terminal device functions

[1297] The terminal device has the following functions:

[1298] User interface: The terminal provides an interface for users to input requests. Keyboard input, voice input, and touch operation are possible. It also has an interface for displaying lesson content and generated materials.

[1299] Use of emotion engine: The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time. The emotion engine adjusts the content provided by the server based on this information.

[1300] User operations

[1301] The user does the following:

[1302] Requesting content: The user uses the device to request what they want to learn and browses the lesson content provided by the server in response to the request. For example, they might type, "Teach me how to solve quadratic equations."

[1303] Emotional feedback: Users can input their emotions into the device or express them spontaneously while learning. The emotion engine analyzes this and the server optimizes the content provided.

[1304] Specific examples

[1305] Example 1: Emotion-based learning adjustment

[1306] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[1307] 2. The device sends this request to the server, analyzes the user's emotional state from their facial expressions and voice, and sends the emotional data to the server.

[1308] 3. The server searches the database for the most appropriate lesson content based on the request and emotion data and sends it to the device.

[1309] 4. The device displays the lesson content to the user.

[1310] 5. The user asks a further question while viewing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3," and the emotion engine detects the sense of anxiety.

[1311] 6. The server uses generative AI to generate gentle explanations in response to the user's follow-up questions and sends them to the device.

[1312] 7. The terminal displays the generated explanation to the user, who then studies it.

[1313] Prompt Sentence Examples

[1314] Q: Does this restaurant have a menu for peanut allergies?

[1315] Emotion: Confused

[1316] answer:

[1317] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1318] Program processing flow

[1319] Step 1:

[1320] The user inputs a request to the terminal. The input method is keyboard input or voice input. This input includes a request such as "I want to learn the basics of fractions." The input data is sent to the server by the terminal.

[1321] Step 2:

[1322] The device captures the user's facial expressions and voice using a camera and microphone and sends them to the emotion recognition means. The emotion recognition means analyzes the voice and facial expression data to recognize the user's emotional state. The data used here is voice data and image data, and the emotional state (e.g., anxiety, joy, confusion, etc.) is output as the analysis result.

[1323] Step 3:

[1324] The server uses a database management means to search for the most suitable lesson content based on the received request and emotional data. The input data includes the request and the emotional state. The server extracts the target content from the database and selects it as the data to be sent to the terminal.

[1325] Step 4:

[1326] The server uses generative AI as needed to generate explanations and supplemental materials based on the user's request and emotional state. The server inputs the request and emotional state as prompts and uses a generative AI model (e.g., GPT-3) to generate appropriate explanations. The generated content is output in text format and sent to the device.

[1327] Step 5:

[1328] The terminal displays the lesson content received from the server and the generated explanations. The display means is a monitor or a smartphone screen. The content is presented to the user visually.

[1329] Step 6:

[1330] The user browses the lesson content and enters any additional questions or requests they may have. At this time, the device sends the input data to the server again, and at the same time sends the user's new emotional state to the emotion recognition means. The new input data and emotional data are analyzed again, and the server takes appropriate action.

[1331] Step 7:

[1332] In response to new questions or requests, the server again uses generative AI to generate answers and explanations and send them to the device, and this process is repeated until the user is fully trained.

[1333] Specific operation example

[1334] Step 1:

[1335] User: Says, "Tell me more about this book."

[1336] Step 2:

[1337] The device transmits the voice data to the server, and at the same time captures the user's facial expression with a camera and transmits it to the emotion recognition means, which then detects "interest" from the user's facial expression.

[1338] Step 3:

[1339] The server receives the request and the emotional state "interest," searches the database for detailed information about the related book, and sends it to the terminal.

[1340] Step 4:

[1341] Use generative AI to generate text like "This book has the following features:..."

[1342] Step 5:

[1343] The terminal displays the received information on the screen so that the user can view it.

[1344] Step 6:

[1345] The user enters a follow-up question: "Please tell me more specifically about the contents of each chapter."

[1346] Step 7:

[1347] The server sends additional questions and the emotional state "interest" as prompts to the generative AI, which then generates a detailed explanation and sends it back to the device. This process is repeated.

[1348] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1349] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1350] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1351] [Fourth embodiment]

[1352] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1353] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1354] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1355] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1356] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1357] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1358] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1359] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1360] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1361] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1362] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1363] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1364] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1365] The present invention relates to a system for effectively providing lesson content in the field of education and for assisting users in answering questions and learning. Specific embodiments of the system will be described below.

[1366] System Overview

[1367] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[1368] server

[1369] The server has the following functions:

[1370] Database Management:

[1371] The server stores lesson content from teachers all over the world, from elementary school teachers to university professors, in a database that includes videos, text materials, interactive assignments, and more.

[1372] The server structures the lesson content and tags each one, including the target age group, subject, and lesson format.

[1373] Content search and delivery:

[1374] The server receives requests from users and searches the database for appropriate lesson content to provide, including requests in text and audio formats.

[1375] The server transmits the lesson content to the user's terminal so that it can be displayed.

[1376] Use of generative AI:

[1377] The server uses generative AI (e.g., GPT) to generate additional explanations and supporting materials in response to the user's question.

[1378] For example, if a user requests a detailed explanation of adding fractions, the generative AI will write out concrete examples and step-by-step instructions.

[1379] Terminal

[1380] The terminal has the following functions:

[1381] User Interface:

[1382] The terminal provides an interface for users to input requests using keyboard input, voice input, touch operation, and other methods.

[1383] The terminal has an interface for displaying lesson content and generated materials.

[1384] Viewing and Interaction:

[1385] The device plays the lesson content received from the server and allows users to view it. It is possible to play videos, display text, and complete interactive assignments.

[1386] The device provides interaction capabilities for users to ask questions or make additional requests, for example, through a chatbot or voice assistant.

[1387] User

[1388] The user performs the following operations:

[1389] Request content:

[1390] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[1391] After making a question or request, the user views the lesson content provided by the server.

[1392] Learning and feedback:

[1393] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[1394] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[1395] Specific examples

[1396] 1. Learning basic mathematics

[1397] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[1398] Step 2: The device sends a request to the server.

[1399] Step 3: The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[1400] Step 4: The device displays the lesson content to the user, who plays the video and views the materials.

[1401] Step 5: The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[1402] Step 6: The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[1403] Step 7: The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[1404] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[1405] The processing flow will be explained below.

[1406] Step 1:

[1407] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[1408] Step 2:

[1409] The device sends the user's request to the server as an HTTP request, which also includes the user's profile information and learning history.

[1410] Step 3:

[1411] The server analyzes the received HTTP request and identifies the lesson content (in this case, the basics of fractions) that the user is looking for from the request content.

[1412] Step 4:

[1413] The server queries the database to find the relevant lesson content, including the results of text analysis, target age group, learning history, etc.

[1414] Step 5:

[1415] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[1416] Step 6:

[1417] The server embeds the acquired lesson content in an HTTP response and returns it to the terminal.

[1418] Step 7:

[1419] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[1420] Step 8:

[1421] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[1422] Step 9:

[1423] The terminal receives the user's additional questions and sends them back to the server.

[1424] Step 10:

[1425] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials (e.g., "Explain the difference between 1 / 2 and 1 / 3").

[1426] Step 11:

[1427] The server returns the generated answer to the terminal as an HTTP response.

[1428] Step 12:

[1429] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[1430] Step 13:

[1431] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[1432] Step 14:

[1433] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[1434] Through this series of steps, users can effectively advance their learning and contribute to improving the quality of the entire system.

[1435] Example 1

[1436] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1437] Conventional educational systems have had problems with the inability to provide lesson content or respond to users' follow-up questions quickly and effectively, resulting in reduced learning efficiency. Furthermore, it has been difficult to provide appropriate responses to user requests and generate supplementary materials, resulting in uneven quality of education.

[1438] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1439] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using a generative artificial intelligence model, a communication means for sending the user's request to the server in the form of an HTTP request, and a communication means for sending the generated explanations and supplementary materials to the terminal. This enables effective learning support by responding quickly and appropriately to the user's learning requests and providing advanced supplementary explanations using the generative artificial intelligence model.

[1440] "Class Content" means information or materials used for educational purposes, including videos, text, interactive assignments, etc.

[1441] "Database means" refers to a system or software for storing and managing lesson content in digital form.

[1442] "Input means" refers to a device or interface that allows a user to input requests or questions to the system, and includes a keyboard, microphone, touch screen, etc.

[1443] The "server means" is a computer system that searches a database for lesson content based on a user request and provides appropriate content.

[1444] "Display means" refers to a device or interface for visually displaying the provided lesson content to the user, and includes a display, monitor, etc.

[1445] "Interaction means" refers to devices or interfaces through which users can ask questions or make additional requests, including chatbots, voice assistants, form filling, etc.

[1446] A "generative artificial intelligence model" is a machine learning model that uses natural language processing technology to automatically generate additional explanations and supplementary materials in response to user questions.

[1447] "Generation means" refers to a device or software that uses a generative artificial intelligence model to generate detailed answers and supplementary materials to a user's question.

[1448] "Communication means" refers to the devices and protocols used to transmit and receive user requests and generated responses between the server and the device, including internet connection, Wi-Fi, mobile data, etc.

[1449] The present invention relates to a system for effectively providing lesson content and assisting users in answering questions and learning. Specific embodiments of the present invention will be described below.

[1450] System Overview

[1451] This system consists of three main elements: a server, terminals, and users. The server stores and manages lesson content and provides appropriate content in response to user requests. Terminals are devices through which users interact with the system, inputting and displaying information. Users are the entities that use the system to obtain educational content and learn.

[1452] server

[1453] The server has the following functions:

[1454] Database Management:

[1455] The server manages the lesson content using a database. Specifically, it uses an SQL database (e.g., MySQL or PostgreSQL) to store videos, text materials, interactive assignments, etc. The lesson content is tagged with information such as target age, subject, and lesson format.

[1456] Content search and delivery:

[1457] The server receives a request from a user and searches the database for appropriate lesson content using a search algorithm, and the search results are sent to the terminal via an HTTP request.

[1458] Use of generative AI:

[1459] The server uses a generative artificial intelligence model (e.g., OpenAI's GPT-3) to generate additional explanations and supplementary materials for the user's question. For example, if a user requests a detailed explanation of adding fractions, the generative AI automatically generates concrete examples and step-by-step explanations.

[1460] Terminal

[1461] The terminal has the following functions:

[1462] User Interface:

[1463] The device provides an interface for users to input requests using keyboard input, voice input, touch input, a microphone with voice recognition capabilities, or a touchscreen display.

[1464] The terminal is equipped with a user interface for displaying lesson content and generated materials. The display includes video playback functions and a PDF viewer.

[1465] Viewing and Interaction:

[1466] The device plays the lesson content received from the server and allows users to view it. Specifically, it has a video playback application, a PDF viewer, and an interactive assignment display function.

[1467] The terminal provides an interaction function for the user to ask questions or make additional requests. For example, if the user types "What is the difference between 1 / 2 and 1 / 3?", the terminal sends the question to the server.

[1468] User

[1469] The user performs the following operations:

[1470] Request content:

[1471] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[1472] After making a question or request, the user views the lesson content provided by the server.

[1473] Learning and feedback:

[1474] Users can browse the lesson content and progress through their studies. If they have any questions or need additional explanation, they can ask the generative AI.

[1475] After completing the learning process, the user provides feedback to the system, which is used to improve the system and generate new content.

[1476] Specific examples

[1477] Learning basic mathematics

[1478] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[1479] 2. The device sends a request to the server.

[1480] 3. The server searches the database for lesson content (videos and materials) on "Fraction Basics" and sends it to the device.

[1481] 4. The device displays the lesson content to the user, who plays the video and views the materials.

[1482] 5. The user asks a follow-up question: "I don't know the difference between 1 / 2 and 1 / 3." The device sends this question to the server.

[1483] 6. The server uses generative AI to generate text explaining the difference between 1 / 2 and 1 / 3 and sends it to the device.

[1484] 7. The device displays the generated explanation to the user, allowing the user to deepen their understanding.

[1485] Prompt Sentence Examples

[1486] If the user requests a detailed explanation of adding fractions, the server sends the following prompt to the generative AI:

[1487] "Please explain fraction addition step by step with concrete examples."

[1488] In this way, the system can provide a high-quality learning experience by providing optimal lesson content in response to user requests and quickly responding to additional questions.

[1489] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1490] Step 1:

[1491] The user inputs what they want to learn on the device. For example, the user might input "I want to learn about the basics of fractions." The input is displayed in an input field on the device. Input can be done using keyboard input or voice input. The device converts the user's input into JSON format data.

[1492] Step 2:

[1493] The device sends the user's input to the server as an HTTP POST request. The input data is in JSON format and is sent to the server's receiving endpoint. At this time, the device uses a network communication module (e.g., Wi-Fi, mobile data). The server analyzes the received request data and extracts the request content.

[1494] Step 3:

[1495] The server uses a database management system (e.g., MySQL) to search the database for lesson content based on the request. Specifically, it executes an SQL query to retrieve lesson content that matches tags or keywords. The search results include the lesson content's metadata and URL.

[1496] Step 4:

[1497] The server converts the acquired lesson content into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received JSON data and extracts the URL and metadata of the lesson content.

[1498] Step 5:

[1499] To display the lesson content to the user, the device launches a video playback application or PDF viewer. Specifically, for video content, a play button is provided, and the video starts when the user clicks it. For text materials or interactive assignments, the appropriate viewer is launched to display the content.

[1500] Step 6:

[1501] The user views the lesson content and, if they have any additional questions, they enter them again in the input field on their device. For example, they might enter, "I don't understand the difference between 1 / 2 and 1 / 3." The input data is again converted to JSON format.

[1502] Step 7:

[1503] The device then sends the additional question to the server as an HTTP POST request, which then analyzes the received question, extracts the specific question, and prepares to send a prompt to the generative AI model based on the question.

[1504] Step 8:

[1505] The server uses a generative AI model (e.g., GPT-3) to generate a detailed answer to the user's question. Specifically, it sends the following prompt to the generative AI model: "Please explain the difference between 1 / 2 and 1 / 3 step by step with concrete examples." The generative AI model then generates an answer text based on this prompt.

[1506] Step 9:

[1507] The server converts the generated response text into JSON format and sends it back to the terminal as an HTTP response. The terminal then analyzes the received JSON data and extracts the generated response text.

[1508] Step 10:

[1509] The terminal displays the generated answer text to the user. Specifically, the text is displayed in a chat window or text viewer. The user can deepen their understanding by viewing the answer. In this way, a quick and appropriate answer to the user's question is provided.

[1510] (Application example 1)

[1511] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1512] Conventional virtual store systems often lack detailed product information and usage instructions that users require, and do not offer real-time question-answering or interactive experiences, which can lead to reduced user satisfaction. Furthermore, it is difficult to incorporate effective information provision and question-answering services that utilize generative artificial intelligence. Our goal is to solve these issues and improve the user experience in virtual stores.

[1513] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1514] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, and a server means for searching and providing lesson content based on the requests. This enables a virtual store means for providing product information and offering users an interactive product experience, and a means for answering questions about products using generative artificial intelligence.

[1515] "Database means" refers to a system for storing lesson content and product information, and enabling efficient search and management.

[1516] An "input means" is a device or interface that allows a user to input a request, and can accept voice or text input.

[1517] The "server means" is a server that searches for and provides appropriate content and information from a database based on a request from a user.

[1518] "Display means" refers to a device or interface for displaying the provided lesson content and product information to the user.

[1519] An "interaction means" is a system that provides a dialogue facility for users to ask questions or make additional requests.

[1520] "Generation means" refers to a system for generating additional explanations and supplementary materials using generative artificial intelligence.

[1521] A "virtual storefront" is a system for providing product information and offering users an interactive product experience.

[1522] "Generative AI" is an AI technology that uses natural language processing technology to analyze user input and generate appropriate responses and information.

[1523] "Product Information" refers to information such as detailed product descriptions, instructions for use, and reviews.

[1524] "User" refers to an individual or organization that uses this system.

[1525] MODE FOR CARRYING OUT THE INVENTION

[1526] The present invention relates to a system for effectively providing product information and responding to user product-related questions in real time. The system is comprised of three parties: a server, a terminal, and a user. Specific embodiments are described below.

[1527] server

[1528] The server has the following functions:

[1529] 1. Database Management:

[1530] It stores a wide range of product information from around the world in a database that includes text, images, videos, interactive reviews, and more.

[1531] Product information is tagged for efficient searching and management, including product categories, features, and use cases.

[1532] 2. Content Search and Delivery:

[1533] It receives a request from a user, searches for appropriate product information from a database, and provides it to the terminal.

[1534] Requests can come in voice or text form.

[1535] 3. Use of generative AI:

[1536] It has the ability to generate answers to user questions using natural language processing technology.

[1537] For example, in response to a question such as "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[1538] Terminal

[1539] The terminal has the following functions:

[1540] 1. User Interface:

[1541] It provides an interface for users to input requests using keyboard, voice, touch, etc.

[1542] It provides an interface for displaying the provided product information and generated answers.

[1543] 2. Viewing and Interaction:

[1544] The product information received from the server is reproduced and made available for viewing by the user.

[1545] Provide interaction features for users to ask questions or make additional requests, for example, by using a chatbot or voice assistant.

[1546] User

[1547] The user does the following:

[1548] 1. Product Information Request:

[1549] Use your device to request product information, for example, "Tell me about the features of this phone."

[1550] After making a question or request, view the product information provided by the server.

[1551] 2. Questions and Feedback:

[1552] Browse product information and learn about uses and features.

[1553] If they don't understand something or need further clarification, they ask the generative AI a question, using prompts such as "Tell me how to clean this coffee maker."

[1554] After the learning is complete, feedback is provided to the system and used to improve the system and generate new content.

[1555] Hardware and software used

[1556] Hardware:

[1557] Smartphone (iOS or Android)

[1558] software:

[1559] Framework: Flask (build server API with Python)

[1560] Database: PostgreSQL (manages product information and user queries)

[1561] Generative AI: OpenAI GPT-3 API (detailed product question and answering)

[1562] This allows the system to provide optimal product information in response to user requests and provide detailed question and answer responses using a generative AI model, providing a high-quality user experience.

[1563] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1564] Step 1:

[1565] The user uses the input means of the smartphone to input product information or a question.

[1566] Input methods include keyboard input, voice input, touch operation, etc. The input request is sent to the device.

[1567] Step 2:

[1568] The terminal sends a request to the server, and the server searches the database based on the request.

[1569] Requests include product specific information and questions, and the database stores tagged product information for efficient searching.

[1570] Step 3:

[1571] The server searches the database for the appropriate product information and returns the results to the terminal.

[1572] This process retrieves relevant text, images, videos, etc. from the database, combines them as needed, and sends them to the device, providing the detailed product information requested by the user.

[1573] Step 4:

[1574] The terminal receives the information from the server and displays it on the user interface.

[1575] The received information may include text materials, images, videos, etc., and the device will display them in an appropriate format for the user to view.

[1576] Step 5:

[1577] The user enters a follow-up question based on the product information.

[1578] For example, you can type "Tell me how to use this product." This request is sent again from the device to the server.

[1579] Step 6:

[1580] The server uses a generative AI model (using natural language processing technology) to generate answers to the user's questions.

[1581] Specifically, it analyzes user input, creates prompts to generate appropriate answers, and then makes API requests to the generative AI. For example, in response to a question like, "How do I clean this coffee maker?", it generates detailed cleaning instructions.

[1582] Step 7:

[1583] The generated answer is sent from the server to the terminal and displayed on the user interface.

[1584] Users can view detailed answers and explanations to deepen their understanding. The generated answers include specific steps and precautions.

[1585] These steps allow users to access detailed product information in a virtual store and get real-time answers to their product-related questions.

[1586] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1587] The present invention relates to a system for providing lesson content, answering users' questions, and assisting learning in the field of education. In particular, the system aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. Specific embodiments of the system are described below.

[1588] System Overview

[1589] This system consists of three main elements: a server, a terminal, and a user. The server stores and manages lesson content and provides appropriate content in response to user requests. The terminal is a device through which users interact with the system, and has input and display functions. Users are the entities that acquire educational content through the system and engage in learning. The system also uses an emotion engine to recognize user emotions and dynamically change the way interactions and content are provided based on those emotions.

[1590] server

[1591] The server has the following functions:

[1592] Database Management:

[1593] The server stores a database of various lesson content provided by educators from elementary school to university, including videos, texts, and interactive assignments.

[1594] Lesson content is tagged based on target age, subject, and lesson format.

[1595] Content search and delivery:

[1596] The server receives a user request, searches the database for the most suitable lesson content, and provides it.

[1597] The lesson content is transmitted to the user's terminal and displayed.

[1598] Use of generative AI:

[1599] The server uses generative AI (e.g., GPT) to generate supporting materials and explanations based on the user's questions and additional requests.

[1600] For example, if a user requests, "Teach me more about adding fractions," the generative AI will generate specific explanations and examples.

[1601] Emotion Engine

[1602] The emotion engine has the following functions:

[1603] Emotion recognition:

[1604] Analyzes emotions from the user's voice, facial expressions, text input, etc.

[1605] Based on the emotion data, the user's current emotional state is recognized.

[1606] Emotion-based interactions:

[1607] The generative AI adjusts the explanations and writing style provided based on the user's emotions.

[1608] For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[1609] Terminal

[1610] The terminal has the following functions:

[1611] User Interface:

[1612] The device provides an interface for users to input requests, which can be keyboard input, voice input, or touch operation.

[1613] It has an interface for displaying lesson content and generated materials.

[1614] Using the Emotion Engine:

[1615] The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time.

[1616] The emotion engine adjusts the content provided by the server based on this.

[1617] User

[1618] The user performs the following operations:

[1619] Request content:

[1620] The user uses the terminal to request what they want to learn, for example, by typing "Teach me how to solve quadratic equations."

[1621] View the lesson content provided by the server in response to the request.

[1622] Emotional feedback:

[1623] Users can input their emotions into the device or express them spontaneously while studying. The emotion engine analyzes these emotions, and the server optimizes the content provided.

[1624] Specific examples

[1625] 1. Emotion-based learning regulation

[1626] Step 1: User types "I want to learn about the basics of fractions" into the terminal.

[1627] Step 2: The device sends this request to the server. At the same time, it analyzes the user's emotional state from their facial expressions and voice and sends the emotional data to the server.

[1628] Step 3: The server searches the database for the most suitable lesson content based on the received request and emotion data, and sends it to the terminal.

[1629] Step 4: The device displays the lesson content to the user.

[1630] Step 5: The user asks a further question while browsing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3." At the same time, the emotion engine detects anxiety in the user's voice.

[1631] Step 6: The server uses generative AI to generate a friendly explanation for the user's follow-up questions and sends it to the device.

[1632] Step 7: The device displays the generated explanation to the user, who can study it and deepen their understanding.

[1633] In this way, the system can provide a personalized learning experience that takes into account the user's emotional state, enabling a user-centered educational environment.

[1634] The processing flow will be explained below.

[1635] Step 1:

[1636] The user types or speaks into the device, "I want to learn about the basics of fractions." The device receives this request and converts the speech to text as needed.

[1637] Step 2:

[1638] The device analyzes the user's emotional state from their facial expressions and voice, along with their request. The emotion engine determines the user's current emotional state and stores the result as data.

[1639] Step 3:

[1640] The device sends the user's request and emotional data to the server as an HTTP request, which also includes the user's profile information and learning history.

[1641] Step 4:

[1642] The server analyzes the received HTTP request and identifies the lesson content the user is looking for based on the request content and emotional data.

[1643] Step 5:

[1644] The server queries the database to find relevant lesson content, including text analysis results, target age group, learning history, and emotional data.

[1645] Step 6:

[1646] The server retrieves the lesson content appropriate for the request from the database. In this case, the lesson video and supplementary materials for "Fractions Basics" apply.

[1647] Step 7:

[1648] The server uses generative AI to adjust the lesson content based on the emotional data. For example, if the emotional data indicates that the user is tired, it generates a short, concise explanation.

[1649] Step 8:

[1650] The server embeds the adjusted lesson content into an HTTP response and sends it to the terminal.

[1651] Step 9:

[1652] The device receives the response from the server and displays the lesson content to the user, such as playing videos and displaying text materials.

[1653] Step 10:

[1654] Users can view the lesson content and progress through their studies. If they have difficulty understanding any part, they can enter additional questions into the device.

[1655] Step 11:

[1656] The device receives the user's additional question and sends it back to the server, along with the additional emotion data.

[1657] Step 12:

[1658] The server analyzes the user's follow-up questions and uses generative AI to generate appropriate answers and supplementary materials. The emotion engine then analyzes the user's emotional state again and adjusts the content accordingly.

[1659] Step 13:

[1660] The server returns the generated answer to the terminal as an HTTP response.

[1661] Step 14:

[1662] The terminal displays the generated answer from the server to the user, who reads the answer and deepens their understanding.

[1663] Step 15:

[1664] After completing the learning, the user provides feedback to the system through the terminal, which then sends the feedback to the server.

[1665] Step 16:

[1666] The server stores the received feedback in a database and uses it to improve the system and generate new content.

[1667] Through this series of steps, users can learn effectively and receive support appropriate to their individual emotional state, improving the quality of the system as a whole.

[1668] Example 2

[1669] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1670] Conventional educational systems struggle to provide content that is tailored to each individual user because they lack the ability to consider the user's level of understanding and emotional state. Furthermore, conventional systems have limited ability to respond immediately to follow-up questions or requests from users. This can lead to problems such as confusion and insufficient understanding during learning. Therefore, there is a need for a system that can analyze the user's emotional state in real time, provide a personalized learning experience based on that information, and quickly provide appropriate learning support.

[1671] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a data storage means for saving lesson content, an input means for receiving a user request, a server means for searching for and providing lesson content based on the request, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations or supplementary materials using generative artificial intelligence, and an emotion recognition means for analyzing emotional data from the user's facial expressions and voice and adjusting the content to be provided based on the emotion. This makes it possible to analyze the user's emotional state and provide appropriate content based on that, thereby effectively supporting the user's learning.

[1672] "Class content" refers to videos, texts, interactive assignments, etc. used for educational purposes.

[1673] "Data storage means" refers to a database or storage system for saving and managing lesson content.

[1674] "Input means" refers to an interface that allows users to input what they want to learn or questions into the system, and includes keyboard input and voice input.

[1675] The "server means" refers to a computer system that receives a user's request, searches for appropriate lesson content from a database, and provides it.

[1676] "Display means" refers to a device for displaying lesson content provided by the server to the user, such as a monitor or screen.

[1677] "Interaction means" refers to the interface or method by which a user can ask questions or make additional requests.

[1678] "Generation means" is a function that uses generative artificial intelligence to generate supplementary materials and explanations in response to user questions and additional requests.

[1679] "Emotion recognition means" refers to a system or function that analyzes emotional data from the user's facial expressions and voice, and dynamically adjusts the content provided based on that data.

[1680] "Generative AI" refers to an AI model that generates sentences and explanations using natural language processing technology.

[1681] This invention is a system for providing lesson content, answering users' questions, and assisting learning in the field of education. Specifically, it aims to provide a more personalized learning experience by combining an emotion engine that recognizes the user's emotions and adjusts the content and interactions generated based on those emotions. This system is composed of the following elements.

[1682] server

[1683] The server has a data storage means for saving and managing lesson content. This data storage means stores various lesson content provided by educators from elementary school to university. This includes videos, texts, and interactive assignments. The lesson content is tagged based on target age, subject, and lesson format.

[1684] The server then receives the user's request using a content search means, searches the database for the most appropriate lesson content, and provides it.Furthermore, the server has a generation means that uses generative artificial intelligence (e.g., GPT-3) to generate supplementary materials and explanations based on the user's questions and additional requests.

[1685] Terminal

[1686] The terminal provides an input means for the user to input a request, which includes keyboard input, voice input, and touch operation. The terminal is equipped with a display means for displaying lesson content received from the server, thereby providing educational content to the user.

[1687] The device also uses an emotion engine to analyze the user's facial expressions and voice, and transmits emotional data to the server in real time, which then becomes the basis for dynamically adjusting the content provided by the server.

[1688] User

[1689] Users can access educational content through this system and study. Specifically, they use their devices to request the subject they want to learn, such as "Teach me how to solve quadratic equations." The server responds to this request and provides the most appropriate lesson content.

[1690] Users can also display or spontaneously express their emotions while learning. The emotion engine analyzes these and the server uses them to optimize content delivery. For example, if the user is recognized as "frustrated because they don't understand," the generative AI will provide more specific, easy-to-understand examples and explanations in a gentler tone.

[1691] Specific examples

[1692] For example, the following shows the sequence of events when a user inputs "I want to learn about the basics of fractions." The user inputs this request into the device, and the device sends this request along with the user's facial expression data to the server. Based on the received request and emotional data, the server searches its database for the most suitable lesson content and provides it to the device. The device displays this lesson content to the user, who then studies it.

[1693] Furthermore, if the user enters a follow-up question such as "I don't know the difference between 1 / 2 and 1 / 3," the emotion engine detects the user's anxiety. The server uses generative AI to generate a specific explanation in a gentle tone and provides it to the device. The device then displays this generated explanation to the user, deepening their understanding.

[1694] Prompt Sentence Examples

[1695] "If a user types in 'I want to learn about the basics of fractions,' please provide appropriate lesson content that takes their feelings into consideration. Also, explain how to respond if they have additional questions."

[1696] With the above-described configuration, the present invention can provide a personalized learning experience that takes into account the user's emotional state, thereby improving learning efficiency.

[1697] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1698] Step 1:

[1699] The user inputs a content request. For example, the user sends a request to the device using keyboard or voice input, such as "I want to learn about the basics of fractions." The input request is sent to the server by the device in real time.

[1700] Step 2:

[1701] The device collects and analyzes emotion data along with the user's request. The device captures the user's facial expressions and voice and analyzes them using emotion recognition means. The analyzed emotion data includes information on whether the user is relaxed or anxious, and transmits this data to the server.

[1702] Step 3:

[1703] The server searches for lesson content based on the received request and emotional data. The server extracts lesson content appropriate for the target age, subject, and emotional state from the data storage means. For example, video content tagged with "fractions," "basic," and "relaxation" is extracted. The server then transmits this to the terminal.

[1704] Step 4:

[1705] The terminal displays the lesson content received from the server. The display means of the terminal plays the video and provides it to the user. The user studies while watching this video.

[1706] Step 5:

[1707] While viewing lesson content, the user can input an additional question. For example, "I don't know the difference between 1 / 2 and 1 / 3." In this case, the device's emotion engine detects anxiety from the user's facial expression and voice, and sends this emotion data to the server.

[1708] Step 6:

[1709] The server uses generative AI to generate an answer based on the additional questions and emotional data. The server inputs the prompt into a generative AI model (e.g., GPT-3) to generate a specific and gentle answer. For example, an explanation such as "1 / 2 is one of two parts, and 1 / 3 is one of three parts" is generated.

[1710] Step 7:

[1711] The device displays the generated answer to the user. The explanation generated by the generative AI is displayed to the user via the display means, allowing the user to gain additional knowledge.

[1712] The above processing steps enable real-time analysis of the user's emotional state and the provision of individually optimized educational content and support based on that analysis.

[1713] (Application example 2)

[1714] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1715] Conventional educational systems provide uniform lesson content without considering the user's emotional state, which can make it difficult for users to fully understand the learning content. Furthermore, when users ask questions or make additional requests, appropriate responses based on their emotions are not provided, resulting in lower user satisfaction. In particular, there is a need for systems that can respond to situations where face-to-face customer support is difficult, such as in real-world environments such as brick-and-mortar stores.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1717] In this invention, the server includes a database means for storing lesson content, an input means for receiving user requests, a server means for searching for and providing lesson content based on the requests, a display means for displaying the provided lesson content, an interaction means for the user to ask questions or make additional requests, a generation means for generating additional explanations and supplementary materials using generative artificial intelligence, an emotion recognition means for recognizing the user's emotions, and an adjustment means for dynamically adjusting the lesson content provided and the generated explanations based on the recognized emotions, thereby making it possible to provide more personalized educational support based on the user's emotions.

[1718] Creating definition statements

[1719] "Course content" is a collective term for digital materials such as text, videos, images, and interactive assignments used for educational purposes.

[1720] "Database means" is a system for organizing, storing, and managing lesson content.

[1721] "Input means" refers to a device or function that provides an interface for users to input requests to the system, and includes keyboards, voice input, touch operations, etc.

[1722] The "server means" is a server or computer system having the function of searching a database for and providing the most suitable lesson content based on a user request received from an input means.

[1723] "Display means" refers to a device or interface for visually presenting lesson content to users, and includes monitors, smartphones, tablets, etc.

[1724] An "interaction means" is a device or function that allows two-way communication for users to ask questions or make additional requests.

[1725] The "generation means" is a computer system that has the function of using generative artificial intelligence to generate explanations and supplementary materials in response to user questions and additional requests.

[1726] "Emotion recognition means" refers to a device or software that analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes their state.

[1727] The "adjustment means" is a system that has the functionality to dynamically adjust the lesson content provided and the explanations generated based on the recognized emotions.

[1728] MODE FOR CARRYING OUT THE INVENTION

[1729] System configuration

[1730] The system of the present invention consists of three main elements: a server device, a terminal device, and a user. The server device stores and manages lesson content and provides appropriate content in response to user requests. The terminal device is a device through which the user interacts with the system and has input and display functions. The user is the entity that acquires educational content through the system and engages in learning. The system is equipped with an emotion engine that can recognize the user's emotions and dynamically change the interaction and content delivery method based on those emotions.

[1731] Server device functions

[1732] The server device includes the following main functions:

[1733] Database management: The server stores lesson contents provided by various educators in a database and tags them based on target age, subject, and lesson format.

[1734] Content search and provision: The server receives a user request, searches the database for the most appropriate lesson content, and provides it. The provided content is sent to the terminal and displayed to the user.

[1735] Use of generative AI: Generative AI (e.g., GPT) is used to generate supplementary material and explanations based on user questions and additional requests. For example, in response to a request such as "Tell me more about adding fractions," generative AI can generate specific explanations and examples.

[1736] Emotion Engine Functions

[1737] The emotion engine has the following features:

[1738] Emotion recognition: Analyzes emotions from the user's voice, facial expressions, text input, etc., and recognizes the user's current emotional state based on the emotional data.

[1739] Emotion-based interaction: The generative AI will adjust the explanations and writing style it provides depending on the user's emotions. For example, if the generative AI recognizes that the user is frustrated because they don't understand something, it will provide more specific and easy-to-understand examples and explanations in a gentler tone.

[1740] Terminal device functions

[1741] The terminal device has the following functions:

[1742] User interface: The terminal provides an interface for users to input requests. Keyboard input, voice input, and touch operation are possible. It also has an interface for displaying lesson content and generated materials.

[1743] Use of emotion engine: The device analyzes the user's facial expressions and voice and transmits their emotional state to the server in real time. The emotion engine adjusts the content provided by the server based on this information.

[1744] User operations

[1745] The user does the following:

[1746] Requesting content: The user uses the device to request what they want to learn and browses the lesson content provided by the server in response to the request. For example, they might type, "Teach me how to solve quadratic equations."

[1747] Emotional feedback: Users can input their emotions into the device or express them spontaneously while learning. The emotion engine analyzes this and the server optimizes the content provided.

[1748] Specific examples

[1749] Example 1: Emotion-based learning adjustment

[1750] 1. A user types "I want to learn about the basics of fractions" into a terminal.

[1751] 2. The device sends this request to the server, analyzes the user's emotional state from their facial expressions and voice, and sends the emotional data to the server.

[1752] 3. The server searches the database for the most appropriate lesson content based on the request and emotion data and sends it to the device.

[1753] 4. The device displays the lesson content to the user.

[1754] 5. The user asks a further question while viewing the lesson content. For example, "I don't know the difference between 1 / 2 and 1 / 3," and the emotion engine detects the sense of anxiety.

[1755] 6. The server uses generative AI to generate gentle explanations in response to the user's follow-up questions and sends them to the device.

[1756] 7. The terminal displays the generated explanation to the user, who then studies it.

[1757] Prompt Sentence Examples

[1758] Q: Does this restaurant have a menu for peanut allergies?

[1759] Emotion: Confused

[1760] answer:

[1761] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1762] Program processing flow

[1763] Step 1:

[1764] The user inputs a request to the terminal. The input method is keyboard input or voice input. This input includes a request such as "I want to learn the basics of fractions." The input data is sent to the server by the terminal.

[1765] Step 2:

[1766] The device captures the user's facial expressions and voice using a camera and microphone and sends them to the emotion recognition means. The emotion recognition means analyzes the voice and facial expression data to recognize the user's emotional state. The data used here is voice data and image data, and the emotional state (e.g., anxiety, joy, confusion, etc.) is output as the analysis result.

[1767] Step 3:

[1768] The server uses a database management means to search for the most suitable lesson content based on the received request and emotional data. The input data includes the request and the emotional state. The server extracts the target content from the database and selects it as the data to be sent to the terminal.

[1769] Step 4:

[1770] The server uses generative AI as needed to generate explanations and supplemental materials based on the user's request and emotional state. The server inputs the request and emotional state as prompts and uses a generative AI model (e.g., GPT-3) to generate appropriate explanations. The generated content is output in text format and sent to the device.

[1771] Step 5:

[1772] The terminal displays the lesson content received from the server and the generated explanations. The display means is a monitor or a smartphone screen. The content is presented to the user visually.

[1773] Step 6:

[1774] The user browses the lesson content and enters any additional questions or requests they may have. At this time, the device sends the input data to the server again, and at the same time sends the user's new emotional state to the emotion recognition means. The new input data and emotional data are analyzed again, and the server takes appropriate action.

[1775] Step 7:

[1776] In response to new questions or requests, the server again uses generative AI to generate answers and explanations and send them to the device, and this process is repeated until the user is fully trained.

[1777] Specific operation example

[1778] Step 1:

[1779] User: Says, "Tell me more about this book."

[1780] Step 2:

[1781] The device transmits the voice data to the server, and at the same time captures the user's facial expression with a camera and transmits it to the emotion recognition means, which then detects "interest" from the user's facial expression.

[1782] Step 3:

[1783] The server receives the request and the emotional state "interest," searches the database for detailed information about the related book, and sends it to the terminal.

[1784] Step 4:

[1785] Use generative AI to generate text like "This book has the following features:..."

[1786] Step 5:

[1787] The terminal displays the received information on the screen so that the user can view it.

[1788] Step 6:

[1789] The user enters a follow-up question: "Please tell me more specifically about the contents of each chapter."

[1790] Step 7:

[1791] The server sends additional questions and the emotional state "interest" as prompts to the generative AI, which then generates a detailed explanation and sends it back to the device. This process is repeated.

[1792] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1793] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1794] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1795] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1796] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1797] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1798] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1799] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1800] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1801] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1802] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1803] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1804] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1805] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1806] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1807] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1808] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1809] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1810] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1811] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1812] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1813] The following is further disclosed regarding the above embodiment.

[1814] (Claim 1)

[1815] A database means for storing lesson content;

[1816] an input means for receiving a user request;

[1817] a server means for searching for and providing lesson content based on the request;

[1818] a display means for displaying the provided lesson content;

[1819] an interaction means for the user to ask questions or make additional requests;

[1820] A generating means for generating additional explanations and supplementary materials using generative artificial intelligence;

[1821] A system including:

[1822] (Claim 2)

[1823] 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

[1824] (Claim 3)

[1825] 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence to analyze the user's question by natural language processing.

[1826] "Example 1"

[1827] (Claim 1)

[1828] A database means for storing lesson content;

[1829] an input means for receiving a user request;

[1830] a server means for searching for and providing lesson content based on the request;

[1831] a display means for displaying the provided lesson content;

[1832] an interaction means for the user to ask questions or make additional requests;

[1833] generating means for generating additional explanations and supplemental materials using a generative artificial intelligence model;

[1834] a communication means for transmitting a user request to a server in the form of an HTTP request;

[1835] a communication means for transmitting the generated explanation and supplementary materials to the terminal;

[1836] A system including:

[1837] (Claim 2)

[1838] 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

[1839] (Claim 3)

[1840] 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence model to analyze the user's question by natural language processing.

[1841] "Application Example 1"

[1842] (Claim 1)

[1843] A database means for storing lesson content;

[1844] an input means for receiving a user request;

[1845] a server means for searching for and providing lesson content based on the request;

[1846] a display means for displaying the provided lesson content;

[1847] an interaction means for the user to ask questions or make additional requests;

[1848] A generating means for generating additional explanations and supplementary materials using generative artificial intelligence;

[1849] a virtual storefront means for providing product information and providing users with an interactive product experience;

[1850] a means for answering questions about a product using generative artificial intelligence;

[1851] A system including:

[1852] (Claim 2)

[1853] 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

[1854] (Claim 3)

[1855] 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence to analyze the user's question by natural language processing.

[1856] "Example 2: Combining Emotion Engines"

[1857] (Claim 1)

[1858] a data storage means for storing lesson content;

[1859] an input means for receiving a user request;

[1860] a server means for searching for and providing lesson content based on the request;

[1861] a display means for displaying the provided lesson content;

[1862] an interaction means for the user to ask questions or make additional requests;

[1863] A generating means for generating additional explanations and supplementary materials using generative artificial intelligence;

[1864] an emotion recognition means for analyzing emotion data from a user's facial expression or voice and adjusting content to be provided based on the emotion;

[1865] A system including:

[1866] (Claim 2)

[1867] 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

[1868] (Claim 3)

[1869] 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence to analyze the user's question by natural language processing.

[1870] (Claim 4)

[1871] 2. The system according to claim 1, further comprising, as the emotion recognition means, means for analyzing the user's facial expressions and voice in real time and transmitting the emotion data to the server.

[1872] "Application example 2 when combining emotion engines"

[1873] (Claim 1)

[1874] A database means for storing lesson content;

[1875] an input means for receiving a user request;

[1876] a server means for searching for and providing lesson content based on the request;

[1877] a display means for displaying the provided lesson content;

[1878] an interaction means for the user to ask questions or make additional requests;

[1879] A generating means for generating additional explanations and supplementary materials using generative artificial intelligence;

[1880] emotion recognition means for recognizing an emotion of a user;

[1881] an adjusting means for dynamically adjusting the lesson content provided and the generated explanations based on the recognized emotions;

[1882] A system including:

[1883] (Claim 2)

[1884] 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

[1885] (Claim 3)

[1886] 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence to analyze the user's question by natural language processing. [Explanation of symbols]

[1887] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A database means for storing lesson content; an input means for receiving a user request; a server means for searching for and providing lesson content based on the request; a display means for displaying the provided lesson content; an interaction means for the user to ask questions or make additional requests; A generating means for generating additional explanations and supplementary materials using generative artificial intelligence; A system including:

2. 2. The system according to claim 1, wherein the input means includes an input means having a voice recognition function.

3. 2. The system according to claim 1, wherein the generating means includes means for the generative artificial intelligence to analyze the user's question by natural language processing.

Images