system

Abstract

Provide a system. [Solution] a means for accepting subject and area selections from learners; means for generating questions based on the subject and scope; a means for providing the generated questions to the learner; a means for receiving answers from learners; a means for determining whether the received answers are correct; A means for providing the assessment result to the learner; A means for identifying learners' weaknesses by analyzing their answer history; a means for generating original questions based on the identified weaknesses; A means for providing the generated original questions to the learner; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Amidst the reality of soaring cram school fees, declining academic achievement, and busy parents unable to find time to support their children's learning, there is a social demand for efficient and individually tailored learning support. Under these circumstances, conventional learning support systems have struggled to effectively identify and reinforce learners' weaknesses. Furthermore, they lack the functionality to individually adjust the suitability of learning content, making it difficult for general learning curricula to flexibly address specific weaknesses. Therefore, providing effective, individually customized learning support systems for each learner is a challenge. [Means for solving the problem]

[0005] The present invention provides a system that accepts a learner's selection of subject and area, generates questions based on this, and provides them to the learner. Furthermore, the system receives the learner's answers, judges whether they are correct, and provides feedback on the results. The system also includes a means for analyzing the answer history, identifying the learner's weak points, and generating original questions based on the analysis results. The original questions are provided to the learner, and repeated answering can enhance the effectiveness of learning. In particular, the system stores the learner's answer history in a database and uses the analysis results to provide a learning plan specialized for strengthening weak points, thereby achieving optimal learning support for each learner. Furthermore, the system aims to improve the efficiency and effectiveness of learning by adjusting the question generation algorithm and difficulty level, and providing a customized learning plan based on the answer history.

[0006] "Student" refers to a person who uses the system to carry out learning activities.

[0007] A "subject" refers to a specific area of ​​knowledge or academic field that a learner is studying.

[0008] "Scope" refers to the specific segment or topic that a learner wants to study within a particular subject.

[0009] "Question" refers to a question or task provided for a learner to answer.

[0010] "Solution" refers to the answer or solution provided by the learner to the problem.

[0011] "Determining correctness" refers to determining whether the learner's answer matches the correct answer to the question.

[0012] "Answer history" refers to a record of questions that a learner has answered in the past and the results of those answers.

[0013] "Weaknesses" refer to areas or types of questions in which a learner tends to repeatedly make mistakes, as identified through an analysis of answer history.

[0014] "Original questions" refer to unique questions created to address specific weaknesses.

[0015] "Feedback" refers to evaluation and advice regarding a learner's answers and learning progress.

[0016] "Database" refers to a system for storing and managing information about learners, such as answer history and analysis results.

[0017] An "algorithm" refers to a set of computational procedures or rules for solving a particular problem.

[0018] "Study Plan" refers to a learning progression plan that is individually created based on the learner's needs and weaknesses. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[0041] System Configuration

[0042] Server part

[0043] The server implements the following functions:

[0044] 1. Acceptance of subject and scope selection

[0045] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0046] 2. Problem Generation

[0047] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0048] 3. Receiving answers and determining whether they are correct or incorrect

[0049] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0050] 4. Saving and analyzing answer history

[0051] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0052] 5. Generating original questions

[0053] Original questions are generated to reinforce weak points identified from the analysis results.

[0054] 6. Customize your study plan

[0055] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0056] Terminal part

[0057] The device implements the following features:

[0058] 1. Submit a learning request

[0059] Learners select the subject and scope through their terminal and send that information to the server.

[0060] 2. Displaying the problem

[0061] The terminal displays the questions received from the server to the learner.

[0062] 3. Enter and submit your answers

[0063] The learner enters the answer and sends it to the server via the terminal.

[0064] 4. Viewing Feedback

[0065] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0066] User Part

[0067] The user does the following:

[0068] 1. Select a learning request

[0069] Select the subject and area you want to study and make a request on your device.

[0070] 2. Answers to the questions

[0071] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0072] 3. Review feedback

[0073] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0074] Specific examples

[0075] When learning multiplication in arithmetic

[0076] 1. User makes a learning request

[0077] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[0078] 2. Generate a problem on the server

[0079] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0080] 3. The questions are displayed on the device and the user answers them.

[0081] The terminal displays "What is 28 x 36?" and the user answers "1008."

[0082] 4. The server receives the answer and determines whether it is correct or not.

[0083] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[0084] 5. The server saves and analyzes the answer history

[0085] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[0086] 6. The original questions are displayed on the device, and the user answers them again.

[0087] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[0088] In this way, learners can effectively improve their learning ability through repeated practice.

[0089] The processing flow will be explained below.

[0090] Step 1:

[0091] A user starts a learning application using a device and selects the subject and area they want to study. Let's consider the case where they select "Multiplication" in "Math."

[0092] Step 2:

[0093] The terminal transmits information about the subject and range selected by the user to the server.

[0094] Step 3:

[0095] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36".

[0096] Step 4:

[0097] The server sends the generated questions to the terminal.

[0098] Step 5:

[0099] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[0100] Step 6:

[0101] The user solves the displayed problem and enters the answer into the terminal. Let's consider the case where the user enters "1008".

[0102] Step 7:

[0103] The terminal transmits the user's answer to the server.

[0104] Step 8:

[0105] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[0106] Step 9:

[0107] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[0108] Step 10:

[0109] The terminal displays the feedback received from the server to the user.

[0110] Step 11:

[0111] The server stores the user's answer history in a database and analyzes it, particularly to determine which types of questions the user is weak at.

[0112] Step 12:

[0113] The server identifies the user's weaknesses based on the analysis results and generates original questions that focus on those weaknesses, such as "34 x 47."

[0114] Step 13:

[0115] The server sends the generated original questions to the terminal.

[0116] Step 14:

[0117] The terminal displays the original question received from the server to the user: "What is 34 x 47?"

[0118] Step 15:

[0119] The user solves the original problem displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[0120] Step 16:

[0121] The terminal transmits the user's answer to the server.

[0122] Step 17:

[0123] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[0124] Step 18:

[0125] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[0126] Step 19:

[0127] The terminal displays the feedback received from the server to the user.

[0128] Step 20:

[0129] Users can check the feedback, understand their weaknesses, and continue their studies. By repeating this process, users can progress through their studies efficiently.

[0130] Example 1

[0131] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] Conventional learning systems have difficulty providing appropriate questions that match the individual academic ability of each learner, and lack the ability to fully utilize answer history during the learning process to identify weak points and generate questions based on those weak points. As a result, learners have had difficulty improving their academic ability efficiently.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0134] In this invention, the server includes means for accepting a selection of a subject and range from a learner, means for using a natural language processing algorithm as a means for generating questions based on the subject and range, means for displaying the generated questions to the learner, means for inputting and transmitting answers from the learner, means for judging the correctness of the received answers, means for providing the judgement result and explanations to the learner, means for saving the answer history in a database, means for analyzing the answer history to identify the learner's weaknesses, means for using a generative AI model as a means for generating original questions based on the identified weaknesses, and means for re-displaying the generated original questions to the learner. This allows learners to be provided with questions according to their individual academic level, enabling them to progress in their studies while efficiently overcoming their weaknesses.

[0135] A "student" is a user who uses the learning support system to study a subject.

[0136] "Subjects" are areas of study offered by the learning support system, such as mathematics, science, and English.

[0137] "Scope" is the specific part or item within the selected subject that the learner wants to learn.

[0138] The "means for accepting selection" is a mechanism for receiving input of the subject and scope from the learner and transmitting it to the server.

[0139] A "means for generating questions" is an algorithm or generative model for creating specific learning questions based on the subject and scope selected by the learner.

[0140] A "natural language processing algorithm" is a computer software process for analyzing and understanding linguistic data.

[0141] A "generative AI model" is a mathematical model that uses machine learning and artificial intelligence to generate specific problems.

[0142] The "means for displaying questions to the learner" refers to a web interface or application that displays the generated questions on the learner's terminal screen.

[0143] The "means for inputting and sending answers" is an interface that allows a learner to input answers to questions and send them to the server.

[0144] A "means for determining correctness" is an algorithm or program that evaluates whether a learner's answer is correct or incorrect.

[0145] The "means for providing the judgment result and explanation" is a mechanism for displaying to the learner the result of whether the answer is correct or incorrect and an explanation based on that result.

[0146] The "means for storing answer history in a database" is a database system that accumulates information on learners' past answers.

[0147] "Means for analyzing answer history" refers to the process of analyzing saved answer data to identify the learner's weaknesses, etc.

[0148] "Means for generating original questions" are algorithms or models that create new learning questions for specific learners based on the analysis results.

[0149] The "means for re-displaying" is an interface that re-displays the original question that was generated to the learner.

[0150] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[0151] Server part

[0152] The server has the following features:

[0153] 1. Acceptance of subject and scope selection

[0154] The server receives learning requests from the devices and identifies the specific subjects and areas. This is done using web server software such as Apache or Nginx.

[0155] 2. Problem Generation

[0156] The server generates questions based on the requested subject and scope information using a Python script or a generative AI model (e.g., GPT-4 (registered trademark)).

[0157] 3. Receiving answers and determining whether they are correct or incorrect

[0158] The server receives the answers sent by the user and compares them with pre-set correct answers to determine whether they are correct or not. This process uses a database such as MySQL (registered trademark).

[0159] 4. Saving and analyzing answer history

[0160] The server stores the answer history in a database and periodically analyzes it using data analysis tools such as Python's Pandas and NumPy.

[0161] 5. Generating original questions

[0162] Based on the analysis results, original questions are generated that address the identified weaknesses, again using a generative AI model.

[0163] 6. Customize your study plan

[0164] The server provides an individually customized learning plan based on the learner's answer history and weakness analysis results. In this process, the customized learning plan is generated in JSON format and sent to the device.

[0165] Terminal part

[0166] The terminal has the following features:

[0167] 1. Submit a learning request

[0168] The user selects the subject and scope through their device and sends that information to the server via a web browser or mobile app interface.

[0169] 2. Displaying the problem

[0170] The terminal receives the questions from the server and displays them to the user, visually displaying the questions using HTML, CSS, and JavaScript.

[0171] 3. Enter and submit your answers

[0172] The user enters the answer into the terminal and sends it to the server, via a form or input field.

[0173] 4. Viewing Feedback

[0174] The correct / incorrect result sent from the server is displayed to the user. Visual feedback is provided using JavaScript and CSS.

[0175] User Part

[0176] The user does the following:

[0177] 1. Select a learning request

[0178] The user selects the subject and area they wish to study and makes a request on their terminal.

[0179] 2. Answers to the questions

[0180] Answers to questions displayed on the terminal are entered and the answers are sent to the terminal.

[0181] 3. Review feedback

[0182] Check the feedback displayed on your device and learn the correct answers and explanations for questions you got wrong.

[0183] Specific examples

[0184] When learning multiplication in arithmetic

[0185] 1. User makes a learning request

[0186] The user selects arithmetic on the terminal and requests the "multiplication" range.

[0187] 2. Generate a problem on the server

[0188] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0189] 3. The questions are displayed on the device and the user answers them.

[0190] The terminal displays "What is 28 x 36?" and the user enters the answer "1008."

[0191] 4. The server receives the answer and determines whether it is correct or not.

[0192] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[0193] 5. The server saves and analyzes the answer history

[0194] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[0195] 6. The original questions are displayed on the device, and the user answers them again.

[0196] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[0197] Prompt Sentence Examples

[0198] "Please enter the answer to the following question: 28 × 36 = ?"

[0199] This system allows users to study efficiently by providing questions appropriate to their academic level. The server, terminal, and user each play their respective roles effectively and work together to provide learning support.

[0200] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0201] Step 1:

[0202] A user submits a learning request.

[0203] The user selects the subject and area of ​​study they wish to study through their device and enters that request into the device's input form. The entered data (subject and area information) is sent to the server as an HTTP request.

[0204] Step 2:

[0205] The server identifies the subject and scope.

[0206] The server analyzes the HTTP request received from the device and extracts information about the subject and scope. Based on this information, it prepares to generate appropriate questions.

[0207] Step 3:

[0208] The server generates the problem.

[0209] The server creates questions using a Python script or a generative AI model (e.g., GPT-4) based on the received subject and scope information. Specifically, it inputs a prompt to the AI ​​model in the form of "Generate the following problem: multiplication, two-digit x two-digit," and generates an appropriate problem (e.g., "28 x 36"). The output problem data is used in the next processing step.

[0210] Step 4:

[0211] The terminal displays the problem.

[0212] The problem data received from the server is displayed to the user using a user interface built with HTML and JavaScript. Specifically, the question "What is 28 x 36?" is displayed on the terminal screen. This display operation allows the user to visually recognize the problem.

[0213] Step 5:

[0214] The user enters and submits the answer.

[0215] The user enters the answer to the displayed question in the input field and clicks the "Submit" button. The entered answer (e.g., "1008") is again sent to the server as an HTTP request.

[0216] Step 6:

[0217] The server receives the answer and determines whether it is correct.

[0218] The server receives the answer data sent from the terminal and compares it with the correct answer data (e.g., "1008") set in advance. The server determines whether the answer is correct based on the comparison result, generates a correctness judgment result (e.g., "That's correct!"), and outputs it.

[0219] Step 7:

[0220] The device will display the result of the correct answer.

[0221] The result of the correct answer received from the server is displayed to the user. JavaScript is used to display feedback on the device screen such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[0222] Step 8:

[0223] The server stores and analyzes the answer history.

[0224] The server stores the user's answer data and the results of correct / incorrect assessment in a database. The stored data is periodically analyzed and the learner's weaknesses are identified using data analysis tools such as Python's Pandas and NumPy.

[0225] Step 9:

[0226] The server generates the original questions.

[0227] The server generates new problems using a generative AI model to improve weaknesses identified based on the analysis results. For example, it inputs a prompt such as "Generate the following problem: multiplication, two-digit x two-digit," and outputs an original problem such as "34 x 47."

[0228] Step 10:

[0229] The terminal displays the original question and the user answers it again.

[0230] The terminal receives new questions from the server and displays them to the user. The user enters answers to the new questions and submits them again. By repeating this process, the user can efficiently improve their academic ability.

[0231] (Application example 1)

[0232] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] While conventional learning support systems contribute to learners' knowledge acquisition, they have limitations in terms of improving practical skills and gaining practical experience in a virtual environment. In particular, there is a lack of effective ways to learn skills required for actual work, such as customer service and sales skills. Furthermore, it is difficult to provide real-time feedback or individual support based on learners' weaknesses.

[0234] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0235] In this invention, the server includes means for accepting a selection of a subject and a range from a learner, means for generating questions based on the subject and range, means for providing the generated questions to the learner, means for receiving answers from the learner, means for determining whether the received answers are correct or incorrect, means for providing the determination result to the learner, means for analyzing the answer history to identify the learner's weaknesses, means for generating original questions based on the identified weaknesses, means for providing the learner with the generated original questions, means for providing the learner with practical skills in a virtual environment, and means for providing the learner with appropriate feedback in real time based on the received answers. This enables the learner to effectively learn skills necessary for actual work in a virtual environment.

[0236] A "learner" is an individual who aims to improve their knowledge and skills by using a learning support system.

[0237] "Subject" refers to the type or field of learning content, including mathematics, science, social studies, etc.

[0238] "Scope" refers to the specific area of ​​study within the selected subject, including specific chapters and topics.

[0239] "Problems" refer to questions or tasks given to assess learners' knowledge and skills.

[0240] "Correctness assessment" is the process of determining whether the answer provided by the learner is correct.

[0241] "Feedback" refers to the evaluation and advice given to a learner in response to their answers.

[0242] An "answer history" is a record of all answers a learner has ever given.

[0243] "Weaknesses" refer to areas where a learner has weaknesses or lacks understanding, identified by analyzing the answer history.

[0244] "Original questions" refer to questions that are specially created to address a learner's weaknesses.

[0245] A "virtual environment" refers to a simulated learning or experiential space created using computers and digital technology.

[0246] "Practical skills" refers to the practical abilities and know-how required in the job.

[0247] "Real-time" refers to immediate response to the information and actions input by the learner.

[0248] A learning support system for specifically implementing this invention comprises three components: a server, a terminal, and a user.

[0249] Server part

[0250] The server implements the following functions:

[0251] 1. Accepting learning requests

[0252] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0253] 2. Problem Generation

[0254] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0255] 3. Receiving answers and determining whether they are correct or incorrect

[0256] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0257] 4. Saving and analyzing answer history

[0258] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0259] 5. Generating original questions

[0260] Original questions are generated to reinforce weak points identified from the analysis results.

[0261] 6. Customize your study plan

[0262] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0263] 7. Providing practical skills in a virtual environment

[0264] Provide simulations that allow learners to gain hands-on experience and learn the skills they need to succeed in a virtual environment.

[0265] 8. Real-time feedback

[0266] Provide appropriate feedback in real time based on the answers provided by the learner.

[0267] Terminal part

[0268] The device implements the following features:

[0269] 1. Submit a learning request

[0270] Learners select the subject and scope through their terminal and send that information to the server.

[0271] 2. Displaying the problem

[0272] The terminal displays the questions received from the server to the learner.

[0273] 3. Enter and submit your answers

[0274] The learner enters the answer and sends it to the server via the terminal.

[0275] 4. Viewing Feedback

[0276] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0277] 5. Running a Virtual Environment

[0278] The devices use smart glasses or head-mounted displays to display virtual environments, allowing learners to learn practical skills.

[0279] User Part

[0280] The user (learner) performs the following operations:

[0281] 1. Select a learning request

[0282] Select the subject and area you want to study and make a request on your device.

[0283] 2. Answers to the questions

[0284] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0285] 3. Review feedback

[0286] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0287] 4. Practical skills training in a virtual environment

[0288] In a virtual environment, practical skills can be learned through simulations, such as practicing customer service and sales skills in a virtual store.

[0289] Hardware and software used

[0290] The server runs programs using languages ​​and frameworks such as Python and Django.

[0291] The terminal provides applications that run on a variety of devices, including smartphones, tablets, smart glasses, and head-mounted displays.

[0292] The database uses a relational database management system such as MySQL or PostgreSQL.

[0293] The generative AI model uses machine learning and natural language processing frameworks (such as TENSORFLOW (registered trademark) and PyTorch) to generate questions and provide feedback.

[0294] Specific examples

[0295] If you're learning customer service skills in a virtual environment:

[0296] 1. User makes a learning request

[0297] The user selects "customer service skills" on the terminal and sends a learning request.

[0298] 2. Generate a problem on the server

[0299] The server generates questions such as "If a customer asks you a question about a product, how would you respond?" and sends them to the terminal.

[0300] 3. The questions are displayed on the device and the user answers them.

[0301] The terminal displays the virtual environment and initiates a dialogue with the virtual customer, who inputs answers and sends them to the server.

[0302] 4. The server receives the answer, determines whether it is correct, and provides feedback.

[0303] The server analyzes the answer and provides real-time feedback such as, "The answer given to the customer was appropriate. Here's what would be best next time."

[0304] Prompt Sentence Examples

[0305] Use the following information to generate questions to teach customer service skills in a virtual store:

[0306] Categories: Product knowledge, complaint handling, sales skills

[0307] Example: Describe the features of this product.

[0308] In this way, learners can effectively learn practical skills in a virtual environment.

[0309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0310] Step 1:

[0311] The terminal accepts learning requests from users. The user selects the subject and scope of the learning they wish to do through the terminal interface and sends the request. The input is the subject and scope information, and the output is the request data sent to the server.

[0312] Step 2:

[0313] The server receives request data sent from the terminal. The input is the learning request data, and the output is a trigger to invoke the problem generation algorithm based on the request data.

[0314] Step 3:

[0315] The server generates questions based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history. The input is the subject and scope information and past answer history data, and the output is the generated question data.

[0316] Step 4:

[0317] The server sends the generated questions to the terminal. The input is the generated question data, and the output is the question data sent to the terminal.

[0318] Step 5:

[0319] The terminal displays the questions sent from the server to the learner. The input is the question data, and the output is the question displayed on the terminal interface.

[0320] Step 6:

[0321] The user answers the questions displayed on the terminal. The user enters the answer into the terminal and presses the send button. The input is the user's answer, and the output is the answer data sent to the server.

[0322] Step 7:

[0323] The server receives the answer from the user and judges whether the answer is correct or not. The input is the user's answer data, and the output is the result of the judgment.

[0324] Step 8:

[0325] The server sends the result of the accuracy judgment to the terminal. The input is the accuracy judgment result, and the output is feedback data sent to the terminal.

[0326] Step 9:

[0327] The terminal displays the result of the judgment and explanation from the server to the user. The input is the feedback data, and the output is the feedback displayed on the terminal interface.

[0328] Step 10:

[0329] The server stores the user's answer history in a database. The input is the answer data and the result of the correct / incorrect judgment, and the output is the answer history data stored in the database.

[0330] Step 11:

[0331] The server periodically analyzes the answer history and identifies the user's weaknesses. The input is the answer history data, and the output is the analysis results and the identified weaknesses.

[0332] Step 12:

[0333] The server generates original questions based on the identified weaknesses. The input is the weakness analysis result, and the output is the generated original question data.

[0334] Step 13:

[0335] The server sends the generated original question to the terminal. The input is the original question data, and the output is the original question data sent to the terminal.

[0336] Step 14:

[0337] The terminal displays the original problem to the user. The input is the original problem data, and the output is the original problem displayed on the interface.

[0338] Step 15:

[0339] The terminal runs the virtual environment and allows users to experience practical skills. The input is the data of the virtual environment simulation, and the output is the virtual environment that the user can view and manipulate.

[0340] Step 16:

[0341] The server provides appropriate feedback in real time based on the answers provided by the user. The input is real-time answer data, and the output is real-time feedback data.

[0342] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0343] A learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine.

[0344] System Configuration

[0345] Server part

[0346] The server implements the following functions:

[0347] 1. Acceptance of subject and scope selection

[0348] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0349] 2. Problem Generation

[0350] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0351] 3. Receiving answers and determining whether they are correct or incorrect

[0352] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0353] 4. Saving and analyzing answer history

[0354] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0355] 5. Generating original questions

[0356] Original questions are generated to reinforce weak points identified from the analysis results.

[0357] 6. Customize your study plan

[0358] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0359] 7. Emotion Data Analysis

[0360] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[0361] Terminal part

[0362] The device implements the following features:

[0363] 1. Submit a learning request

[0364] Learners select the subject and scope through their terminal and send that information to the server.

[0365] 2. Displaying the problem

[0366] The terminal displays the questions received from the server to the learner.

[0367] 3. Enter and submit your answers

[0368] The learner enters the answer and sends it to the server via the terminal.

[0369] 4. Viewing Feedback

[0370] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0371] 5. Collecting Emotional Data

[0372] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[0373] User Part

[0374] The user does the following:

[0375] 1. Select a learning request

[0376] Select the subject and area you want to study and make a request on your device.

[0377] 2. Answers to the questions

[0378] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0379] 3. Review feedback

[0380] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0381] 4. Providing Emotion Data

[0382] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[0383] Emotion engine part

[0384] The emotion engine implements the following functions:

[0385] 1. Emotion recognition and collection

[0386] Recognize and collect emotions from learners' facial expressions and voice data.

[0387] 2. Transmission of emotional information

[0388] The recognized emotion information is sent to the server.

[0389] 3. Emotion-Based Adjustment

[0390] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[0391] Specific examples

[0392] Learning multiplication in arithmetic using an emotion engine

[0393] 1. User makes a learning request

[0394] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[0395] 2. Generate a problem on the server

[0396] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0397] 3. Display the problem on the device and collect sentiment data

[0398] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[0399] 4. The user enters the answer

[0400] The user enters the answer "1008".

[0401] 5. The server receives the answer and analyzes the emotion data.

[0402] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[0403] 6. Feedback and difficulty adjustments

[0404] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[0405] 7. View feedback on your device

[0406] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[0407] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[0408] The processing flow will be explained below.

[0409] Step 1:

[0410] The user starts the learning application on the device and selects the subject and area they want to study. Let's assume that they select "Multiplication" in "Math."

[0411] Step 2:

[0412] The terminal transmits information about the subject and range selected by the user to the server.

[0413] Step 3:

[0414] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36."

[0415] Step 4:

[0416] The server sends the generated questions to the terminal.

[0417] Step 5:

[0418] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[0419] Step 6:

[0420] At the same time as displaying the image, the device sends the user's facial expressions and voice data to the emotion engine.

[0421] Step 7:

[0422] The emotion engine recognizes and analyzes the user's emotions from the facial and voice data it receives. For example, if the user has a troubled expression, it will determine that they are feeling stressed.

[0423] Step 8:

[0424] The user answers the displayed question and enters the answer into the terminal. Let's consider the case where the user enters "1008."

[0425] Step 9:

[0426] The device sends the user's answer to the server and also sends the emotion engine's analysis results to the server.

[0427] Step 10:

[0428] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[0429] Step 11:

[0430] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[0431] Step 12:

[0432] The server stores the answer history in a database and analyzes it, identifying the learner's weaknesses based on the analysis results.

[0433] Step 13:

[0434] The server adjusts the next learning step based on the analysis results and data from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, it adjusts the difficulty of the questions.

[0435] Step 14:

[0436] The server generates new original questions and sends the information to the terminal. It generates new questions such as "What is 34 x 47?"

[0437] Step 15:

[0438] The terminal displays the original questions received from the server to the user.

[0439] Step 16:

[0440] The device again sends the user's facial expressions and voice data to the emotion engine.

[0441] Step 17:

[0442] The emotion engine recognizes and analyzes emotions from new facial and voice data.

[0443] Step 18:

[0444] The user answers the original question displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[0445] Step 19:

[0446] The device sends the user's answer to the server, as well as the emotion engine's new analysis results.

[0447] Step 20:

[0448] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[0449] Step 21:

[0450] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[0451] Step 22:

[0452] The device displays the feedback received from the server to the user, including encouragement and advice based on the user's emotions.

[0453] Step 23:

[0454] By repeating this series of processes, users can reinforce their weaknesses while progressing with optimal learning based on emotional data.

[0455] Example 2

[0456] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0457] Conventional learning support systems generate questions based on the learner's academic level and reinforce weak points by analyzing the learner's answer history, but they lack the ability to individually customize the system to take into account the learner's emotional state. As a result, learners' stress and decreased motivation can affect their learning outcomes. Therefore, the present invention aims to provide a more effective and personalized learning support system that takes into account the learner's emotional state.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0459] In this invention, the server includes means for receiving selection of a subject and range from the learner, means for generating questions based on the subject and range, and means for analyzing collected emotional data to adjust the difficulty of the study plan and questions, thereby making it possible to provide a study plan that takes into account the emotional state of the learner.

[0460] "Learner" refers to a person who uses the system to acquire knowledge or skills.

[0461] A "subject" refers to a particular field or topic that a learner will study.

[0462] "Scope" refers to the part of the subject that the learner particularly wants to study.

[0463] "Problems" are questions or tasks provided to assess a learner's understanding or skills.

[0464] "Answer" refers to the answer to the question submitted by the learner.

[0465] "Correct or incorrect" refers to the result of determining whether the learner's answer is correct or incorrect.

[0466] "Answer history" refers to a record of answers previously provided by a learner.

[0467] "Weak points" refer to areas in which a learner is particularly weak, as identified by analyzing the answer history.

[0468] "Original questions" refer to questions that are specially created to reinforce a learner's weak points.

[0469] "Emotional data" refers to information about emotions collected from learners' facial expressions, voice, etc.

[0470] "Emotion engine" refers to technology for analyzing collected emotional data and providing the results to the system.

[0471] "Study plan" refers to the learning progress plan and content provided to learners.

[0472] "Database" refers to an information management system for storing learners' answer history and other information.

[0473] The learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine. The functions and specific processes of each component will be explained in detail below.

[0474] System Configuration

[0475] Server part

[0476] The server has the following functions:

[0477] 1. Acceptance of subject and scope selection

[0478] The server receives the learning request sent from the device and identifies the specific subject and area, and stores this information in a database.

[0479] Example: When a request for "arithmetic" and "multiplication" is sent from the terminal, the server receives it.

[0480] 2. Problem Generation

[0481] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0482] Example: Generate questions of appropriate difficulty, such as "15 x 12," based on the learner's academic level.

[0483] 3. Receiving answers and determining whether they are correct or incorrect

[0484] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0485] Example: A learner sends the answer "180" to the server, and the server determines this to be the "correct answer."

[0486] 4. Saving and analyzing answer history

[0487] The server stores the answer history in a database and periodically analyzes it to identify the learner's weak points.

[0488] Example: Save a history of frequent mistakes a learner makes on multiplication problems and analyze the data.

[0489] 5. Generating original questions

[0490] Original questions are generated to reinforce weak points identified from the analysis results.

[0491] For example, if a weakness in "multiplication" is identified, generate and provide similar multiplication problems.

[0492] 6. Customize your study plan

[0493] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0494] Example: Propose a lesson plan aimed at strengthening multiplication skills.

[0495] 7. Emotion Data Analysis

[0496] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[0497] For example: temporarily reducing the difficulty of questions when a learner is feeling stressed.

[0498] Terminal part

[0499] The device has the following features:

[0500] 1. Submit a learning request

[0501] Learners select the subject and scope through their terminal and send that information to the server.

[0502] Example: If a learner selects "arithmetic" and "multiplication", the selection is sent to the server.

[0503] 2. Displaying the problem

[0504] The terminal displays the questions received from the server to the learner.

[0505] For example, display the question as "What is 15 x 12?"

[0506] 3. Enter and submit your answers

[0507] The learner enters the answer and sends it to the server via the terminal.

[0508] Example: Enter the answer "180" and submit.

[0509] 4. Viewing Feedback

[0510] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0511] Example: Display "You got it right! Congratulations!"

[0512] 5. Collecting Emotional Data

[0513] The device collects the learner's facial expressions and voice data and sends it to the emotion engine.

[0514] Example: Recognizing a user's smile through a camera and sending that data to an emotion engine.

[0515] User Part

[0516] The user performs the following operations:

[0517] 1. Select a learning request

[0518] Select the subject and area you want to study and make a request on your device.

[0519] Example: User selects "Math" and "Multiplication".

[0520] 2. Answers to the questions

[0521] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0522] For example, enter "15 × 12 = 180".

[0523] 3. Review feedback

[0524] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0525] For example: "You got it right! Congratulations!"

[0526] 4. Providing Emotion Data

[0527] Emotions during training are collected by the device, and facial and voice data are analyzed by the emotion engine.

[0528] For example, when a learner smiles while solving a problem, their facial expression data is analyzed.

[0529] Emotion engine part

[0530] The emotion engine has the following functions:

[0531] 1. Emotion recognition and collection

[0532] Recognize and collect emotions from learners' facial expressions and voice data.

[0533] Example: Recognizing "joy" from a learner's smiling face.

[0534] 2. Transmission of emotional information

[0535] The recognized emotion information is sent to the server.

[0536] Example: Sending learner "joy" data to the server.

[0537] 3. Emotion-Based Adjustment

[0538] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[0539] For example: If a learner is feeling stressed, provide them with easy problems.

[0540] Specific examples

[0541] Example of learning multiplication using emotion engine

[0542] 1. User makes a learning request

[0543] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[0544] 2. Generate a problem on the server

[0545] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0546] 3. Display the problem on the device and collect sentiment data

[0547] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[0548] 4. The user enters the answer

[0549] The user enters the answer "1008".

[0550] 5. The server receives the answer and analyzes the emotion data.

[0551] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[0552] 6. Feedback and difficulty adjustments

[0553] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[0554] 7. View feedback on your device

[0555] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[0556] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[0557] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0558] Program processing steps and detailed explanations

[0559] Step 1:

[0560] Submit a learning request

[0561] Terminal processing: An interface is displayed that allows the user to select the subject and range of study they wish to do via the terminal. The user selects "arithmetic" and "multiplication" and sends the request.

[0562] Input: User selection information for the subject "arithmetic" and range "multiplication".

[0563] Output: The training request sent to the server.

[0564] Specific operation: The terminal sends the selected subject and range to the server as an HTTP request.

[0565] Step 2:

[0566] Processing subject and scope information

[0567] Server processing: The server analyzes the received learning request, stores it in a database, and then performs preprocessing to generate appropriate questions.

[0568] Input: The training request sent from the device.

[0569] Output: Subject and scope information stored in database.

[0570] Specific operation: The server receives the HTTP request and records it in the database.

[0571] Step 3:

[0572] Problem Generation

[0573] Server processing: The server runs a question generation algorithm based on the subject and scope information stored in the database, taking into account the learner's past answer history and academic level.

[0574] Input: subject and scope information, past answer history, academic level information.

[0575] Output: The generated questions.

[0576] Specific operation: The server retrieves the necessary information from the database and generates a problem such as "15 x 12".

[0577] Step 4:

[0578] Viewing Issues

[0579] Terminal processing: The generated questions sent from the server are displayed to the user.

[0580] Input: The generated question sent by the server.

[0581] Output: The problem as it appears on the user's screen.

[0582] Specific operation: The terminal receives the response from the server and displays the question "What is 15 x 12?"

[0583] Step 5:

[0584] Enter your answer

[0585] User operation: The user inputs an answer to the displayed question. For example, the answer is "180."

[0586] Input: The user's answer.

[0587] Output: The answer typed into the terminal.

[0588] Specific action: The user enters "180" in the input field of the terminal.

[0589] Step 6:

[0590] Submit your answer

[0591] Terminal processing: The terminal sends the user's answer to the server.

[0592] Input: The user's answer.

[0593] Output: The answer sent to the server.

[0594] Specific operation: The terminal sends the entered answer "180" to the server via an HTTP request.

[0595] Step 7:

[0596] Judging whether the answer is correct or incorrect

[0597] Server processing: The server compares the received answer with the correct answer set in advance and determines whether it is correct or incorrect.

[0598] Input: User's answer "180".

[0599] Output: Correct / incorrect result.

[0600] Specific operation: The server compares the answer "180" with the correct answer "180" and determines that it is correct.

[0601] Step 8:

[0602] Save answer history

[0603] Server processing: The answer results are saved in a database and recorded as a history for future analysis.

[0604] Input: User's answer and verdict.

[0605] Output: Answer history stored in a database.

[0606] Specific operation: The server stores the answers and judgment results in a database.

[0607] Step 9:

[0608] Collecting Emotional Data

[0609] Device processing: The device collects the learner's facial expressions and voice data and sends them to the emotion engine.

[0610] Input: Learner's facial expressions and voice data.

[0611] Output: Emotion data sent to the emotion engine.

[0612] Specific operation: The device collects the user's emotional data through the camera and microphone and sends it to the emotion engine.

[0613] Step 10:

[0614] Emotional Data Analysis

[0615] Emotion Engine Processing: The emotion engine analyzes the received data and determines the learner's current emotional state.

[0616] Input: Emotion data sent from the device.

[0617] Output: Learner's emotional state.

[0618] Specific operation: The emotion engine uses a facial expression recognition algorithm to analyze the user's smile and stress state.

[0619] Step 11:

[0620] Generate customized feedback

[0621] Server processing: Based on answer history and emotion data, appropriate feedback and the next learning plan are generated.

[0622] Input: Answer history and emotion data.

[0623] Output: Customized feedback and study plan.

[0624] Specific operation: The server takes into account the learner's situation and generates feedback such as "That's correct! The next question will be easier."

[0625] Step 12:

[0626] View Feedback

[0627] Terminal processing: Display the feedback sent from the server to the user.

[0628] Input: Customized feedback.

[0629] Output: Feedback displayed on the user's screen.

[0630] Specific operation: The device displays "Correct! The next question will be easy."

[0631] By following these processing steps, personalized learning support can be provided to the learner.

[0632] (Application example 2)

[0633] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0634] Conventional learning support systems are unable to take into account learners' emotions or real-time reactions, and because learning plans and questions are provided uniformly, effective personalization is difficult. Furthermore, it is difficult to utilize customer emotional data in brick-and-mortar stores, resulting in a lack of services and proposals that meet customer needs. A system that can solve these problems and enable more effective and personalized learning and customer service is needed.

[0635] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a subject and range selection from a learner; means for generating questions based on the subject and range; means for providing the generated questions to the learner; means for receiving answers from the learner; means for determining whether the received answers are correct or incorrect; means for providing the determination results to the learner; means for analyzing the answer history to identify the learner's weaknesses; means for generating original questions based on the identified weaknesses; means for providing the generated original questions to the learner; means for collecting emotional data; means for analyzing the collected emotional data; means for providing a learning plan or feedback corresponding to the learner's emotions based on the analysis results; and means for analyzing customer emotional data in real time at a physical store and displaying information and suggestions suitable for the customer. This enables learning support optimized for individual learners and further enables services to be provided in physical stores that meet customer needs.

[0636] "Learner" refers to an individual who is learning educational content and is a user who intends to acquire knowledge and skills using this system.

[0637] "Means for accepting selection of subject and scope" is a function that provides an interface for a learner to specify a particular subject and scope that they wish to study.

[0638] The "means for generating questions" is a function that automatically creates questions that are relevant to the content based on the subject and scope selected by the learner.

[0639] The "means for providing the generated questions to the learner" is a function for displaying the questions generated by the server on the learner's terminal.

[0640] The "means for receiving answers" is a function for sending answers entered by learners to the server.

[0641] The "means for determining whether the answer is correct or incorrect" is a function that compares the learner's answer with pre-set correct answer data and determines whether the answer is correct or incorrect.

[0642] "Means for identifying learners' weaknesses by analyzing answer history" is a function that analyzes a learner's past answer data and identifies areas and question types in which the learner is weak.

[0643] The "means for generating original questions" is a function for creating new questions specifically for strengthening the weak points of identified learners.

[0644] "Means for collecting emotional data" is a function for obtaining emotional data from the facial expressions, voices, etc. of learners and customers.

[0645] "Means for analyzing emotional data" is a function for analyzing the emotional state of learners and customers based on collected emotional data.

[0646] "Means for providing learning plans and feedback" refers to a function that provides learning plans and feedback that correspond to the learner's emotions based on the analysis results.

[0647] "A means of analyzing customer emotional data in real time in physical stores and displaying information and suggestions appropriate to the customer" is a function that instantly analyzes customer emotions in stores and suggests the most suitable products and services to the customer based on the results.

[0648] MODE FOR CARRYING OUT THE INVENTION

[0649] The learning support system and customer service support system for brick-and-mortar stores of the present invention are composed of four components: a server, a terminal, a user, and an emotion engine.

[0650] System Configuration

[0651] Server part

[0652] The server implements the following functions:

[0653] 1. Acceptance of subject and scope selection

[0654] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0655] 2. Problem Generation

[0656] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0657] 3. Receiving answers and determining whether they are correct or incorrect

[0658] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0659] 4. Saving and analyzing answer history

[0660] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0661] 5. Generating original questions

[0662] Original questions are generated to reinforce weak points identified from the analysis results.

[0663] 6. Customize your study plan

[0664] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0665] 7. Emotion Data Analysis

[0666] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[0667] 8. Analysis and Proposal of Emotional Data in Physical Stores

[0668] Customer sentiment data obtained in physical stores is analyzed in real time, and the results are used to provide optimal product suggestions and customer service methods to customers.

[0669] Terminal part

[0670] The device implements the following features:

[0671] 1. Submit a learning request

[0672] Learners select the subject and scope through their terminal and send that information to the server.

[0673] 2. Displaying the problem

[0674] The terminal displays the questions received from the server to the learner.

[0675] 3. Enter and submit your answers

[0676] The learner enters the answer and sends it to the server via the terminal.

[0677] 4. Viewing Feedback

[0678] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0679] 5. Collecting Emotional Data

[0680] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[0681] User Part

[0682] The user does the following:

[0683] 1. Select a learning request

[0684] Select the subject and area you want to study and make a request on your device.

[0685] 2. Answers to the questions

[0686] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0687] 3. Review feedback

[0688] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0689] 4. Providing Emotion Data

[0690] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[0691] Emotion engine part

[0692] The emotion engine implements the following functions:

[0693] 1. Emotion recognition and collection

[0694] Recognize and collect emotions from learners' and customers' facial expressions and voice data.

[0695] 2. Transmission of emotional information

[0696] The recognized emotion information is sent to the server.

[0697] 3. Emotion-Based Adjustment

[0698] Based on the emotional information provided to the server, the system will assess the learner's stress level and concentration level, and adjust the difficulty of the questions and the content of the feedback. In addition, in the case of a physical store, the system will provide appropriate information and product suggestions based on the customer's emotions.

[0699] Specific examples

[0700] Learning multiplication in arithmetic using an emotion engine

[0701] 1. User makes a learning request

[0702] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[0703] 2. Generate a problem on the server

[0704] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0705] 3. Display the problem on the device and collect sentiment data

[0706] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[0707] 4. The user enters the answer

[0708] The user enters the answer "1008".

[0709] 5. The server receives the answer and analyzes the emotion data.

[0710] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[0711] 6. Feedback and difficulty adjustments

[0712] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[0713] 7. View feedback on your device

[0714] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[0715] Emotion analysis and customer support in brick-and-mortar stores

[0716] 1. The customer is choosing a product

[0717] Smart glasses in physical stores detect customers' facial expressions and voices.

[0718] 2. The emotion engine analyzes the emotion data

[0719] If the customer is confused, the emotion engine sends this data to the server.

[0720] 3. The server displays the proposal

[0721] Based on the analysis results, the server displays appropriate product suggestions and explanations on the smart glasses.

[0722] 4. Staff make suggestions

[0723] Staff will then make optimal product recommendations to customers based on the displayed information.

[0724] Example prompt for a generative AI model:

[0725] "If a customer is interested in your product but seems confused, what explanation or suggestion would be most appropriate? For example, explaining new features or showing examples of use."

[0726] This allows for effective and personalized assistance that takes into account the emotions of learners and customers.

[0727] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0728] Step 1:

[0729] The learner selects the subject and range on the terminal and sends a learning request to the server. The input is the subject and range selected by the learner. The output is the learning request data sent to the server. Here, the terminal provides the interface and the learner makes the selection.

[0730] Step 2:

[0731] Based on the learning request data received by the server, questions are generated for the relevant subject and scope. The input is the learning request data, and the output is the generated question data. To do this, the server retrieves relevant information from a database and creates appropriate questions using an algorithm.

[0732] Step 3:

[0733] The server sends the generated questions to the terminal, which then displays them to the learner. The input is the generated question data, and the output is the question displayed on the terminal. The terminal displays the question using a display interface.

[0734] Step 4:

[0735] The learner inputs the answer through the terminal and sends it to the server. The input is the answer entered by the learner, and the output is the answer data sent to the server. The terminal provides the answer interface, and the learner inputs the answer.

[0736] Step 5:

[0737] The server compares the received answer with the correct answer data to determine whether it is correct or not. The input is the learner's answer data and the preset correct answer data, and the output is the result of the correct or incorrect decision. The server uses logic to perform the comparison and calculate the result.

[0738] Step 6:

[0739] The server sends the judgment result to the terminal, which displays it to the learner. The input is the correct / incorrect judgment result, and the output is the feedback displayed on the terminal. The terminal provides a feedback screen and displays the result.

[0740] Step 7:

[0741] The device collects the learner's facial and voice data and sends it to the emotion engine. The input is facial and voice data, and the output is emotion data that is sent to the emotion engine. The device captures data using sensors.

[0742] Step 8:

[0743] The emotion engine analyzes the received emotion data and sends the analysis results to the server. The input is emotion data and the output is the analysis results. The emotion engine analyzes the data using emotion recognition algorithms.

[0744] Step 9:

[0745] The server adjusts the difficulty of questions and feedback based on the results of sentiment analysis. The input is the analysis results, and the output is the adjusted questions and feedback data. The server then processes the learning plan to dynamically customize it.

[0746] Step 10:

[0747] The server sends the adjusted questions and feedback to the device, which then displays them to the learner. The input is the adjusted data from the server, and the output is the adjusted content displayed on the device. The device again updates the interface and displays the information.

[0748] Step 11:

[0749] In a physical store, smart glasses collect customer facial and voice data and send it to an emotion engine. The input is the customer's facial and voice data, and the output is emotion data sent to the emotion engine. The smart glasses act as a collection device.

[0750] Step 12:

[0751] The emotion engine analyzes customer emotion data in real time and sends the analysis results to the server. The input is emotion data, and the output is the real-time analysis results. The emotion engine quickly performs data calculations.

[0752] Step 13:

[0753] The server sends and displays appropriate product and service suggestions based on the analysis results to the smart glasses. The input is the analysis results and suggestion data, and the output is the suggestions displayed on the smart glasses. The server generates and sends the suggestions, which are then displayed by the smart glasses.

[0754] This will enable personalized learning and customer support that takes into account real-time emotional data from learners and customers.

[0755] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0756] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0757] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0758] [Second embodiment]

[0759] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0760] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0761] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0762] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0763] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0764] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0765] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0766] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0767] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0768] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0769] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0770] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0771] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[0772] System Configuration

[0773] Server part

[0774] The server implements the following functions:

[0775] 1. Acceptance of subject and scope selection

[0776] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0777] 2. Problem Generation

[0778] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0779] 3. Receiving answers and determining whether they are correct or incorrect

[0780] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0781] 4. Saving and analyzing answer history

[0782] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0783] 5. Generating original questions

[0784] Original questions are generated to reinforce weak points identified from the analysis results.

[0785] 6. Customize your study plan

[0786] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0787] Terminal part

[0788] The device implements the following features:

[0789] 1. Submit a learning request

[0790] Learners select the subject and scope through their terminal and send that information to the server.

[0791] 2. Displaying the problem

[0792] The terminal displays the questions received from the server to the learner.

[0793] 3. Enter and submit your answers

[0794] The learner enters the answer and sends it to the server via the terminal.

[0795] 4. Viewing Feedback

[0796] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[0797] User Part

[0798] The user does the following:

[0799] 1. Select a learning request

[0800] Select the subject and area you want to study and make a request on your device.

[0801] 2. Answers to the questions

[0802] Answer the questions displayed on the terminal and enter the answers into the terminal.

[0803] 3. Review feedback

[0804] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[0805] Specific examples

[0806] When learning multiplication in arithmetic

[0807] 1. User makes a learning request

[0808] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[0809] 2. Generate a problem on the server

[0810] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0811] 3. The questions are displayed on the device and the user answers them.

[0812] The terminal displays "What is 28 x 36?" and the user answers "1008."

[0813] 4. The server receives the answer and determines whether it is correct or not.

[0814] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[0815] 5. The server saves and analyzes the answer history

[0816] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[0817] 6. The original questions are displayed on the device, and the user answers them again.

[0818] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[0819] In this way, learners can effectively improve their learning ability through repeated practice.

[0820] The processing flow will be explained below.

[0821] Step 1:

[0822] A user starts a learning application using a device and selects the subject and area they want to study. Let's consider the case where they select "Multiplication" in "Math."

[0823] Step 2:

[0824] The terminal transmits information about the subject and range selected by the user to the server.

[0825] Step 3:

[0826] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36".

[0827] Step 4:

[0828] The server sends the generated questions to the terminal.

[0829] Step 5:

[0830] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[0831] Step 6:

[0832] The user solves the displayed problem and enters the answer into the terminal. Let's consider the case where the user enters "1008".

[0833] Step 7:

[0834] The terminal transmits the user's answer to the server.

[0835] Step 8:

[0836] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[0837] Step 9:

[0838] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[0839] Step 10:

[0840] The terminal displays the feedback received from the server to the user.

[0841] Step 11:

[0842] The server stores the user's answer history in a database and analyzes it, particularly to determine which types of questions the user is weak at.

[0843] Step 12:

[0844] The server identifies the user's weaknesses based on the analysis results and generates original questions that focus on those weaknesses, such as "34 x 47."

[0845] Step 13:

[0846] The server sends the generated original questions to the terminal.

[0847] Step 14:

[0848] The terminal displays the original question received from the server to the user: "What is 34 x 47?"

[0849] Step 15:

[0850] The user solves the original problem displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[0851] Step 16:

[0852] The terminal transmits the user's answer to the server.

[0853] Step 17:

[0854] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[0855] Step 18:

[0856] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[0857] Step 19:

[0858] The terminal displays the feedback received from the server to the user.

[0859] Step 20:

[0860] Users can check the feedback, understand their weaknesses, and continue their studies. By repeating this process, users can progress through their studies efficiently.

[0861] Example 1

[0862] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] Conventional learning systems have difficulty providing appropriate questions that match the individual academic ability of each learner, and lack the ability to fully utilize answer history during the learning process to identify weak points and generate questions based on those weak points. As a result, learners have had difficulty improving their academic ability efficiently.

[0864] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0865] In this invention, the server includes means for accepting a selection of a subject and range from a learner, means for using a natural language processing algorithm as a means for generating questions based on the subject and range, means for displaying the generated questions to the learner, means for inputting and transmitting answers from the learner, means for judging the correctness of the received answers, means for providing the judgement result and explanations to the learner, means for saving the answer history in a database, means for analyzing the answer history to identify the learner's weaknesses, means for using a generative AI model as a means for generating original questions based on the identified weaknesses, and means for re-displaying the generated original questions to the learner. This allows learners to be provided with questions according to their individual academic level, enabling them to progress in their studies while efficiently overcoming their weaknesses.

[0866] A "student" is a user who uses the learning support system to study a subject.

[0867] "Subjects" are areas of study offered by the learning support system, such as mathematics, science, and English.

[0868] "Scope" is the specific part or item within the selected subject that the learner wants to learn.

[0869] The "means for accepting selection" is a mechanism for receiving input of the subject and scope from the learner and transmitting it to the server.

[0870] A "means for generating questions" is an algorithm or generative model for creating specific learning questions based on the subject and scope selected by the learner.

[0871] A "natural language processing algorithm" is a computer software process for analyzing and understanding linguistic data.

[0872] A "generative AI model" is a mathematical model that uses machine learning and artificial intelligence to generate specific problems.

[0873] The "means for displaying questions to the learner" refers to a web interface or application that displays the generated questions on the learner's terminal screen.

[0874] The "means for inputting and sending answers" is an interface that allows a learner to input answers to questions and send them to the server.

[0875] A "means for determining correctness" is an algorithm or program that evaluates whether a learner's answer is correct or incorrect.

[0876] The "means for providing the judgment result and explanation" is a mechanism for displaying to the learner the result of whether the answer is correct or incorrect and an explanation based on that result.

[0877] The "means for storing answer history in a database" is a database system that accumulates information on learners' past answers.

[0878] "Means for analyzing answer history" refers to the process of analyzing saved answer data to identify the learner's weaknesses, etc.

[0879] "Means for generating original questions" are algorithms or models that create new learning questions for specific learners based on the analysis results.

[0880] The "means for re-displaying" is an interface that re-displays the original question that was generated to the learner.

[0881] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[0882] Server part

[0883] The server has the following features:

[0884] 1. Acceptance of subject and scope selection

[0885] The server receives learning requests from the devices and identifies the specific subjects and areas of interest. This is done using web server software such as Apache or Nginx.

[0886] 2. Problem Generation

[0887] The server generates questions based on the requested subject and scope information using a Python script or a generative AI model (e.g., GPT-4).

[0888] 3. Receiving answers and determining whether they are correct or incorrect

[0889] The server receives the answers sent by the user and compares them with the pre-set correct answers to determine whether they are correct or not. This process uses a database such as MySQL.

[0890] 4. Saving and analyzing answer history

[0891] The server stores the answer history in a database and periodically analyzes it using data analysis tools such as Python's Pandas and NumPy.

[0892] 5. Generating original questions

[0893] Based on the analysis results, original questions are generated that address the identified weaknesses, again using a generative AI model.

[0894] 6. Customize your study plan

[0895] The server provides an individually customized learning plan based on the learner's answer history and weakness analysis results. In this process, the customized learning plan is generated in JSON format and sent to the device.

[0896] Terminal part

[0897] The terminal has the following features:

[0898] 1. Submit a learning request

[0899] The user selects the subject and scope through their device and sends that information to the server via a web browser or mobile app interface.

[0900] 2. Displaying the problem

[0901] The device receives the questions from the server and displays them to the user, visually displaying the questions using HTML, CSS, and JavaScript.

[0902] 3. Enter and submit your answers

[0903] The user enters the answer into the terminal and sends it to the server, via a form or input field.

[0904] 4. Viewing Feedback

[0905] The correct / incorrect result sent from the server is displayed to the user. Visual feedback is provided using JavaScript and CSS.

[0906] User Part

[0907] The user does the following:

[0908] 1. Select a learning request

[0909] The user selects the subject and area they wish to study and makes a request on their terminal.

[0910] 2. Answers to the questions

[0911] Answers to questions displayed on the terminal are entered and the answers are sent to the terminal.

[0912] 3. Review feedback

[0913] Check the feedback displayed on your device and learn the correct answers and explanations for questions you got wrong.

[0914] Specific examples

[0915] When learning multiplication in arithmetic

[0916] 1. User makes a learning request

[0917] The user selects arithmetic on the terminal and requests the "multiplication" range.

[0918] 2. Generate a problem on the server

[0919] The server generates a problem such as "28 x 36" and sends it to the terminal.

[0920] 3. The questions are displayed on the device and the user answers them.

[0921] The terminal displays "What is 28 x 36?" and the user enters the answer "1008."

[0922] 4. The server receives the answer and determines whether it is correct or not.

[0923] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[0924] 5. The server saves and analyzes the answer history

[0925] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[0926] 6. The original questions are displayed on the device, and the user answers them again.

[0927] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[0928] Prompt Sentence Examples

[0929] "Please enter the answer to the following question: 28 × 36 = ?"

[0930] This system allows users to study efficiently by providing questions appropriate to their academic level. The server, terminal, and user each play their respective roles effectively and work together to provide learning support.

[0931] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0932] Step 1:

[0933] A user submits a learning request.

[0934] The user selects the subject and area of ​​study they wish to study through their device and enters that request into the device's input form. The entered data (subject and area information) is sent to the server as an HTTP request.

[0935] Step 2:

[0936] The server identifies the subject and scope.

[0937] The server analyzes the HTTP request received from the device and extracts information about the subject and scope. Based on this information, it prepares to generate appropriate questions.

[0938] Step 3:

[0939] The server generates the problem.

[0940] The server creates questions using a Python script or a generative AI model (e.g., GPT-4) based on the received subject and scope information. Specifically, it inputs a prompt to the AI ​​model in the form of "Generate the following problem: multiplication, two-digit x two-digit," and generates an appropriate problem (e.g., "28 x 36"). The output problem data is used in the next processing step.

[0941] Step 4:

[0942] The terminal displays the problem.

[0943] The problem data received from the server is displayed to the user using a user interface built with HTML and JavaScript. Specifically, the question "What is 28 x 36?" is displayed on the terminal screen. This display operation allows the user to visually recognize the problem.

[0944] Step 5:

[0945] The user enters and submits the answer.

[0946] The user enters the answer to the displayed question in the input field and clicks the "Submit" button. The entered answer (e.g., "1008") is again sent to the server as an HTTP request.

[0947] Step 6:

[0948] The server receives the answer and determines whether it is correct.

[0949] The server receives the answer data sent from the terminal and compares it with the correct answer data (e.g., "1008") set in advance. The server determines whether the answer is correct based on the comparison result, generates a correctness judgment result (e.g., "That's correct!"), and outputs it.

[0950] Step 7:

[0951] The device will display the result of the correct answer.

[0952] The result of the correct answer received from the server is displayed to the user. JavaScript is used to display feedback on the device screen such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[0953] Step 8:

[0954] The server stores and analyzes the answer history.

[0955] The server stores the user's answer data and the results of correct / incorrect assessment in a database. The stored data is periodically analyzed and the learner's weaknesses are identified using data analysis tools such as Python's Pandas and NumPy.

[0956] Step 9:

[0957] The server generates the original questions.

[0958] The server generates new problems using a generative AI model to improve weaknesses identified based on the analysis results. For example, it inputs a prompt such as "Generate the following problem: multiplication, two-digit x two-digit," and outputs an original problem such as "34 x 47."

[0959] Step 10:

[0960] The terminal displays the original question and the user answers it again.

[0961] The terminal receives new questions from the server and displays them to the user. The user enters answers to the new questions and submits them again. By repeating this process, the user can efficiently improve their academic ability.

[0962] (Application example 1)

[0963] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0964] While conventional learning support systems contribute to learners' knowledge acquisition, they have limitations in terms of improving practical skills and gaining practical experience in a virtual environment. In particular, there is a lack of effective ways to learn skills required for actual work, such as customer service and sales skills. Furthermore, it is difficult to provide real-time feedback or individual support based on learners' weaknesses.

[0965] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0966] In this invention, the server includes means for accepting a selection of a subject and a range from a learner, means for generating questions based on the subject and range, means for providing the generated questions to the learner, means for receiving answers from the learner, means for determining whether the received answers are correct or incorrect, means for providing the determination result to the learner, means for analyzing the answer history to identify the learner's weaknesses, means for generating original questions based on the identified weaknesses, means for providing the learner with the generated original questions, means for providing the learner with practical skills in a virtual environment, and means for providing the learner with appropriate feedback in real time based on the received answers. This enables the learner to effectively learn skills necessary for actual work in a virtual environment.

[0967] A "learner" is an individual who aims to improve their knowledge and skills by using a learning support system.

[0968] "Subject" refers to the type or field of learning content, including mathematics, science, social studies, etc.

[0969] "Scope" refers to the specific area of ​​study within the selected subject, including specific chapters and topics.

[0970] "Problems" refer to questions or tasks given to assess learners' knowledge and skills.

[0971] "Correctness assessment" is the process of determining whether the answer provided by the learner is correct.

[0972] "Feedback" refers to the evaluation and advice given to a learner in response to their answers.

[0973] An "answer history" is a record of all answers a learner has ever given.

[0974] "Weaknesses" refer to areas where a learner has weaknesses or lacks understanding, identified by analyzing the answer history.

[0975] "Original questions" refer to questions that are specially created to address a learner's weaknesses.

[0976] A "virtual environment" refers to a simulated learning or experiential space created using computers and digital technology.

[0977] "Practical skills" refers to the practical abilities and know-how required in the job.

[0978] "Real-time" refers to immediate response to the information and actions input by the learner.

[0979] A learning support system for specifically implementing this invention comprises three components: a server, a terminal, and a user.

[0980] Server part

[0981] The server implements the following functions:

[0982] 1. Accepting learning requests

[0983] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[0984] 2. Problem Generation

[0985] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[0986] 3. Receiving answers and determining whether they are correct or incorrect

[0987] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[0988] 4. Saving and analyzing answer history

[0989] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[0990] 5. Generating original questions

[0991] Original questions are generated to reinforce weak points identified from the analysis results.

[0992] 6. Customize your study plan

[0993] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[0994] 7. Providing practical skills in a virtual environment

[0995] Provide simulations that allow learners to gain hands-on experience and learn the skills they need to succeed in a virtual environment.

[0996] 8. Real-time feedback

[0997] Provide appropriate feedback in real time based on the answers provided by the learner.

[0998] Terminal part

[0999] The device implements the following features:

[1000] 1. Submit a learning request

[1001] Learners select the subject and scope through their terminal and send that information to the server.

[1002] 2. Displaying the problem

[1003] The terminal displays the questions received from the server to the learner.

[1004] 3. Enter and submit your answers

[1005] The learner enters the answer and sends it to the server via the terminal.

[1006] 4. Viewing Feedback

[1007] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1008] 5. Running a Virtual Environment

[1009] The devices use smart glasses or head-mounted displays to display virtual environments, allowing learners to learn practical skills.

[1010] User Part

[1011] The user (learner) performs the following operations:

[1012] 1. Select a learning request

[1013] Select the subject and area you want to study and make a request on your device.

[1014] 2. Answers to the questions

[1015] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1016] 3. Review feedback

[1017] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1018] 4. Practical skills training in a virtual environment

[1019] In a virtual environment, practical skills can be learned through simulations, such as practicing customer service and sales skills in a virtual store.

[1020] Hardware and software used

[1021] The server runs programs using languages ​​and frameworks such as Python and Django.

[1022] The terminal provides applications that run on a variety of devices, including smartphones, tablets, smart glasses, and head-mounted displays.

[1023] The database uses a relational database management system such as MySQL or PostgreSQL.

[1024] The generative AI model uses machine learning and natural language processing frameworks (such as TensorFlow and PyTorch) to generate questions and provide feedback.

[1025] Specific examples

[1026] If you're learning customer service skills in a virtual environment:

[1027] 1. User makes a learning request

[1028] The user selects "customer service skills" on the terminal and sends a learning request.

[1029] 2. Generate a problem on the server

[1030] The server generates questions such as "If a customer asks you a question about a product, how would you respond?" and sends them to the terminal.

[1031] 3. The questions are displayed on the device and the user answers them.

[1032] The terminal displays the virtual environment and initiates a dialogue with the virtual customer, who inputs answers and sends them to the server.

[1033] 4. The server receives the answer, determines whether it is correct, and provides feedback.

[1034] The server analyzes the answer and provides real-time feedback such as, "The answer given to the customer was appropriate. Here's what would be best next time."

[1035] Prompt Sentence Examples

[1036] Use the following information to generate questions to teach customer service skills in a virtual store:

[1037] Categories: Product knowledge, complaint handling, sales skills

[1038] Example: Describe the features of this product.

[1039] In this way, learners can effectively learn practical skills in a virtual environment.

[1040] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1041] Step 1:

[1042] The terminal accepts learning requests from users. The user selects the subject and scope of the learning they wish to do through the terminal interface and sends the request. The input is the subject and scope information, and the output is the request data sent to the server.

[1043] Step 2:

[1044] The server receives request data sent from the terminal. The input is the learning request data, and the output is a trigger to invoke the problem generation algorithm based on the request data.

[1045] Step 3:

[1046] The server generates questions based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history. The input is the subject and scope information and past answer history data, and the output is the generated question data.

[1047] Step 4:

[1048] The server sends the generated questions to the terminal. The input is the generated question data, and the output is the question data sent to the terminal.

[1049] Step 5:

[1050] The terminal displays the questions sent from the server to the learner. The input is the question data, and the output is the question displayed on the terminal interface.

[1051] Step 6:

[1052] The user answers the questions displayed on the terminal. The user enters the answer into the terminal and presses the send button. The input is the user's answer, and the output is the answer data sent to the server.

[1053] Step 7:

[1054] The server receives the answer from the user and judges whether the answer is correct or not. The input is the user's answer data, and the output is the result of the judgment.

[1055] Step 8:

[1056] The server sends the result of the accuracy judgment to the terminal. The input is the accuracy judgment result, and the output is feedback data sent to the terminal.

[1057] Step 9:

[1058] The terminal displays the result of the judgment and explanation from the server to the user. The input is the feedback data, and the output is the feedback displayed on the terminal interface.

[1059] Step 10:

[1060] The server stores the user's answer history in a database. The input is the answer data and the result of the correct / incorrect judgment, and the output is the answer history data stored in the database.

[1061] Step 11:

[1062] The server periodically analyzes the answer history and identifies the user's weaknesses. The input is the answer history data, and the output is the analysis results and the identified weaknesses.

[1063] Step 12:

[1064] The server generates original questions based on the identified weaknesses. The input is the weakness analysis result, and the output is the generated original question data.

[1065] Step 13:

[1066] The server sends the generated original question to the terminal. The input is the original question data, and the output is the original question data sent to the terminal.

[1067] Step 14:

[1068] The terminal displays the original problem to the user. The input is the original problem data, and the output is the original problem displayed on the interface.

[1069] Step 15:

[1070] The terminal runs the virtual environment and allows users to experience practical skills. The input is the data of the virtual environment simulation, and the output is the virtual environment that the user can view and manipulate.

[1071] Step 16:

[1072] The server provides appropriate feedback in real time based on the answers provided by the user. The input is real-time answer data, and the output is real-time feedback data.

[1073] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1074] A learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine.

[1075] System Configuration

[1076] Server part

[1077] The server implements the following functions:

[1078] 1. Acceptance of subject and scope selection

[1079] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[1080] 2. Problem Generation

[1081] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1082] 3. Receiving answers and determining whether they are correct or incorrect

[1083] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1084] 4. Saving and analyzing answer history

[1085] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[1086] 5. Generating original questions

[1087] Original questions are generated to reinforce weak points identified from the analysis results.

[1088] 6. Customize your study plan

[1089] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1090] 7. Emotion Data Analysis

[1091] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[1092] Terminal part

[1093] The device implements the following features:

[1094] 1. Submit a learning request

[1095] Learners select the subject and scope through their terminal and send that information to the server.

[1096] 2. Displaying the problem

[1097] The terminal displays the questions received from the server to the learner.

[1098] 3. Enter and submit your answers

[1099] The learner enters the answer and sends it to the server via the terminal.

[1100] 4. Viewing Feedback

[1101] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1102] 5. Collecting Emotional Data

[1103] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[1104] User Part

[1105] The user does the following:

[1106] 1. Select a learning request

[1107] Select the subject and area you want to study and make a request on your device.

[1108] 2. Answers to the questions

[1109] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1110] 3. Review feedback

[1111] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1112] 4. Providing Emotion Data

[1113] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[1114] Emotion engine part

[1115] The emotion engine implements the following functions:

[1116] 1. Emotion recognition and collection

[1117] Recognize and collect emotions from learners' facial expressions and voice data.

[1118] 2. Transmission of emotional information

[1119] The recognized emotion information is sent to the server.

[1120] 3. Emotion-Based Adjustment

[1121] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[1122] Specific examples

[1123] Learning multiplication in arithmetic using an emotion engine

[1124] 1. User makes a learning request

[1125] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[1126] 2. Generate a problem on the server

[1127] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1128] 3. Display the problem on the device and collect sentiment data

[1129] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[1130] 4. The user enters the answer

[1131] The user enters the answer "1008".

[1132] 5. The server receives the answer and analyzes the emotion data.

[1133] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[1134] 6. Feedback and difficulty adjustments

[1135] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[1136] 7. View feedback on your device

[1137] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[1138] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[1139] The processing flow will be explained below.

[1140] Step 1:

[1141] The user starts the learning application on the device and selects the subject and area they want to study. Let's assume that they select "Multiplication" in "Math."

[1142] Step 2:

[1143] The terminal transmits information about the subject and range selected by the user to the server.

[1144] Step 3:

[1145] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36."

[1146] Step 4:

[1147] The server sends the generated questions to the terminal.

[1148] Step 5:

[1149] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[1150] Step 6:

[1151] At the same time as displaying the image, the device sends the user's facial expressions and voice data to the emotion engine.

[1152] Step 7:

[1153] The emotion engine recognizes and analyzes the user's emotions from the facial and voice data it receives. For example, if the user has a troubled expression, it will determine that they are feeling stressed.

[1154] Step 8:

[1155] The user answers the displayed question and enters the answer into the terminal. Let's consider the case where the user enters "1008."

[1156] Step 9:

[1157] The device sends the user's answer to the server and also sends the emotion engine's analysis results to the server.

[1158] Step 10:

[1159] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[1160] Step 11:

[1161] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[1162] Step 12:

[1163] The server stores the answer history in a database and analyzes it, identifying the learner's weaknesses based on the analysis results.

[1164] Step 13:

[1165] The server adjusts the next learning step based on the analysis results and data from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, it adjusts the difficulty of the questions.

[1166] Step 14:

[1167] The server generates new original questions and sends the information to the terminal. It generates new questions such as "What is 34 x 47?"

[1168] Step 15:

[1169] The terminal displays the original questions received from the server to the user.

[1170] Step 16:

[1171] The device again sends the user's facial expressions and voice data to the emotion engine.

[1172] Step 17:

[1173] The emotion engine recognizes and analyzes emotions from new facial and voice data.

[1174] Step 18:

[1175] The user answers the original question displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[1176] Step 19:

[1177] The device sends the user's answer to the server, as well as the emotion engine's new analysis results.

[1178] Step 20:

[1179] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[1180] Step 21:

[1181] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[1182] Step 22:

[1183] The device displays the feedback received from the server to the user, including encouragement and advice based on the user's emotions.

[1184] Step 23:

[1185] By repeating this series of processes, users can reinforce their weaknesses while progressing with optimal learning based on emotional data.

[1186] Example 2

[1187] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1188] Conventional learning support systems generate questions based on the learner's academic level and reinforce weak points by analyzing the learner's answer history, but they lack the ability to individually customize the system to take into account the learner's emotional state. As a result, learners' stress and decreased motivation can affect their learning outcomes. Therefore, the present invention aims to provide a more effective and personalized learning support system that takes into account the learner's emotional state.

[1189] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1190] In this invention, the server includes means for receiving selection of a subject and range from the learner, means for generating questions based on the subject and range, and means for analyzing collected emotional data to adjust the difficulty of the study plan and questions, thereby making it possible to provide a study plan that takes into account the emotional state of the learner.

[1191] "Learner" refers to a person who uses the system to acquire knowledge or skills.

[1192] A "subject" refers to a particular field or topic that a learner will study.

[1193] "Scope" refers to the part of the subject that the learner particularly wants to study.

[1194] "Problems" are questions or tasks provided to assess a learner's understanding or skills.

[1195] "Answer" refers to the answer to the question submitted by the learner.

[1196] "Correct or incorrect" refers to the result of determining whether the learner's answer is correct or incorrect.

[1197] "Answer history" refers to a record of answers previously provided by a learner.

[1198] "Weak points" refer to areas in which a learner is particularly weak, as identified by analyzing the answer history.

[1199] "Original questions" refer to questions that are specially created to reinforce a learner's weak points.

[1200] "Emotional data" refers to information about emotions collected from learners' facial expressions, voice, etc.

[1201] "Emotion engine" refers to technology for analyzing collected emotional data and providing the results to the system.

[1202] "Study plan" refers to the learning progress plan and content provided to learners.

[1203] "Database" refers to an information management system for storing learners' answer history and other information.

[1204] The learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine. The functions and specific processes of each component will be explained in detail below.

[1205] System Configuration

[1206] Server part

[1207] The server has the following functions:

[1208] 1. Acceptance of subject and scope selection

[1209] The server receives the learning request sent from the device and identifies the specific subject and area, and stores this information in a database.

[1210] Example: When a request for "arithmetic" and "multiplication" is sent from the terminal, the server receives it.

[1211] 2. Problem Generation

[1212] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1213] Example: Generate questions of appropriate difficulty, such as "15 x 12," based on the learner's academic level.

[1214] 3. Receiving answers and determining whether they are correct or incorrect

[1215] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1216] Example: A learner sends the answer "180" to the server, and the server determines this to be the "correct answer."

[1217] 4. Saving and analyzing answer history

[1218] The server stores the answer history in a database and periodically analyzes it to identify the learner's weak points.

[1219] Example: Save a history of frequent mistakes a learner makes on multiplication problems and analyze the data.

[1220] 5. Generating original questions

[1221] Original questions are generated to reinforce weak points identified from the analysis results.

[1222] For example, if a weakness in "multiplication" is identified, generate and provide similar multiplication problems.

[1223] 6. Customize your study plan

[1224] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1225] Example: Propose a lesson plan aimed at strengthening multiplication skills.

[1226] 7. Emotion Data Analysis

[1227] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[1228] For example: temporarily reducing the difficulty of questions when a learner is feeling stressed.

[1229] Terminal part

[1230] The device has the following features:

[1231] 1. Submit a learning request

[1232] Learners select the subject and scope through their terminal and send that information to the server.

[1233] Example: If a learner selects "arithmetic" and "multiplication", the selection is sent to the server.

[1234] 2. Displaying the problem

[1235] The terminal displays the questions received from the server to the learner.

[1236] For example, display the question as "What is 15 x 12?"

[1237] 3. Enter and submit your answers

[1238] The learner enters the answer and sends it to the server via the terminal.

[1239] Example: Enter the answer "180" and submit.

[1240] 4. Viewing Feedback

[1241] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1242] Example: Display "You got it right! Congratulations!"

[1243] 5. Collecting Emotional Data

[1244] The device collects the learner's facial expressions and voice data and sends it to the emotion engine.

[1245] Example: Recognizing a user's smile through a camera and sending that data to an emotion engine.

[1246] User Part

[1247] The user performs the following operations:

[1248] 1. Select a learning request

[1249] Select the subject and area you want to study and make a request on your device.

[1250] Example: User selects "Math" and "Multiplication".

[1251] 2. Answers to the questions

[1252] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1253] For example, enter "15 × 12 = 180".

[1254] 3. Review feedback

[1255] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1256] For example: "You got it right! Congratulations!"

[1257] 4. Providing Emotion Data

[1258] Emotions during training are collected by the device, and facial and voice data are analyzed by the emotion engine.

[1259] For example, when a learner smiles while solving a problem, their facial expression data is analyzed.

[1260] Emotion engine part

[1261] The emotion engine has the following functions:

[1262] 1. Emotion recognition and collection

[1263] Recognize and collect emotions from learners' facial expressions and voice data.

[1264] Example: Recognizing "joy" from a learner's smiling face.

[1265] 2. Transmission of emotional information

[1266] The recognized emotion information is sent to the server.

[1267] Example: Sending learner "joy" data to the server.

[1268] 3. Emotion-Based Adjustment

[1269] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[1270] For example: If a learner is feeling stressed, provide them with easy problems.

[1271] Specific examples

[1272] Example of learning multiplication using emotion engine

[1273] 1. User makes a learning request

[1274] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[1275] 2. Generate a problem on the server

[1276] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1277] 3. Display the problem on the device and collect sentiment data

[1278] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[1279] 4. The user enters the answer

[1280] The user enters the answer "1008".

[1281] 5. The server receives the answer and analyzes the emotion data.

[1282] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[1283] 6. Feedback and difficulty adjustments

[1284] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[1285] 7. View feedback on your device

[1286] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[1287] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[1288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1289] Program processing steps and detailed explanations

[1290] Step 1:

[1291] Submit a learning request

[1292] Terminal processing: An interface is displayed that allows the user to select the subject and range of study they wish to do via the terminal. The user selects "arithmetic" and "multiplication" and sends the request.

[1293] Input: User selection information for the subject "arithmetic" and range "multiplication".

[1294] Output: The training request sent to the server.

[1295] Specific operation: The terminal sends the selected subject and range to the server as an HTTP request.

[1296] Step 2:

[1297] Processing subject and scope information

[1298] Server processing: The server analyzes the received learning request, stores it in a database, and then performs preprocessing to generate appropriate questions.

[1299] Input: The training request sent from the device.

[1300] Output: Subject and scope information stored in database.

[1301] Specific operation: The server receives the HTTP request and records it in the database.

[1302] Step 3:

[1303] Problem Generation

[1304] Server processing: The server runs a question generation algorithm based on the subject and scope information stored in the database, taking into account the learner's past answer history and academic level.

[1305] Input: subject and scope information, past answer history, academic level information.

[1306] Output: The generated questions.

[1307] Specific operation: The server retrieves the necessary information from the database and generates a problem such as "15 x 12".

[1308] Step 4:

[1309] Viewing Issues

[1310] Terminal processing: The generated questions sent from the server are displayed to the user.

[1311] Input: The generated question sent by the server.

[1312] Output: The problem as it appears on the user's screen.

[1313] Specific operation: The terminal receives the response from the server and displays the question "What is 15 x 12?"

[1314] Step 5:

[1315] Enter your answer

[1316] User operation: The user inputs an answer to the displayed question. For example, the answer is "180."

[1317] Input: The user's answer.

[1318] Output: The answer typed into the terminal.

[1319] Specific action: The user enters "180" in the input field of the terminal.

[1320] Step 6:

[1321] Submit your answer

[1322] Terminal processing: The terminal sends the user's answer to the server.

[1323] Input: The user's answer.

[1324] Output: The answer sent to the server.

[1325] Specific operation: The terminal sends the entered answer "180" to the server via an HTTP request.

[1326] Step 7:

[1327] Judging whether the answer is correct or incorrect

[1328] Server processing: The server compares the received answer with the correct answer set in advance and determines whether it is correct or incorrect.

[1329] Input: User's answer "180".

[1330] Output: Correct / incorrect result.

[1331] Specific operation: The server compares the answer "180" with the correct answer "180" and determines that it is correct.

[1332] Step 8:

[1333] Save answer history

[1334] Server processing: The answer results are saved in a database and recorded as a history for future analysis.

[1335] Input: User's answer and verdict.

[1336] Output: Answer history stored in a database.

[1337] Specific operation: The server stores the answers and judgment results in a database.

[1338] Step 9:

[1339] Collecting Emotional Data

[1340] Device processing: The device collects the learner's facial expressions and voice data and sends them to the emotion engine.

[1341] Input: Learner's facial expressions and voice data.

[1342] Output: Emotion data sent to the emotion engine.

[1343] Specific operation: The device collects the user's emotional data through the camera and microphone and sends it to the emotion engine.

[1344] Step 10:

[1345] Emotional Data Analysis

[1346] Emotion Engine Processing: The emotion engine analyzes the received data and determines the learner's current emotional state.

[1347] Input: Emotion data sent from the device.

[1348] Output: Learner's emotional state.

[1349] Specific operation: The emotion engine uses a facial expression recognition algorithm to analyze the user's smile and stress state.

[1350] Step 11:

[1351] Generate customized feedback

[1352] Server processing: Based on answer history and emotion data, appropriate feedback and the next learning plan are generated.

[1353] Input: Answer history and emotion data.

[1354] Output: Customized feedback and study plan.

[1355] Specific operation: The server takes into account the learner's situation and generates feedback such as "That's correct! The next question will be easier."

[1356] Step 12:

[1357] View Feedback

[1358] Terminal processing: Display the feedback sent from the server to the user.

[1359] Input: Customized feedback.

[1360] Output: Feedback displayed on the user's screen.

[1361] Specific operation: The device displays "Correct! The next question will be easy."

[1362] By following these processing steps, personalized learning support can be provided to the learner.

[1363] (Application example 2)

[1364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1365] Conventional learning support systems are unable to take into account learners' emotions or real-time reactions, and because learning plans and questions are provided uniformly, effective personalization is difficult. Furthermore, it is difficult to utilize customer emotional data in brick-and-mortar stores, resulting in a lack of services and proposals that meet customer needs. A system that can solve these problems and enable more effective and personalized learning and customer service is needed.

[1366] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a subject and range selection from a learner; means for generating questions based on the subject and range; means for providing the generated questions to the learner; means for receiving answers from the learner; means for determining whether the received answers are correct or incorrect; means for providing the determination results to the learner; means for analyzing the answer history to identify the learner's weaknesses; means for generating original questions based on the identified weaknesses; means for providing the generated original questions to the learner; means for collecting emotional data; means for analyzing the collected emotional data; means for providing a learning plan or feedback corresponding to the learner's emotions based on the analysis results; and means for analyzing customer emotional data in real time at a physical store and displaying information and suggestions suitable for the customer. This enables learning support optimized for individual learners and further enables services to be provided in physical stores that meet customer needs.

[1367] "Learner" refers to an individual who is learning educational content and is a user who intends to acquire knowledge and skills using this system.

[1368] "Means for accepting selection of subject and scope" is a function that provides an interface for a learner to specify a particular subject and scope that they wish to study.

[1369] The "means for generating questions" is a function that automatically creates questions that are relevant to the content based on the subject and scope selected by the learner.

[1370] The "means for providing the generated questions to the learner" is a function for displaying the questions generated by the server on the learner's terminal.

[1371] The "means for receiving answers" is a function for sending answers entered by learners to the server.

[1372] The "means for determining whether the answer is correct or incorrect" is a function that compares the learner's answer with pre-set correct answer data and determines whether the answer is correct or incorrect.

[1373] "Means for identifying learners' weaknesses by analyzing answer history" is a function that analyzes a learner's past answer data and identifies areas and question types in which the learner is weak.

[1374] The "means for generating original questions" is a function for creating new questions specifically for strengthening the weak points of identified learners.

[1375] "Means for collecting emotional data" is a function for obtaining emotional data from the facial expressions, voices, etc. of learners and customers.

[1376] "Means for analyzing emotional data" is a function for analyzing the emotional state of learners and customers based on collected emotional data.

[1377] "Means for providing learning plans and feedback" refers to a function that provides learning plans and feedback that correspond to the learner's emotions based on the analysis results.

[1378] "A means of analyzing customer emotional data in real time in physical stores and displaying information and suggestions appropriate to the customer" is a function that instantly analyzes customer emotions in stores and suggests the most suitable products and services to the customer based on the results.

[1379] MODE FOR CARRYING OUT THE INVENTION

[1380] The learning support system and customer service support system for brick-and-mortar stores of the present invention are composed of four components: a server, a terminal, a user, and an emotion engine.

[1381] System Configuration

[1382] Server part

[1383] The server implements the following functions:

[1384] 1. Acceptance of subject and scope selection

[1385] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[1386] 2. Problem Generation

[1387] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1388] 3. Receiving answers and determining whether they are correct or incorrect

[1389] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1390] 4. Saving and analyzing answer history

[1391] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[1392] 5. Generating original questions

[1393] Original questions are generated to reinforce weak points identified from the analysis results.

[1394] 6. Customize your study plan

[1395] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1396] 7. Emotion Data Analysis

[1397] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[1398] 8. Analysis and Proposal of Emotional Data in Physical Stores

[1399] Customer sentiment data obtained in physical stores is analyzed in real time, and the results are used to provide optimal product suggestions and customer service methods to customers.

[1400] Terminal part

[1401] The device implements the following features:

[1402] 1. Submit a learning request

[1403] Learners select the subject and scope through their terminal and send that information to the server.

[1404] 2. Displaying the problem

[1405] The terminal displays the questions received from the server to the learner.

[1406] 3. Enter and submit your answers

[1407] The learner enters the answer and sends it to the server via the terminal.

[1408] 4. Viewing Feedback

[1409] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1410] 5. Collecting Emotional Data

[1411] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[1412] User Part

[1413] The user does the following:

[1414] 1. Select a learning request

[1415] Select the subject and area you want to study and make a request on your device.

[1416] 2. Answers to the questions

[1417] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1418] 3. Review feedback

[1419] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1420] 4. Providing Emotion Data

[1421] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[1422] Emotion engine part

[1423] The emotion engine implements the following functions:

[1424] 1. Emotion recognition and collection

[1425] Recognize and collect emotions from learners' and customers' facial expressions and voice data.

[1426] 2. Transmission of emotional information

[1427] The recognized emotion information is sent to the server.

[1428] 3. Emotion-Based Adjustment

[1429] Based on the emotional information provided to the server, the system will assess the learner's stress level and concentration level, and adjust the difficulty of the questions and the content of the feedback. In addition, in the case of a physical store, the system will provide appropriate information and product suggestions based on the customer's emotions.

[1430] Specific examples

[1431] Learning multiplication in arithmetic using an emotion engine

[1432] 1. User makes a learning request

[1433] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[1434] 2. Generate a problem on the server

[1435] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1436] 3. Display the problem on the device and collect sentiment data

[1437] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[1438] 4. The user enters the answer

[1439] The user enters the answer "1008".

[1440] 5. The server receives the answer and analyzes the emotion data.

[1441] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[1442] 6. Feedback and difficulty adjustments

[1443] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[1444] 7. View feedback on your device

[1445] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[1446] Emotion analysis and customer support in brick-and-mortar stores

[1447] 1. The customer is choosing a product

[1448] Smart glasses in physical stores detect customers' facial expressions and voices.

[1449] 2. The emotion engine analyzes the emotion data

[1450] If the customer is confused, the emotion engine sends this data to the server.

[1451] 3. The server displays the proposal

[1452] Based on the analysis results, the server displays appropriate product suggestions and explanations on the smart glasses.

[1453] 4. Staff make suggestions

[1454] Staff will then make optimal product recommendations to customers based on the displayed information.

[1455] Example prompt for a generative AI model:

[1456] "If a customer is interested in your product but seems confused, what explanation or suggestion would be most appropriate? For example, explaining new features or showing examples of use."

[1457] This allows for effective and personalized assistance that takes into account the emotions of learners and customers.

[1458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1459] Step 1:

[1460] The learner selects the subject and range on the terminal and sends a learning request to the server. The input is the subject and range selected by the learner. The output is the learning request data sent to the server. Here, the terminal provides the interface and the learner makes the selection.

[1461] Step 2:

[1462] Based on the learning request data received by the server, questions are generated for the relevant subject and scope. The input is the learning request data, and the output is the generated question data. To do this, the server retrieves relevant information from a database and creates appropriate questions using an algorithm.

[1463] Step 3:

[1464] The server sends the generated questions to the terminal, which then displays them to the learner. The input is the generated question data, and the output is the question displayed on the terminal. The terminal displays the question using a display interface.

[1465] Step 4:

[1466] The learner inputs the answer through the terminal and sends it to the server. The input is the answer entered by the learner, and the output is the answer data sent to the server. The terminal provides the answer interface, and the learner inputs the answer.

[1467] Step 5:

[1468] The server compares the received answer with the correct answer data to determine whether it is correct or not. The input is the learner's answer data and the preset correct answer data, and the output is the result of the correct or incorrect decision. The server uses logic to perform the comparison and calculate the result.

[1469] Step 6:

[1470] The server sends the judgment result to the terminal, which displays it to the learner. The input is the correct / incorrect judgment result, and the output is the feedback displayed on the terminal. The terminal provides a feedback screen and displays the result.

[1471] Step 7:

[1472] The device collects the learner's facial and voice data and sends it to the emotion engine. The input is facial and voice data, and the output is emotion data that is sent to the emotion engine. The device captures data using sensors.

[1473] Step 8:

[1474] The emotion engine analyzes the received emotion data and sends the analysis results to the server. The input is emotion data and the output is the analysis results. The emotion engine analyzes the data using emotion recognition algorithms.

[1475] Step 9:

[1476] The server adjusts the difficulty of questions and feedback based on the results of sentiment analysis. The input is the analysis results, and the output is the adjusted questions and feedback data. The server then processes the learning plan to dynamically customize it.

[1477] Step 10:

[1478] The server sends the adjusted questions and feedback to the device, which then displays them to the learner. The input is the adjusted data from the server, and the output is the adjusted content displayed on the device. The device again updates the interface and displays the information.

[1479] Step 11:

[1480] In a physical store, smart glasses collect customer facial and voice data and send it to an emotion engine. The input is the customer's facial and voice data, and the output is emotion data sent to the emotion engine. The smart glasses act as a collection device.

[1481] Step 12:

[1482] The emotion engine analyzes customer emotion data in real time and sends the analysis results to the server. The input is emotion data, and the output is the real-time analysis results. The emotion engine quickly performs data calculations.

[1483] Step 13:

[1484] The server sends and displays appropriate product and service suggestions based on the analysis results to the smart glasses. The input is the analysis results and suggestion data, and the output is the suggestions displayed on the smart glasses. The server generates and sends the suggestions, which are then displayed by the smart glasses.

[1485] This will enable personalized learning and customer support that takes into account real-time emotional data from learners and customers.

[1486] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1487] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1488] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1489] [Third embodiment]

[1490] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1491] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1492] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1493] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1494] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1495] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1496] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1497] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1498] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1499] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1500] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1501] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1502] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[1503] System Configuration

[1504] Server part

[1505] The server implements the following functions:

[1506] 1. Acceptance of subject and scope selection

[1507] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[1508] 2. Problem Generation

[1509] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1510] 3. Receiving answers and determining whether they are correct or incorrect

[1511] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1512] 4. Saving and analyzing answer history

[1513] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[1514] 5. Generating original questions

[1515] Original questions are generated to reinforce weak points identified from the analysis results.

[1516] 6. Customize your study plan

[1517] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1518] Terminal part

[1519] The device implements the following features:

[1520] 1. Submit a learning request

[1521] Learners select the subject and scope through their terminal and send that information to the server.

[1522] 2. Displaying the problem

[1523] The terminal displays the questions received from the server to the learner.

[1524] 3. Enter and submit your answers

[1525] The learner enters the answer and sends it to the server via the terminal.

[1526] 4. Viewing Feedback

[1527] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1528] User Part

[1529] The user does the following:

[1530] 1. Select a learning request

[1531] Select the subject and area you want to study and make a request on your device.

[1532] 2. Answers to the questions

[1533] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1534] 3. Review feedback

[1535] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1536] Specific examples

[1537] When learning multiplication in arithmetic

[1538] 1. User makes a learning request

[1539] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[1540] 2. Generate a problem on the server

[1541] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1542] 3. The questions are displayed on the device and the user answers them.

[1543] The terminal displays "What is 28 x 36?" and the user answers "1008."

[1544] 4. The server receives the answer and determines whether it is correct or not.

[1545] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[1546] 5. The server saves and analyzes the answer history

[1547] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[1548] 6. The original questions are displayed on the device, and the user answers them again.

[1549] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[1550] In this way, learners can effectively improve their learning ability through repeated practice.

[1551] The processing flow will be explained below.

[1552] Step 1:

[1553] A user starts a learning application using a device and selects the subject and area they want to study. Let's consider the case where they select "Multiplication" in "Math."

[1554] Step 2:

[1555] The terminal transmits information about the subject and range selected by the user to the server.

[1556] Step 3:

[1557] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36".

[1558] Step 4:

[1559] The server sends the generated questions to the terminal.

[1560] Step 5:

[1561] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[1562] Step 6:

[1563] The user solves the displayed problem and enters the answer into the terminal. Let's consider the case where the user enters "1008".

[1564] Step 7:

[1565] The terminal transmits the user's answer to the server.

[1566] Step 8:

[1567] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[1568] Step 9:

[1569] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[1570] Step 10:

[1571] The terminal displays the feedback received from the server to the user.

[1572] Step 11:

[1573] The server stores the user's answer history in a database and analyzes it, particularly to determine which types of questions the user is weak at.

[1574] Step 12:

[1575] The server identifies the user's weaknesses based on the analysis results and generates original questions that focus on those weaknesses, such as "34 x 47."

[1576] Step 13:

[1577] The server sends the generated original questions to the terminal.

[1578] Step 14:

[1579] The terminal displays the original question received from the server to the user: "What is 34 x 47?"

[1580] Step 15:

[1581] The user solves the original problem displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[1582] Step 16:

[1583] The terminal transmits the user's answer to the server.

[1584] Step 17:

[1585] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[1586] Step 18:

[1587] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[1588] Step 19:

[1589] The terminal displays the feedback received from the server to the user.

[1590] Step 20:

[1591] Users can check the feedback, understand their weaknesses, and continue their studies. By repeating this process, users can progress through their studies efficiently.

[1592] Example 1

[1593] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1594] Conventional learning systems have difficulty providing appropriate questions that match the individual academic ability of each learner, and lack the ability to fully utilize answer history during the learning process to identify weak points and generate questions based on those weak points. As a result, learners have had difficulty improving their academic ability efficiently.

[1595] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1596] In this invention, the server includes means for accepting a selection of a subject and range from a learner, means for using a natural language processing algorithm as a means for generating questions based on the subject and range, means for displaying the generated questions to the learner, means for inputting and transmitting answers from the learner, means for judging the correctness of the received answers, means for providing the judgement result and explanations to the learner, means for saving the answer history in a database, means for analyzing the answer history to identify the learner's weaknesses, means for using a generative AI model as a means for generating original questions based on the identified weaknesses, and means for re-displaying the generated original questions to the learner. This allows learners to be provided with questions according to their individual academic level, enabling them to progress in their studies while efficiently overcoming their weaknesses.

[1597] A "student" is a user who uses the learning support system to study a subject.

[1598] "Subjects" are areas of study offered by the learning support system, such as mathematics, science, and English.

[1599] "Scope" is the specific part or item within the selected subject that the learner wants to learn.

[1600] The "means for accepting selection" is a mechanism for receiving input of the subject and scope from the learner and transmitting it to the server.

[1601] A "means for generating questions" is an algorithm or generative model for creating specific learning questions based on the subject and scope selected by the learner.

[1602] A "natural language processing algorithm" is a computer software process for analyzing and understanding linguistic data.

[1603] A "generative AI model" is a mathematical model that uses machine learning and artificial intelligence to generate specific problems.

[1604] The "means for displaying questions to the learner" refers to a web interface or application that displays the generated questions on the learner's terminal screen.

[1605] The "means for inputting and sending answers" is an interface that allows a learner to input answers to questions and send them to the server.

[1606] A "means for determining correctness" is an algorithm or program that evaluates whether a learner's answer is correct or incorrect.

[1607] The "means for providing the judgment result and explanation" is a mechanism for displaying to the learner the result of whether the answer is correct or incorrect and an explanation based on that result.

[1608] The "means for storing answer history in a database" is a database system that accumulates information on learners' past answers.

[1609] "Means for analyzing answer history" refers to the process of analyzing saved answer data to identify the learner's weaknesses, etc.

[1610] "Means for generating original questions" are algorithms or models that create new learning questions for specific learners based on the analysis results.

[1611] The "means for re-displaying" is an interface that re-displays the original question that was generated to the learner.

[1612] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[1613] Server part

[1614] The server has the following features:

[1615] 1. Acceptance of subject and scope selection

[1616] The server receives learning requests from the devices and identifies the specific subjects and areas of interest. This is done using web server software such as Apache or Nginx.

[1617] 2. Problem Generation

[1618] The server generates questions based on the requested subject and scope information using a Python script or a generative AI model (e.g., GPT-4).

[1619] 3. Receiving answers and determining whether they are correct or incorrect

[1620] The server receives the answers sent by the user and compares them with the pre-set correct answers to determine whether they are correct or not. This process uses a database such as MySQL.

[1621] 4. Saving and analyzing answer history

[1622] The server stores the answer history in a database and periodically analyzes it using data analysis tools such as Python's Pandas and NumPy.

[1623] 5. Generating original questions

[1624] Based on the analysis results, original questions are generated that address the identified weaknesses, again using a generative AI model.

[1625] 6. Customize your study plan

[1626] The server provides an individually customized learning plan based on the learner's answer history and weakness analysis results. In this process, the customized learning plan is generated in JSON format and sent to the device.

[1627] Terminal part

[1628] The terminal has the following features:

[1629] 1. Submit a learning request

[1630] The user selects the subject and scope through their device and sends that information to the server via a web browser or mobile app interface.

[1631] 2. Displaying the problem

[1632] The device receives the questions from the server and displays them to the user, visually displaying the questions using HTML, CSS, and JavaScript.

[1633] 3. Enter and submit your answers

[1634] The user enters the answer into the terminal and sends it to the server, via a form or input field.

[1635] 4. Viewing Feedback

[1636] The correct / incorrect result sent from the server is displayed to the user. Visual feedback is provided using JavaScript and CSS.

[1637] User Part

[1638] The user does the following:

[1639] 1. Select a learning request

[1640] The user selects the subject and area they wish to study and makes a request on their terminal.

[1641] 2. Answers to the questions

[1642] Answers to questions displayed on the terminal are entered and the answers are sent to the terminal.

[1643] 3. Review feedback

[1644] Check the feedback displayed on your device and learn the correct answers and explanations for questions you got wrong.

[1645] Specific examples

[1646] When learning multiplication in arithmetic

[1647] 1. User makes a learning request

[1648] The user selects arithmetic on the terminal and requests the "multiplication" range.

[1649] 2. Generate a problem on the server

[1650] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1651] 3. The questions are displayed on the device and the user answers them.

[1652] The terminal displays "What is 28 x 36?" and the user enters the answer "1008."

[1653] 4. The server receives the answer and determines whether it is correct or not.

[1654] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[1655] 5. The server saves and analyzes the answer history

[1656] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[1657] 6. The original questions are displayed on the device, and the user answers them again.

[1658] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[1659] Prompt Sentence Examples

[1660] "Please enter the answer to the following question: 28 × 36 = ?"

[1661] This system allows users to study efficiently by providing questions appropriate to their academic level. The server, terminal, and user each play their respective roles effectively and work together to provide learning support.

[1662] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1663] Step 1:

[1664] A user submits a learning request.

[1665] The user selects the subject and area of ​​study they wish to study through their device and enters that request into the device's input form. The entered data (subject and area information) is sent to the server as an HTTP request.

[1666] Step 2:

[1667] The server identifies the subject and scope.

[1668] The server analyzes the HTTP request received from the device and extracts information about the subject and scope. Based on this information, it prepares to generate appropriate questions.

[1669] Step 3:

[1670] The server generates the problem.

[1671] The server creates questions using a Python script or a generative AI model (e.g., GPT-4) based on the received subject and scope information. Specifically, it inputs a prompt to the AI ​​model in the form of "Generate the following problem: multiplication, two-digit x two-digit," and generates an appropriate problem (e.g., "28 x 36"). The output problem data is used in the next processing step.

[1672] Step 4:

[1673] The terminal displays the problem.

[1674] The problem data received from the server is displayed to the user using a user interface built with HTML and JavaScript. Specifically, the question "What is 28 x 36?" is displayed on the terminal screen. This display operation allows the user to visually recognize the problem.

[1675] Step 5:

[1676] The user enters and submits the answer.

[1677] The user enters the answer to the displayed question in the input field and clicks the "Submit" button. The entered answer (e.g., "1008") is again sent to the server as an HTTP request.

[1678] Step 6:

[1679] The server receives the answer and determines whether it is correct.

[1680] The server receives the answer data sent from the terminal and compares it with the correct answer data (e.g., "1008") set in advance. The server determines whether the answer is correct based on the comparison result, generates a correctness judgment result (e.g., "That's correct!"), and outputs it.

[1681] Step 7:

[1682] The device will display the result of the correct answer.

[1683] The result of the correct answer received from the server is displayed to the user. JavaScript is used to display feedback on the device screen such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[1684] Step 8:

[1685] The server stores and analyzes the answer history.

[1686] The server stores the user's answer data and the results of correct / incorrect assessment in a database. The stored data is periodically analyzed and the learner's weaknesses are identified using data analysis tools such as Python's Pandas and NumPy.

[1687] Step 9:

[1688] The server generates the original questions.

[1689] The server generates new problems using a generative AI model to improve weaknesses identified based on the analysis results. For example, it inputs a prompt such as "Generate the following problem: multiplication, two-digit x two-digit," and outputs an original problem such as "34 x 47."

[1690] Step 10:

[1691] The terminal displays the original question and the user answers it again.

[1692] The terminal receives new questions from the server and displays them to the user. The user enters answers to the new questions and submits them again. By repeating this process, the user can efficiently improve their academic ability.

[1693] (Application example 1)

[1694] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1695] While conventional learning support systems contribute to learners' knowledge acquisition, they have limitations in terms of improving practical skills and gaining practical experience in a virtual environment. In particular, there is a lack of effective ways to learn skills required for actual work, such as customer service and sales skills. Furthermore, it is difficult to provide real-time feedback or individual support based on learners' weaknesses.

[1696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1697] In this invention, the server includes means for accepting a selection of a subject and a range from a learner, means for generating questions based on the subject and range, means for providing the generated questions to the learner, means for receiving answers from the learner, means for determining whether the received answers are correct or incorrect, means for providing the determination result to the learner, means for analyzing the answer history to identify the learner's weaknesses, means for generating original questions based on the identified weaknesses, means for providing the learner with the generated original questions, means for providing the learner with practical skills in a virtual environment, and means for providing the learner with appropriate feedback in real time based on the received answers. This enables the learner to effectively learn skills necessary for actual work in a virtual environment.

[1698] A "learner" is an individual who aims to improve their knowledge and skills by using a learning support system.

[1699] "Subject" refers to the type or field of learning content, including mathematics, science, social studies, etc.

[1700] "Scope" refers to the specific area of ​​study within the selected subject, including specific chapters and topics.

[1701] "Problems" refer to questions or tasks given to assess learners' knowledge and skills.

[1702] "Correctness assessment" is the process of determining whether the answer provided by the learner is correct.

[1703] "Feedback" refers to the evaluation and advice given to a learner in response to their answers.

[1704] An "answer history" is a record of all answers a learner has ever given.

[1705] "Weaknesses" refer to areas where a learner has weaknesses or lacks understanding, identified by analyzing the answer history.

[1706] "Original questions" refer to questions that are specially created to address a learner's weaknesses.

[1707] A "virtual environment" refers to a simulated learning or experiential space created using computers and digital technology.

[1708] "Practical skills" refers to the practical abilities and know-how required in the job.

[1709] "Real-time" refers to immediate response to the information and actions input by the learner.

[1710] A learning support system for specifically implementing this invention comprises three components: a server, a terminal, and a user.

[1711] Server part

[1712] The server implements the following functions:

[1713] 1. Accepting learning requests

[1714] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[1715] 2. Problem Generation

[1716] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1717] 3. Receiving answers and determining whether they are correct or incorrect

[1718] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1719] 4. Saving and analyzing answer history

[1720] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[1721] 5. Generating original questions

[1722] Original questions are generated to reinforce weak points identified from the analysis results.

[1723] 6. Customize your study plan

[1724] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1725] 7. Providing practical skills in a virtual environment

[1726] Provide simulations that allow learners to gain hands-on experience and learn the skills they need to succeed in a virtual environment.

[1727] 8. Real-time feedback

[1728] Provide appropriate feedback in real time based on the answers provided by the learner.

[1729] Terminal part

[1730] The device implements the following features:

[1731] 1. Submit a learning request

[1732] Learners select the subject and scope through their terminal and send that information to the server.

[1733] 2. Displaying the problem

[1734] The terminal displays the questions received from the server to the learner.

[1735] 3. Enter and submit your answers

[1736] The learner enters the answer and sends it to the server via the terminal.

[1737] 4. Viewing Feedback

[1738] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1739] 5. Running a Virtual Environment

[1740] The devices use smart glasses or head-mounted displays to display virtual environments, allowing learners to learn practical skills.

[1741] User Part

[1742] The user (learner) performs the following operations:

[1743] 1. Select a learning request

[1744] Select the subject and area you want to study and make a request on your device.

[1745] 2. Answers to the questions

[1746] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1747] 3. Review feedback

[1748] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1749] 4. Practical skills training in a virtual environment

[1750] In a virtual environment, practical skills can be learned through simulations, such as practicing customer service and sales skills in a virtual store.

[1751] Hardware and software used

[1752] The server runs programs using languages ​​and frameworks such as Python and Django.

[1753] The terminal provides applications that run on a variety of devices, including smartphones, tablets, smart glasses, and head-mounted displays.

[1754] The database uses a relational database management system such as MySQL or PostgreSQL.

[1755] The generative AI model uses machine learning and natural language processing frameworks (such as TensorFlow and PyTorch) to generate questions and provide feedback.

[1756] Specific examples

[1757] If you're learning customer service skills in a virtual environment:

[1758] 1. User makes a learning request

[1759] The user selects "customer service skills" on the terminal and sends a learning request.

[1760] 2. Generate a problem on the server

[1761] The server generates questions such as "If a customer asks you a question about a product, how would you respond?" and sends them to the terminal.

[1762] 3. The questions are displayed on the device and the user answers them.

[1763] The terminal displays the virtual environment and initiates a dialogue with the virtual customer, who inputs answers and sends them to the server.

[1764] 4. The server receives the answer, determines whether it is correct, and provides feedback.

[1765] The server analyzes the answer and provides real-time feedback such as, "The answer given to the customer was appropriate. Here's what would be best next time."

[1766] Prompt Sentence Examples

[1767] Use the following information to generate questions to teach customer service skills in a virtual store:

[1768] Categories: Product knowledge, complaint handling, sales skills

[1769] Example: Describe the features of this product.

[1770] In this way, learners can effectively learn practical skills in a virtual environment.

[1771] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1772] Step 1:

[1773] The terminal accepts learning requests from users. The user selects the subject and scope of the learning they wish to do through the terminal interface and sends the request. The input is the subject and scope information, and the output is the request data sent to the server.

[1774] Step 2:

[1775] The server receives request data sent from the terminal. The input is the learning request data, and the output is a trigger to invoke the problem generation algorithm based on the request data.

[1776] Step 3:

[1777] The server generates questions based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history. The input is the subject and scope information and past answer history data, and the output is the generated question data.

[1778] Step 4:

[1779] The server sends the generated questions to the terminal. The input is the generated question data, and the output is the question data sent to the terminal.

[1780] Step 5:

[1781] The terminal displays the questions sent from the server to the learner. The input is the question data, and the output is the question displayed on the terminal interface.

[1782] Step 6:

[1783] The user answers the questions displayed on the terminal. The user enters the answer into the terminal and presses the send button. The input is the user's answer, and the output is the answer data sent to the server.

[1784] Step 7:

[1785] The server receives the answer from the user and judges whether the answer is correct or not. The input is the user's answer data, and the output is the result of the judgment.

[1786] Step 8:

[1787] The server sends the result of the accuracy judgment to the terminal. The input is the accuracy judgment result, and the output is feedback data sent to the terminal.

[1788] Step 9:

[1789] The terminal displays the result of the judgment and explanation from the server to the user. The input is the feedback data, and the output is the feedback displayed on the terminal interface.

[1790] Step 10:

[1791] The server stores the user's answer history in a database. The input is the answer data and the result of the correct / incorrect judgment, and the output is the answer history data stored in the database.

[1792] Step 11:

[1793] The server periodically analyzes the answer history and identifies the user's weaknesses. The input is the answer history data, and the output is the analysis results and the identified weaknesses.

[1794] Step 12:

[1795] The server generates original questions based on the identified weaknesses. The input is the weakness analysis result, and the output is the generated original question data.

[1796] Step 13:

[1797] The server sends the generated original question to the terminal. The input is the original question data, and the output is the original question data sent to the terminal.

[1798] Step 14:

[1799] The terminal displays the original problem to the user. The input is the original problem data, and the output is the original problem displayed on the interface.

[1800] Step 15:

[1801] The terminal runs the virtual environment and allows users to experience practical skills. The input is the data of the virtual environment simulation, and the output is the virtual environment that the user can view and manipulate.

[1802] Step 16:

[1803] The server provides appropriate feedback in real time based on the answers provided by the user. The input is real-time answer data, and the output is real-time feedback data.

[1804] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1805] A learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine.

[1806] System Configuration

[1807] Server part

[1808] The server implements the following functions:

[1809] 1. Acceptance of subject and scope selection

[1810] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[1811] 2. Problem Generation

[1812] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1813] 3. Receiving answers and determining whether they are correct or incorrect

[1814] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1815] 4. Saving and analyzing answer history

[1816] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[1817] 5. Generating original questions

[1818] Original questions are generated to reinforce weak points identified from the analysis results.

[1819] 6. Customize your study plan

[1820] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1821] 7. Emotion Data Analysis

[1822] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[1823] Terminal part

[1824] The device implements the following features:

[1825] 1. Submit a learning request

[1826] Learners select the subject and scope through their terminal and send that information to the server.

[1827] 2. Displaying the problem

[1828] The terminal displays the questions received from the server to the learner.

[1829] 3. Enter and submit your answers

[1830] The learner enters the answer and sends it to the server via the terminal.

[1831] 4. Viewing Feedback

[1832] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1833] 5. Collecting Emotional Data

[1834] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[1835] User Part

[1836] The user does the following:

[1837] 1. Select a learning request

[1838] Select the subject and area you want to study and make a request on your device.

[1839] 2. Answers to the questions

[1840] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1841] 3. Review feedback

[1842] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1843] 4. Providing Emotion Data

[1844] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[1845] Emotion engine part

[1846] The emotion engine implements the following functions:

[1847] 1. Emotion recognition and collection

[1848] Recognize and collect emotions from learners' facial expressions and voice data.

[1849] 2. Transmission of emotional information

[1850] The recognized emotion information is sent to the server.

[1851] 3. Emotion-Based Adjustment

[1852] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[1853] Specific examples

[1854] Learning multiplication in arithmetic using an emotion engine

[1855] 1. User makes a learning request

[1856] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[1857] 2. Generate a problem on the server

[1858] The server generates a problem such as "28 x 36" and sends it to the terminal.

[1859] 3. Display the problem on the device and collect sentiment data

[1860] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[1861] 4. The user enters the answer

[1862] The user enters the answer "1008".

[1863] 5. The server receives the answer and analyzes the emotion data.

[1864] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[1865] 6. Feedback and difficulty adjustments

[1866] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[1867] 7. View feedback on your device

[1868] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[1869] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[1870] The processing flow will be explained below.

[1871] Step 1:

[1872] The user starts the learning application on the device and selects the subject and area they want to study. Let's assume that they select "Multiplication" in "Math."

[1873] Step 2:

[1874] The terminal transmits information about the subject and range selected by the user to the server.

[1875] Step 3:

[1876] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36."

[1877] Step 4:

[1878] The server sends the generated questions to the terminal.

[1879] Step 5:

[1880] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[1881] Step 6:

[1882] At the same time as displaying the image, the device sends the user's facial expressions and voice data to the emotion engine.

[1883] Step 7:

[1884] The emotion engine recognizes and analyzes the user's emotions from the facial and voice data it receives. For example, if the user has a troubled expression, it will determine that they are feeling stressed.

[1885] Step 8:

[1886] The user answers the displayed question and enters the answer into the terminal. Let's consider the case where the user enters "1008."

[1887] Step 9:

[1888] The device sends the user's answer to the server and also sends the emotion engine's analysis results to the server.

[1889] Step 10:

[1890] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[1891] Step 11:

[1892] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[1893] Step 12:

[1894] The server stores the answer history in a database and analyzes it, identifying the learner's weaknesses based on the analysis results.

[1895] Step 13:

[1896] The server adjusts the next learning step based on the analysis results and data from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, it adjusts the difficulty of the questions.

[1897] Step 14:

[1898] The server generates new original questions and sends the information to the terminal. It generates new questions such as "What is 34 x 47?"

[1899] Step 15:

[1900] The terminal displays the original questions received from the server to the user.

[1901] Step 16:

[1902] The device again sends the user's facial expressions and voice data to the emotion engine.

[1903] Step 17:

[1904] The emotion engine recognizes and analyzes emotions from new facial and voice data.

[1905] Step 18:

[1906] The user answers the original question displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[1907] Step 19:

[1908] The device sends the user's answer to the server, as well as the emotion engine's new analysis results.

[1909] Step 20:

[1910] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[1911] Step 21:

[1912] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[1913] Step 22:

[1914] The device displays the feedback received from the server to the user, including encouragement and advice based on the user's emotions.

[1915] Step 23:

[1916] By repeating this series of processes, users can reinforce their weaknesses while progressing with optimal learning based on emotional data.

[1917] Example 2

[1918] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1919] Conventional learning support systems generate questions based on the learner's academic level and reinforce weak points by analyzing the learner's answer history, but they lack the ability to individually customize the system to take into account the learner's emotional state. As a result, learners' stress and decreased motivation can affect their learning outcomes. Therefore, the present invention aims to provide a more effective and personalized learning support system that takes into account the learner's emotional state.

[1920] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1921] In this invention, the server includes means for receiving selection of a subject and range from the learner, means for generating questions based on the subject and range, and means for analyzing collected emotional data to adjust the difficulty of the study plan and questions, thereby making it possible to provide a study plan that takes into account the emotional state of the learner.

[1922] "Learner" refers to a person who uses the system to acquire knowledge or skills.

[1923] A "subject" refers to a particular field or topic that a learner will study.

[1924] "Scope" refers to the part of the subject that the learner particularly wants to study.

[1925] "Problems" are questions or tasks provided to assess a learner's understanding or skills.

[1926] "Answer" refers to the answer to the question submitted by the learner.

[1927] "Correct or incorrect" refers to the result of determining whether the learner's answer is correct or incorrect.

[1928] "Answer history" refers to a record of answers previously provided by a learner.

[1929] "Weak points" refer to areas in which a learner is particularly weak, as identified by analyzing the answer history.

[1930] "Original questions" refer to questions that are specially created to reinforce a learner's weak points.

[1931] "Emotional data" refers to information about emotions collected from learners' facial expressions, voice, etc.

[1932] "Emotion engine" refers to technology for analyzing collected emotional data and providing the results to the system.

[1933] "Study plan" refers to the learning progress plan and content provided to learners.

[1934] "Database" refers to an information management system for storing learners' answer history and other information.

[1935] The learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine. The functions and specific processes of each component will be explained in detail below.

[1936] System Configuration

[1937] Server part

[1938] The server has the following functions:

[1939] 1. Acceptance of subject and scope selection

[1940] The server receives the learning request sent from the device and identifies the specific subject and area, and stores this information in a database.

[1941] Example: When a request for "arithmetic" and "multiplication" is sent from the terminal, the server receives it.

[1942] 2. Problem Generation

[1943] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[1944] Example: Generate questions of appropriate difficulty, such as "15 x 12," based on the learner's academic level.

[1945] 3. Receiving answers and determining whether they are correct or incorrect

[1946] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[1947] Example: A learner sends the answer "180" to the server, and the server determines this to be the "correct answer."

[1948] 4. Saving and analyzing answer history

[1949] The server stores the answer history in a database and periodically analyzes it to identify the learner's weak points.

[1950] Example: Save a history of frequent mistakes a learner makes on multiplication problems and analyze the data.

[1951] 5. Generating original questions

[1952] Original questions are generated to reinforce weak points identified from the analysis results.

[1953] For example, if a weakness in "multiplication" is identified, generate and provide similar multiplication problems.

[1954] 6. Customize your study plan

[1955] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[1956] Example: Propose a lesson plan aimed at strengthening multiplication skills.

[1957] 7. Emotion Data Analysis

[1958] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[1959] For example: temporarily reducing the difficulty of questions when a learner is feeling stressed.

[1960] Terminal part

[1961] The device has the following features:

[1962] 1. Submit a learning request

[1963] Learners select the subject and scope through their terminal and send that information to the server.

[1964] Example: If a learner selects "arithmetic" and "multiplication", the selection is sent to the server.

[1965] 2. Displaying the problem

[1966] The terminal displays the questions received from the server to the learner.

[1967] For example, display the question as "What is 15 x 12?"

[1968] 3. Enter and submit your answers

[1969] The learner enters the answer and sends it to the server via the terminal.

[1970] Example: Enter the answer "180" and submit.

[1971] 4. Viewing Feedback

[1972] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[1973] Example: Display "You got it right! Congratulations!"

[1974] 5. Collecting Emotional Data

[1975] The device collects the learner's facial expressions and voice data and sends it to the emotion engine.

[1976] Example: Recognizing a user's smile through a camera and sending that data to an emotion engine.

[1977] User Part

[1978] The user performs the following operations:

[1979] 1. Select a learning request

[1980] Select the subject and area you want to study and make a request on your device.

[1981] Example: User selects "Math" and "Multiplication".

[1982] 2. Answers to the questions

[1983] Answer the questions displayed on the terminal and enter the answers into the terminal.

[1984] For example, enter "15 × 12 = 180".

[1985] 3. Review feedback

[1986] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[1987] For example: "You got it right! Congratulations!"

[1988] 4. Providing Emotion Data

[1989] Emotions during training are collected by the device, and facial and voice data are analyzed by the emotion engine.

[1990] For example, when a learner smiles while solving a problem, their facial expression data is analyzed.

[1991] Emotion engine part

[1992] The emotion engine has the following functions:

[1993] 1. Emotion recognition and collection

[1994] Recognize and collect emotions from learners' facial expressions and voice data.

[1995] Example: Recognizing "joy" from a learner's smiling face.

[1996] 2. Transmission of emotional information

[1997] The recognized emotion information is sent to the server.

[1998] Example: Sending learner "joy" data to the server.

[1999] 3. Emotion-Based Adjustment

[2000] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[2001] For example: If a learner is feeling stressed, provide them with easy problems.

[2002] Specific examples

[2003] Example of learning multiplication using emotion engine

[2004] 1. User makes a learning request

[2005] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[2006] 2. Generate a problem on the server

[2007] The server generates a problem such as "28 x 36" and sends it to the terminal.

[2008] 3. Display the problem on the device and collect sentiment data

[2009] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[2010] 4. The user enters the answer

[2011] The user enters the answer "1008".

[2012] 5. The server receives the answer and analyzes the emotion data.

[2013] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[2014] 6. Feedback and difficulty adjustments

[2015] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[2016] 7. View feedback on your device

[2017] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[2018] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[2019] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2020] Program processing steps and detailed explanations

[2021] Step 1:

[2022] Submit a learning request

[2023] Terminal processing: An interface is displayed that allows the user to select the subject and range of study they wish to do via the terminal. The user selects "arithmetic" and "multiplication" and sends the request.

[2024] Input: User selection information for the subject "arithmetic" and range "multiplication".

[2025] Output: The training request sent to the server.

[2026] Specific operation: The terminal sends the selected subject and range to the server as an HTTP request.

[2027] Step 2:

[2028] Processing subject and scope information

[2029] Server processing: The server analyzes the received learning request, stores it in a database, and then performs preprocessing to generate appropriate questions.

[2030] Input: The training request sent from the device.

[2031] Output: Subject and scope information stored in database.

[2032] Specific operation: The server receives the HTTP request and records it in the database.

[2033] Step 3:

[2034] Problem Generation

[2035] Server processing: The server runs a question generation algorithm based on the subject and scope information stored in the database, taking into account the learner's past answer history and academic level.

[2036] Input: subject and scope information, past answer history, academic level information.

[2037] Output: The generated questions.

[2038] Specific operation: The server retrieves the necessary information from the database and generates a problem such as "15 x 12".

[2039] Step 4:

[2040] Viewing Issues

[2041] Terminal processing: The generated questions sent from the server are displayed to the user.

[2042] Input: The generated question sent by the server.

[2043] Output: The problem as it appears on the user's screen.

[2044] Specific operation: The terminal receives the response from the server and displays the question "What is 15 x 12?"

[2045] Step 5:

[2046] Enter your answer

[2047] User operation: The user inputs an answer to the displayed question. For example, the answer is "180."

[2048] Input: The user's answer.

[2049] Output: The answer typed into the terminal.

[2050] Specific action: The user enters "180" in the input field of the terminal.

[2051] Step 6:

[2052] Submit your answer

[2053] Terminal processing: The terminal sends the user's answer to the server.

[2054] Input: The user's answer.

[2055] Output: The answer sent to the server.

[2056] Specific operation: The terminal sends the entered answer "180" to the server via an HTTP request.

[2057] Step 7:

[2058] Judging whether the answer is correct or incorrect

[2059] Server processing: The server compares the received answer with the correct answer set in advance and determines whether it is correct or incorrect.

[2060] Input: User's answer "180".

[2061] Output: Correct / incorrect result.

[2062] Specific operation: The server compares the answer "180" with the correct answer "180" and determines that it is correct.

[2063] Step 8:

[2064] Save answer history

[2065] Server processing: The answer results are saved in a database and recorded as a history for future analysis.

[2066] Input: User's answer and verdict.

[2067] Output: Answer history stored in a database.

[2068] Specific operation: The server stores the answers and judgment results in a database.

[2069] Step 9:

[2070] Collecting Emotional Data

[2071] Device processing: The device collects the learner's facial expressions and voice data and sends them to the emotion engine.

[2072] Input: Learner's facial expressions and voice data.

[2073] Output: Emotion data sent to the emotion engine.

[2074] Specific operation: The device collects the user's emotional data through the camera and microphone and sends it to the emotion engine.

[2075] Step 10:

[2076] Emotional Data Analysis

[2077] Emotion Engine Processing: The emotion engine analyzes the received data and determines the learner's current emotional state.

[2078] Input: Emotion data sent from the device.

[2079] Output: Learner's emotional state.

[2080] Specific operation: The emotion engine uses a facial expression recognition algorithm to analyze the user's smile and stress state.

[2081] Step 11:

[2082] Generate customized feedback

[2083] Server processing: Based on answer history and emotion data, appropriate feedback and the next learning plan are generated.

[2084] Input: Answer history and emotion data.

[2085] Output: Customized feedback and study plan.

[2086] Specific operation: The server takes into account the learner's situation and generates feedback such as "That's correct! The next question will be easier."

[2087] Step 12:

[2088] View Feedback

[2089] Terminal processing: Display the feedback sent from the server to the user.

[2090] Input: Customized feedback.

[2091] Output: Feedback displayed on the user's screen.

[2092] Specific operation: The device displays "Correct! The next question will be easy."

[2093] By following these processing steps, personalized learning support can be provided to the learner.

[2094] (Application example 2)

[2095] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2096] Conventional learning support systems are unable to take into account learners' emotions or real-time reactions, and because learning plans and questions are provided uniformly, effective personalization is difficult. Furthermore, it is difficult to utilize customer emotional data in brick-and-mortar stores, resulting in a lack of services and proposals that meet customer needs. A system that can solve these problems and enable more effective and personalized learning and customer service is needed.

[2097] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a subject and range selection from a learner; means for generating questions based on the subject and range; means for providing the generated questions to the learner; means for receiving answers from the learner; means for determining whether the received answers are correct or incorrect; means for providing the determination results to the learner; means for analyzing the answer history to identify the learner's weaknesses; means for generating original questions based on the identified weaknesses; means for providing the generated original questions to the learner; means for collecting emotional data; means for analyzing the collected emotional data; means for providing a learning plan or feedback corresponding to the learner's emotions based on the analysis results; and means for analyzing customer emotional data in real time at a physical store and displaying information and suggestions suitable for the customer. This enables learning support optimized for individual learners and further enables services to be provided in physical stores that meet customer needs.

[2098] "Learner" refers to an individual who is learning educational content and is a user who intends to acquire knowledge and skills using this system.

[2099] "Means for accepting selection of subject and scope" is a function that provides an interface for a learner to specify a particular subject and scope that they wish to study.

[2100] The "means for generating questions" is a function that automatically creates questions that are relevant to the content based on the subject and scope selected by the learner.

[2101] The "means for providing the generated questions to the learner" is a function for displaying the questions generated by the server on the learner's terminal.

[2102] The "means for receiving answers" is a function for sending answers entered by learners to the server.

[2103] The "means for determining whether the answer is correct or incorrect" is a function that compares the learner's answer with pre-set correct answer data and determines whether the answer is correct or incorrect.

[2104] "Means for identifying learners' weaknesses by analyzing answer history" is a function that analyzes a learner's past answer data and identifies areas and question types in which the learner is weak.

[2105] The "means for generating original questions" is a function for creating new questions specifically for strengthening the weak points of identified learners.

[2106] "Means for collecting emotional data" is a function for obtaining emotional data from the facial expressions, voices, etc. of learners and customers.

[2107] "Means for analyzing emotional data" is a function for analyzing the emotional state of learners and customers based on collected emotional data.

[2108] "Means for providing learning plans and feedback" refers to a function that provides learning plans and feedback that correspond to the learner's emotions based on the analysis results.

[2109] "A means of analyzing customer emotional data in real time in physical stores and displaying information and suggestions appropriate to the customer" is a function that instantly analyzes customer emotions in stores and suggests the most suitable products and services to the customer based on the results.

[2110] MODE FOR CARRYING OUT THE INVENTION

[2111] The learning support system and customer service support system for brick-and-mortar stores of the present invention are composed of four components: a server, a terminal, a user, and an emotion engine.

[2112] System Configuration

[2113] Server part

[2114] The server implements the following functions:

[2115] 1. Acceptance of subject and scope selection

[2116] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[2117] 2. Problem Generation

[2118] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[2119] 3. Receiving answers and determining whether they are correct or incorrect

[2120] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[2121] 4. Saving and analyzing answer history

[2122] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[2123] 5. Generating original questions

[2124] Original questions are generated to reinforce weak points identified from the analysis results.

[2125] 6. Customize your study plan

[2126] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[2127] 7. Emotion Data Analysis

[2128] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[2129] 8. Analysis and Proposal of Emotional Data in Physical Stores

[2130] Customer sentiment data obtained in physical stores is analyzed in real time, and the results are used to provide optimal product suggestions and customer service methods to customers.

[2131] Terminal part

[2132] The device implements the following features:

[2133] 1. Submit a learning request

[2134] Learners select the subject and scope through their terminal and send that information to the server.

[2135] 2. Displaying the problem

[2136] The terminal displays the questions received from the server to the learner.

[2137] 3. Enter and submit your answers

[2138] The learner enters the answer and sends it to the server via the terminal.

[2139] 4. Viewing Feedback

[2140] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[2141] 5. Collecting Emotional Data

[2142] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[2143] User Part

[2144] The user does the following:

[2145] 1. Select a learning request

[2146] Select the subject and area you want to study and make a request on your device.

[2147] 2. Answers to the questions

[2148] Answer the questions displayed on the terminal and enter the answers into the terminal.

[2149] 3. Review feedback

[2150] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[2151] 4. Providing Emotion Data

[2152] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[2153] Emotion engine part

[2154] The emotion engine implements the following functions:

[2155] 1. Emotion recognition and collection

[2156] Recognize and collect emotions from learners' and customers' facial expressions and voice data.

[2157] 2. Transmission of emotional information

[2158] The recognized emotion information is sent to the server.

[2159] 3. Emotion-Based Adjustment

[2160] Based on the emotional information provided to the server, the system will assess the learner's stress level and concentration level, and adjust the difficulty of the questions and the content of the feedback. In addition, in the case of a physical store, the system will provide appropriate information and product suggestions based on the customer's emotions.

[2161] Specific examples

[2162] Learning multiplication in arithmetic using an emotion engine

[2163] 1. User makes a learning request

[2164] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[2165] 2. Generate a problem on the server

[2166] The server generates a problem such as "28 x 36" and sends it to the terminal.

[2167] 3. Display the problem on the device and collect sentiment data

[2168] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[2169] 4. The user enters the answer

[2170] The user enters the answer "1008".

[2171] 5. The server receives the answer and analyzes the emotion data.

[2172] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[2173] 6. Feedback and difficulty adjustments

[2174] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[2175] 7. View feedback on your device

[2176] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[2177] Emotion analysis and customer support in brick-and-mortar stores

[2178] 1. The customer is choosing a product

[2179] Smart glasses in physical stores detect customers' facial expressions and voices.

[2180] 2. The emotion engine analyzes the emotion data

[2181] If the customer is confused, the emotion engine sends this data to the server.

[2182] 3. The server displays the proposal

[2183] Based on the analysis results, the server displays appropriate product suggestions and explanations on the smart glasses.

[2184] 4. Staff make suggestions

[2185] Staff will then make optimal product recommendations to customers based on the displayed information.

[2186] Example prompt for a generative AI model:

[2187] "If a customer is interested in your product but seems confused, what explanation or suggestion would be most appropriate? For example, explaining new features or showing examples of use."

[2188] This allows for effective and personalized assistance that takes into account the emotions of learners and customers.

[2189] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2190] Step 1:

[2191] The learner selects the subject and range on the terminal and sends a learning request to the server. The input is the subject and range selected by the learner. The output is the learning request data sent to the server. Here, the terminal provides the interface and the learner makes the selection.

[2192] Step 2:

[2193] Based on the learning request data received by the server, questions are generated for the relevant subject and scope. The input is the learning request data, and the output is the generated question data. To do this, the server retrieves relevant information from a database and creates appropriate questions using an algorithm.

[2194] Step 3:

[2195] The server sends the generated questions to the terminal, which then displays them to the learner. The input is the generated question data, and the output is the question displayed on the terminal. The terminal displays the question using a display interface.

[2196] Step 4:

[2197] The learner inputs the answer through the terminal and sends it to the server. The input is the answer entered by the learner, and the output is the answer data sent to the server. The terminal provides the answer interface, and the learner inputs the answer.

[2198] Step 5:

[2199] The server compares the received answer with the correct answer data to determine whether it is correct or not. The input is the learner's answer data and the preset correct answer data, and the output is the result of the correct or incorrect decision. The server uses logic to perform the comparison and calculate the result.

[2200] Step 6:

[2201] The server sends the judgment result to the terminal, which displays it to the learner. The input is the correct / incorrect judgment result, and the output is the feedback displayed on the terminal. The terminal provides a feedback screen and displays the result.

[2202] Step 7:

[2203] The device collects the learner's facial and voice data and sends it to the emotion engine. The input is facial and voice data, and the output is emotion data that is sent to the emotion engine. The device captures data using sensors.

[2204] Step 8:

[2205] The emotion engine analyzes the received emotion data and sends the analysis results to the server. The input is emotion data and the output is the analysis results. The emotion engine analyzes the data using emotion recognition algorithms.

[2206] Step 9:

[2207] The server adjusts the difficulty of questions and feedback based on the results of sentiment analysis. The input is the analysis results, and the output is the adjusted questions and feedback data. The server then processes the learning plan to dynamically customize it.

[2208] Step 10:

[2209] The server sends the adjusted questions and feedback to the device, which then displays them to the learner. The input is the adjusted data from the server, and the output is the adjusted content displayed on the device. The device again updates the interface and displays the information.

[2210] Step 11:

[2211] In a physical store, smart glasses collect customer facial and voice data and send it to an emotion engine. The input is the customer's facial and voice data, and the output is emotion data sent to the emotion engine. The smart glasses act as a collection device.

[2212] Step 12:

[2213] The emotion engine analyzes customer emotion data in real time and sends the analysis results to the server. The input is emotion data, and the output is the real-time analysis results. The emotion engine quickly performs data calculations.

[2214] Step 13:

[2215] The server sends and displays appropriate product and service suggestions based on the analysis results to the smart glasses. The input is the analysis results and suggestion data, and the output is the suggestions displayed on the smart glasses. The server generates and sends the suggestions, which are then displayed by the smart glasses.

[2216] This will enable personalized learning and customer support that takes into account real-time emotional data from learners and customers.

[2217] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2218] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2219] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2220] [Fourth embodiment]

[2221] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2222] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2223] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2224] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2225] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2226] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2227] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2228] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2229] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2230] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2231] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2232] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2233] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2234] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[2235] System Configuration

[2236] Server part

[2237] The server implements the following functions:

[2238] 1. Acceptance of subject and scope selection

[2239] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[2240] 2. Problem Generation

[2241] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[2242] 3. Receiving answers and determining whether they are correct or incorrect

[2243] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[2244] 4. Saving and analyzing answer history

[2245] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[2246] 5. Generating original questions

[2247] Original questions are generated to reinforce weak points identified from the analysis results.

[2248] 6. Customize your study plan

[2249] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[2250] Terminal part

[2251] The device implements the following features:

[2252] 1. Submit a learning request

[2253] Learners select the subject and scope through their terminal and send that information to the server.

[2254] 2. Displaying the problem

[2255] The terminal displays the questions received from the server to the learner.

[2256] 3. Enter and submit your answers

[2257] The learner enters the answer and sends it to the server via the terminal.

[2258] 4. Viewing Feedback

[2259] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[2260] User Part

[2261] The user does the following:

[2262] 1. Select a learning request

[2263] Select the subject and area you want to study and make a request on your device.

[2264] 2. Answers to the questions

[2265] Answer the questions displayed on the terminal and enter the answers into the terminal.

[2266] 3. Review feedback

[2267] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[2268] Specific examples

[2269] When learning multiplication in arithmetic

[2270] 1. User makes a learning request

[2271] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[2272] 2. Generate a problem on the server

[2273] The server generates a problem such as "28 x 36" and sends it to the terminal.

[2274] 3. The questions are displayed on the device and the user answers them.

[2275] The terminal displays "What is 28 x 36?" and the user answers "1008."

[2276] 4. The server receives the answer and determines whether it is correct or not.

[2277] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[2278] 5. The server saves and analyzes the answer history

[2279] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[2280] 6. The original questions are displayed on the device, and the user answers them again.

[2281] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[2282] In this way, learners can effectively improve their learning ability through repeated practice.

[2283] The processing flow will be explained below.

[2284] Step 1:

[2285] A user starts a learning application using a device and selects the subject and area they want to study. Let's consider the case where they select "Multiplication" in "Math."

[2286] Step 2:

[2287] The terminal transmits information about the subject and range selected by the user to the server.

[2288] Step 3:

[2289] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36".

[2290] Step 4:

[2291] The server sends the generated questions to the terminal.

[2292] Step 5:

[2293] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[2294] Step 6:

[2295] The user solves the displayed problem and enters the answer into the terminal. Let's consider the case where the user enters "1008".

[2296] Step 7:

[2297] The terminal transmits the user's answer to the server.

[2298] Step 8:

[2299] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[2300] Step 9:

[2301] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[2302] Step 10:

[2303] The terminal displays the feedback received from the server to the user.

[2304] Step 11:

[2305] The server stores the user's answer history in a database and analyzes it, particularly to determine which types of questions the user is weak at.

[2306] Step 12:

[2307] The server identifies the user's weaknesses based on the analysis results and generates original questions that focus on those weaknesses, such as "34 x 47."

[2308] Step 13:

[2309] The server sends the generated original questions to the terminal.

[2310] Step 14:

[2311] The terminal displays the original question received from the server to the user: "What is 34 x 47?"

[2312] Step 15:

[2313] The user solves the original problem displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[2314] Step 16:

[2315] The terminal transmits the user's answer to the server.

[2316] Step 17:

[2317] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[2318] Step 18:

[2319] The server sends the result of the accuracy judgment to the terminal, and sends feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[2320] Step 19:

[2321] The terminal displays the feedback received from the server to the user.

[2322] Step 20:

[2323] Users can check the feedback, understand their weaknesses, and continue their studies. By repeating this process, users can progress through their studies efficiently.

[2324] Example 1

[2325] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2326] Conventional learning systems have difficulty providing appropriate questions that match the individual academic ability of each learner, and lack the ability to fully utilize answer history during the learning process to identify weak points and generate questions based on those weak points. As a result, learners have had difficulty improving their academic ability efficiently.

[2327] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2328] In this invention, the server includes means for accepting a selection of a subject and range from a learner, means for using a natural language processing algorithm as a means for generating questions based on the subject and range, means for displaying the generated questions to the learner, means for inputting and transmitting answers from the learner, means for judging the correctness of the received answers, means for providing the judgement result and explanations to the learner, means for saving the answer history in a database, means for analyzing the answer history to identify the learner's weaknesses, means for using a generative AI model as a means for generating original questions based on the identified weaknesses, and means for re-displaying the generated original questions to the learner. This allows learners to be provided with questions according to their individual academic level, enabling them to progress in their studies while efficiently overcoming their weaknesses.

[2329] A "student" is a user who uses the learning support system to study a subject.

[2330] "Subjects" are areas of study offered by the learning support system, such as mathematics, science, and English.

[2331] "Scope" is the specific part or item within the selected subject that the learner wants to learn.

[2332] The "means for accepting selection" is a mechanism for receiving input of the subject and scope from the learner and transmitting it to the server.

[2333] A "means for generating questions" is an algorithm or generative model for creating specific learning questions based on the subject and scope selected by the learner.

[2334] A "natural language processing algorithm" is a computer software process for analyzing and understanding linguistic data.

[2335] A "generative AI model" is a mathematical model that uses machine learning and artificial intelligence to generate specific problems.

[2336] The "means for displaying questions to the learner" refers to a web interface or application that displays the generated questions on the learner's terminal screen.

[2337] The "means for inputting and sending answers" is an interface that allows a learner to input answers to questions and send them to the server.

[2338] A "means for determining correctness" is an algorithm or program that evaluates whether a learner's answer is correct or incorrect.

[2339] The "means for providing the judgment result and explanation" is a mechanism for displaying to the learner the result of whether the answer is correct or incorrect and an explanation based on that result.

[2340] The "means for storing answer history in a database" is a database system that accumulates information on learners' past answers.

[2341] "Means for analyzing answer history" refers to the process of analyzing saved answer data to identify the learner's weaknesses, etc.

[2342] "Means for generating original questions" are algorithms or models that create new learning questions for specific learners based on the analysis results.

[2343] The "means for re-displaying" is an interface that re-displays the original question that was generated to the learner.

[2344] A learning support system for specifically implementing the present invention comprises three components: a server, a terminal, and a user.

[2345] Server part

[2346] The server has the following features:

[2347] 1. Acceptance of subject and scope selection

[2348] The server receives learning requests from the devices and identifies the specific subjects and areas of interest. This is done using web server software such as Apache or Nginx.

[2349] 2. Problem Generation

[2350] The server generates questions based on the requested subject and scope information using a Python script or a generative AI model (e.g., GPT-4).

[2351] 3. Receiving answers and determining whether they are correct or incorrect

[2352] The server receives the answers sent by the user and compares them with the pre-set correct answers to determine whether they are correct or not. This process uses a database such as MySQL.

[2353] 4. Saving and analyzing answer history

[2354] The server stores the answer history in a database and periodically analyzes it using data analysis tools such as Python's Pandas and NumPy.

[2355] 5. Generating original questions

[2356] Based on the analysis results, original questions are generated that address the identified weaknesses, again using a generative AI model.

[2357] 6. Customize your study plan

[2358] The server provides an individually customized learning plan based on the learner's answer history and weakness analysis results. In this process, the customized learning plan is generated in JSON format and sent to the device.

[2359] Terminal part

[2360] The terminal has the following features:

[2361] 1. Submit a learning request

[2362] The user selects the subject and scope through their device and sends that information to the server via a web browser or mobile app interface.

[2363] 2. Displaying the problem

[2364] The device receives the questions from the server and displays them to the user, visually displaying the questions using HTML, CSS, and JavaScript.

[2365] 3. Enter and submit your answers

[2366] The user enters the answer into the terminal and sends it to the server, via a form or input field.

[2367] 4. Viewing Feedback

[2368] The correct / incorrect result sent from the server is displayed to the user. Visual feedback is provided using JavaScript and CSS.

[2369] User Part

[2370] The user does the following:

[2371] 1. Select a learning request

[2372] The user selects the subject and area they wish to study and makes a request on their terminal.

[2373] 2. Answers to the questions

[2374] Answers to questions displayed on the terminal are entered and the answers are sent to the terminal.

[2375] 3. Review feedback

[2376] Check the feedback displayed on your device and learn the correct answers and explanations for questions you got wrong.

[2377] Specific examples

[2378] When learning multiplication in arithmetic

[2379] 1. User makes a learning request

[2380] The user selects arithmetic on the terminal and requests the "multiplication" range.

[2381] 2. Generate a problem on the server

[2382] The server generates a problem such as "28 x 36" and sends it to the terminal.

[2383] 3. The questions are displayed on the device and the user answers them.

[2384] The terminal displays "What is 28 x 36?" and the user enters the answer "1008."

[2385] 4. The server receives the answer and determines whether it is correct or not.

[2386] The server determines whether the answer "1008" is correct and sends feedback to the terminal saying "That's correct!" or "That's incorrect. The correct answer is 1008."

[2387] 5. The server saves and analyzes the answer history

[2388] The server stores the user's answer history in a database, analyzes it, and generates original questions based on the identified weaknesses.

[2389] 6. The original questions are displayed on the device, and the user answers them again.

[2390] The terminal displays a new question, "What is 34 x 47?" and the user enters the answer.

[2391] Prompt Sentence Examples

[2392] "Please enter the answer to the following question: 28 × 36 = ?"

[2393] This system allows users to study efficiently by providing questions appropriate to their academic level. The server, terminal, and user each play their respective roles effectively and work together to provide learning support.

[2394] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2395] Step 1:

[2396] A user submits a learning request.

[2397] The user selects the subject and area of ​​study they wish to study through their device and enters that request into the device's input form. The entered data (subject and area information) is sent to the server as an HTTP request.

[2398] Step 2:

[2399] The server identifies the subject and scope.

[2400] The server analyzes the HTTP request received from the device and extracts information about the subject and scope. Based on this information, it prepares to generate appropriate questions.

[2401] Step 3:

[2402] The server generates the problem.

[2403] The server creates questions using a Python script or a generative AI model (e.g., GPT-4) based on the received subject and scope information. Specifically, it inputs a prompt to the AI ​​model in the form of "Generate the following problem: multiplication, two-digit x two-digit," and generates an appropriate problem (e.g., "28 x 36"). The output problem data is used in the next processing step.

[2404] Step 4:

[2405] The terminal displays the problem.

[2406] The problem data received from the server is displayed to the user using a user interface built with HTML and JavaScript. Specifically, the question "What is 28 x 36?" is displayed on the terminal screen. This display operation allows the user to visually recognize the problem.

[2407] Step 5:

[2408] The user enters and submits the answer.

[2409] The user enters the answer to the displayed question in the input field and clicks the "Submit" button. The entered answer (e.g., "1008") is again sent to the server as an HTTP request.

[2410] Step 6:

[2411] The server receives the answer and determines whether it is correct.

[2412] The server receives the answer data sent from the terminal and compares it with the correct answer data (e.g., "1008") set in advance. The server determines whether the answer is correct based on the comparison result, generates a correctness judgment result (e.g., "That's correct!"), and outputs it.

[2413] Step 7:

[2414] The device will display the result of the correct answer.

[2415] The result of the correct answer received from the server is displayed to the user. JavaScript is used to display feedback on the device screen such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[2416] Step 8:

[2417] The server stores and analyzes the answer history.

[2418] The server stores the user's answer data and the results of correct / incorrect assessment in a database. The stored data is periodically analyzed and the learner's weaknesses are identified using data analysis tools such as Python's Pandas and NumPy.

[2419] Step 9:

[2420] The server generates the original questions.

[2421] The server generates new problems using a generative AI model to improve weaknesses identified based on the analysis results. For example, it inputs a prompt such as "Generate the following problem: multiplication, two-digit x two-digit," and outputs an original problem such as "34 x 47."

[2422] Step 10:

[2423] The terminal displays the original question and the user answers it again.

[2424] The terminal receives new questions from the server and displays them to the user. The user enters answers to the new questions and submits them again. By repeating this process, the user can efficiently improve their academic ability.

[2425] (Application example 1)

[2426] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2427] While conventional learning support systems contribute to learners' knowledge acquisition, they have limitations in terms of improving practical skills and gaining practical experience in a virtual environment. In particular, there is a lack of effective ways to learn skills required for actual work, such as customer service and sales skills. Furthermore, it is difficult to provide real-time feedback or individual support based on learners' weaknesses.

[2428] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2429] In this invention, the server includes means for accepting a selection of a subject and a range from a learner, means for generating questions based on the subject and range, means for providing the generated questions to the learner, means for receiving answers from the learner, means for determining whether the received answers are correct or incorrect, means for providing the determination result to the learner, means for analyzing the answer history to identify the learner's weaknesses, means for generating original questions based on the identified weaknesses, means for providing the learner with the generated original questions, means for providing the learner with practical skills in a virtual environment, and means for providing the learner with appropriate feedback in real time based on the received answers. This enables the learner to effectively learn skills necessary for actual work in a virtual environment.

[2430] A "learner" is an individual who aims to improve their knowledge and skills by using a learning support system.

[2431] "Subject" refers to the type or field of learning content, including mathematics, science, social studies, etc.

[2432] "Scope" refers to the specific area of ​​study within the selected subject, including specific chapters and topics.

[2433] "Problems" refer to questions or tasks given to assess learners' knowledge and skills.

[2434] "Correctness assessment" is the process of determining whether the answer provided by the learner is correct.

[2435] "Feedback" refers to the evaluation and advice given to a learner in response to their answers.

[2436] An "answer history" is a record of all answers a learner has ever given.

[2437] "Weaknesses" refer to areas where a learner has weaknesses or lacks understanding, identified by analyzing the answer history.

[2438] "Original questions" refer to questions that are specially created to address a learner's weaknesses.

[2439] A "virtual environment" refers to a simulated learning or experiential space created using computers and digital technology.

[2440] "Practical skills" refers to the practical abilities and know-how required in the job.

[2441] "Real-time" refers to immediate response to the information and actions input by the learner.

[2442] A learning support system for specifically implementing this invention comprises three components: a server, a terminal, and a user.

[2443] Server part

[2444] The server implements the following functions:

[2445] 1. Accepting learning requests

[2446] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[2447] 2. Problem Generation

[2448] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[2449] 3. Receiving answers and determining whether they are correct or incorrect

[2450] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[2451] 4. Saving and analyzing answer history

[2452] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[2453] 5. Generating original questions

[2454] Original questions are generated to reinforce weak points identified from the analysis results.

[2455] 6. Customize your study plan

[2456] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[2457] 7. Providing practical skills in a virtual environment

[2458] Provide simulations that allow learners to gain hands-on experience and learn the skills they need to succeed in a virtual environment.

[2459] 8. Real-time feedback

[2460] Provide appropriate feedback in real time based on the answers provided by the learner.

[2461] Terminal part

[2462] The device implements the following features:

[2463] 1. Submit a learning request

[2464] Learners select the subject and scope through their terminal and send that information to the server.

[2465] 2. Displaying the problem

[2466] The terminal displays the questions received from the server to the learner.

[2467] 3. Enter and submit your answers

[2468] The learner enters the answer and sends it to the server via the terminal.

[2469] 4. Viewing Feedback

[2470] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[2471] 5. Running a Virtual Environment

[2472] The devices use smart glasses or head-mounted displays to display virtual environments, allowing learners to learn practical skills.

[2473] User Part

[2474] The user (learner) performs the following operations:

[2475] 1. Select a learning request

[2476] Select the subject and area you want to study and make a request on your device.

[2477] 2. Answers to the questions

[2478] Answer the questions displayed on the terminal and enter the answers into the terminal.

[2479] 3. Review feedback

[2480] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[2481] 4. Practical skills training in a virtual environment

[2482] In a virtual environment, practical skills can be learned through simulations, such as practicing customer service and sales skills in a virtual store.

[2483] Hardware and software used

[2484] The server runs programs using languages ​​and frameworks such as Python and Django.

[2485] The terminal provides applications that run on a variety of devices, including smartphones, tablets, smart glasses, and head-mounted displays.

[2486] The database uses a relational database management system such as MySQL or PostgreSQL.

[2487] The generative AI model uses machine learning and natural language processing frameworks (such as TensorFlow and PyTorch) to generate questions and provide feedback.

[2488] Specific examples

[2489] If you're learning customer service skills in a virtual environment:

[2490] 1. User makes a learning request

[2491] The user selects "customer service skills" on the terminal and sends a learning request.

[2492] 2. Generate a problem on the server

[2493] The server generates questions such as "If a customer asks you a question about a product, how would you respond?" and sends them to the terminal.

[2494] 3. The questions are displayed on the device and the user answers them.

[2495] The terminal displays the virtual environment and initiates a dialogue with the virtual customer, who inputs answers and sends them to the server.

[2496] 4. The server receives the answer, determines whether it is correct, and provides feedback.

[2497] The server analyzes the answer and provides real-time feedback such as, "The answer given to the customer was appropriate. Here's what would be best next time."

[2498] Prompt Sentence Examples

[2499] Use the following information to generate questions to teach customer service skills in a virtual store:

[2500] Categories: Product knowledge, complaint handling, sales skills

[2501] Example: Describe the features of this product.

[2502] In this way, learners can effectively learn practical skills in a virtual environment.

[2503] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2504] Step 1:

[2505] The terminal accepts learning requests from users. The user selects the subject and scope of the learning they wish to do through the terminal interface and sends the request. The input is the subject and scope information, and the output is the request data sent to the server.

[2506] Step 2:

[2507] The server receives request data sent from the terminal. The input is the learning request data, and the output is a trigger to invoke the problem generation algorithm based on the request data.

[2508] Step 3:

[2509] The server generates questions based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history. The input is the subject and scope information and past answer history data, and the output is the generated question data.

[2510] Step 4:

[2511] The server sends the generated questions to the terminal. The input is the generated question data, and the output is the question data sent to the terminal.

[2512] Step 5:

[2513] The terminal displays the questions sent from the server to the learner. The input is the question data, and the output is the question displayed on the terminal interface.

[2514] Step 6:

[2515] The user answers the questions displayed on the terminal. The user enters the answer into the terminal and presses the send button. The input is the user's answer, and the output is the answer data sent to the server.

[2516] Step 7:

[2517] The server receives the answer from the user and judges whether the answer is correct or not. The input is the user's answer data, and the output is the result of the judgment.

[2518] Step 8:

[2519] The server sends the result of the accuracy judgment to the terminal. The input is the accuracy judgment result, and the output is feedback data sent to the terminal.

[2520] Step 9:

[2521] The terminal displays the result of the judgment and explanation from the server to the user. The input is the feedback data, and the output is the feedback displayed on the terminal interface.

[2522] Step 10:

[2523] The server stores the user's answer history in a database. The input is the answer data and the result of the correct / incorrect judgment, and the output is the answer history data stored in the database.

[2524] Step 11:

[2525] The server periodically analyzes the answer history and identifies the user's weaknesses. The input is the answer history data, and the output is the analysis results and the identified weaknesses.

[2526] Step 12:

[2527] The server generates original questions based on the identified weaknesses. The input is the weakness analysis result, and the output is the generated original question data.

[2528] Step 13:

[2529] The server sends the generated original question to the terminal. The input is the original question data, and the output is the original question data sent to the terminal.

[2530] Step 14:

[2531] The terminal displays the original problem to the user. The input is the original problem data, and the output is the original problem displayed on the interface.

[2532] Step 15:

[2533] The terminal runs the virtual environment and allows users to experience practical skills. The input is the data of the virtual environment simulation, and the output is the virtual environment that the user can view and manipulate.

[2534] Step 16:

[2535] The server provides appropriate feedback in real time based on the answers provided by the user. The input is real-time answer data, and the output is real-time feedback data.

[2536] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2537] A learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine.

[2538] System Configuration

[2539] Server part

[2540] The server implements the following functions:

[2541] 1. Acceptance of subject and scope selection

[2542] The server receives a learning request sent from a terminal and identifies a specific subject and area.

[2543] 2. Problem Generation

[2544] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[2545] 3. Receiving answers and determining whether they are correct or incorrect

[2546] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[2547] 4. Saving and analyzing answer history

[2548] The server stores the answer history in a database and periodically analyzes it to identify the learner's weaknesses.

[2549] 5. Generating original questions

[2550] Original questions are generated to reinforce weak points identified from the analysis results.

[2551] 6. Customize your study plan

[2552] We provide individually customized study plans based on the learner's answer history and weakness analysis results.

[2553] 7. Emotion Data Analysis

[2554] The server analyzes the learner's emotional data obtained from the emotion engine and adjusts the difficulty of the questions and the learning plan based on that data.

[2555] Terminal part

[2556] The device implements the following features:

[2557] 1. Submit a learning request

[2558] Learners select the subject and scope through their terminal and send that information to the server.

[2559] 2. Displaying the problem

[2560] The terminal displays the questions received from the server to the learner.

[2561] 3. Enter and submit your answers

[2562] The learner enters the answer and sends it to the server via the terminal.

[2563] 4. Viewing Feedback

[2564] The device displays the correct / incorrect results and explanations from the server and provides feedback to the learner.

[2565] 5. Collecting Emotional Data

[2566] The device collects emotional data from the learner's facial expressions and voice, and sends it to the emotion engine.

[2567] User Part

[2568] The user does the following:

[2569] 1. Select a learning request

[2570] Select the subject and area you want to study and make a request on your device.

[2571] 2. Answers to the questions

[2572] Answer the questions displayed on the terminal and enter the answers into the terminal.

[2573] 3. Review feedback

[2574] Check the feedback displayed on your device and learn the correct answers and explanations for the questions you got wrong.

[2575] 4. Providing Emotion Data

[2576] Emotions during learning are collected by the device, and data such as facial expressions and voice are analyzed by the emotion engine.

[2577] Emotion engine part

[2578] The emotion engine implements the following functions:

[2579] 1. Emotion recognition and collection

[2580] Recognize and collect emotions from learners' facial expressions and voice data.

[2581] 2. Transmission of emotional information

[2582] The recognized emotion information is sent to the server.

[2583] 3. Emotion-Based Adjustment

[2584] Based on the emotional information provided to the server, the learner's stress level and concentration level are determined and the difficulty of the questions and the content of the feedback are adjusted accordingly.

[2585] Specific examples

[2586] Learning multiplication in arithmetic using an emotion engine

[2587] 1. User makes a learning request

[2588] The user selects the mathematics subject on the terminal and requests the range for "multiplication."

[2589] 2. Generate a problem on the server

[2590] The server generates a problem such as "28 x 36" and sends it to the terminal.

[2591] 3. Display the problem on the device and collect sentiment data

[2592] The device displays the question "What is 28 x 36?" and simultaneously transmits the user's facial expressions and voice data to the emotion engine.

[2593] 4. The user enters the answer

[2594] The user enters the answer "1008".

[2595] 5. The server receives the answer and analyzes the emotion data.

[2596] The server determines whether the answer "1008" is correct and uses the results of the emotional data analysis along with feedback.

[2597] 6. Feedback and difficulty adjustments

[2598] Based on the analysis results of the emotion engine, the server adjusts the feedback and changes the difficulty of the questions appropriately.

[2599] 7. View feedback on your device

[2600] The device will display "That's correct!" or "That's incorrect. The correct answer is 1008." and will also display encouragement or advice depending on the user's emotions.

[2601] In this way, incorporating learners' emotions makes it possible to provide more effective and personalized learning support.

[2602] The processing flow will be explained below.

[2603] Step 1:

[2604] The user starts the learning application on the device and selects the subject and area they want to study. Let's assume that they select "Multiplication" in "Math."

[2605] Step 2:

[2606] The terminal transmits information about the subject and range selected by the user to the server.

[2607] Step 3:

[2608] The server analyzes the received subject and scope information and generates appropriate questions, such as multiplication questions like "28 x 36."

[2609] Step 4:

[2610] The server sends the generated questions to the terminal.

[2611] Step 5:

[2612] The terminal displays the question received from the server to the user: "What is 28 x 36?"

[2613] Step 6:

[2614] At the same time as displaying the image, the device sends the user's facial expressions and voice data to the emotion engine.

[2615] Step 7:

[2616] The emotion engine recognizes and analyzes the user's emotions from the facial and voice data it receives. For example, if the user has a troubled expression, it will determine that they are feeling stressed.

[2617] Step 8:

[2618] The user answers the displayed question and enters the answer into the terminal. Let's consider the case where the user enters "1008."

[2619] Step 9:

[2620] The device sends the user's answer to the server and also sends the emotion engine's analysis results to the server.

[2621] Step 10:

[2622] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1008" is correct.

[2623] Step 11:

[2624] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1008."

[2625] Step 12:

[2626] The server stores the answer history in a database and analyzes it, identifying the learner's weaknesses based on the analysis results.

[2627] Step 13:

[2628] The server adjusts the next learning step based on the analysis results and data from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, it adjusts the difficulty of the questions.

[2629] Step 14:

[2630] The server generates new original questions and sends the information to the terminal. It generates new questions such as "What is 34 x 47?"

[2631] Step 15:

[2632] The terminal displays the original questions received from the server to the user.

[2633] Step 16:

[2634] The device again sends the user's facial expressions and voice data to the emotion engine.

[2635] Step 17:

[2636] The emotion engine recognizes and analyzes emotions from new facial and voice data.

[2637] Step 18:

[2638] The user answers the original question displayed and enters the answer into the terminal. Let's consider the case where the user enters "1598."

[2639] Step 19:

[2640] The device sends the user's answer to the server, as well as the emotion engine's new analysis results.

[2641] Step 20:

[2642] The server compares the received answer with the correct answer to determine whether it is correct. In this case, it checks whether the answer "1598" is correct.

[2643] Step 21:

[2644] The server sends the result of the accuracy judgment to the terminal, sending feedback such as "That's correct!" or "That's incorrect. The correct answer is 1598."

[2645] Step 22:

[2646] The device displays the feedback received from the server to the user, including encouragement and advice based on the user's emotions.

[2647] Step 23:

[2648] By repeating this series of processes, users can reinforce their weaknesses while progressing with optimal learning based on emotional data.

[2649] Example 2

[2650] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2651] Conventional learning support systems generate questions based on the learner's academic level and reinforce weak points by analyzing the learner's answer history, but they lack the ability to individually customize the system to take into account the learner's emotional state. As a result, learners' stress and decreased motivation can affect their learning outcomes. Therefore, the present invention aims to provide a more effective and personalized learning support system that takes into account the learner's emotional state.

[2652] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2653] In this invention, the server includes means for receiving selection of a subject and range from the learner, means for generating questions based on the subject and range, and means for analyzing collected emotional data to adjust the difficulty of the study plan and questions, thereby making it possible to provide a study plan that takes into account the emotional state of the learner.

[2654] "Learner" refers to a person who uses the system to acquire knowledge or skills.

[2655] A "subject" refers to a particular field or topic that a learner will study.

[2656] "Scope" refers to the part of the subject that the learner particularly wants to study.

[2657] "Problems" are questions or tasks provided to assess a learner's understanding or skills.

[2658] "Answer" refers to the answer to the question submitted by the learner.

[2659] "Correct or incorrect" refers to the result of determining whether the learner's answer is correct or incorrect.

[2660] "Answer history" refers to a record of answers previously provided by a learner.

[2661] "Weak points" refer to areas in which a learner is particularly weak, as identified by analyzing the answer history.

[2662] "Original questions" refer to questions that are specially created to reinforce a learner's weak points.

[2663] "Emotional data" refers to information about emotions collected from learners' facial expressions, voice, etc.

[2664] "Emotion engine" refers to technology for analyzing collected emotional data and providing the results to the system.

[2665] "Study plan" refers to the learning progress plan and content provided to learners.

[2666] "Database" refers to an information management system for storing learners' answer history and other information.

[2667] The learning support system for specifically implementing the present invention is composed of four components: a server, a terminal, a user, and an emotion engine. The functions and specific processes of each component will be explained in detail below.

[2668] System Configuration

[2669] Server part

[2670] The server has the following functions:

[2671] 1. Acceptance of subject and scope selection

[2672] The server receives the learning request sent from the device and identifies the specific subject and area, and stores this information in a database.

[2673] Example: When a request for "arithmetic" and "multiplication" is sent from the terminal, the server receives it.

[2674] 2. Problem Generation

[2675] The server generates questions at an appropriate level based on the received subject and scope information. The question generation algorithm takes into account the learner's academic level and previous answer history.

[2676] Example: Generate questions of appropriate difficulty, such as "15 x 12," based on the learner's academic level.

[2677] 3. Receiving answers and determining whether they are correct or incorrect

[2678] The server receives the learner's answer and compares it with the preset correct answers to determine whether it is correct or incorrect.

[2679] Example: A learner sends the answer "180" to the server, and the server determines this to be the "correct answer."

[2680] 4. Saving and analyzing answer history

[2681] The server stores the answer history in a database and periodically analyzes it to identify the learner's weak points.

[2682] Example: Save a history of frequent mistakes a learner makes on multiplication problems and analyze the data.

[2683] 5. Generating original questions

[2684] Original questions are generated to reinforce weak points identified from the analysis results.

[2685] For example, if a weakness in "multiplication" is identified, generate and provide similar multiplication problems.

[2686] 6. Customize your study plan

[2687] We provide individually customized study plans based on the ...

Claims

1. a means for accepting subject and area selections from learners; means for generating questions based on the subject and scope; a means for providing the generated questions to the learner; a means for receiving answers from learners; a means for determining whether the received answers are correct; A means for providing the assessment result to the learner; A means for identifying learners' weaknesses by analyzing their answer history; a means for generating original questions based on the identified weaknesses; A means for providing the generated original questions to the learner; A system including:

2. 2. The system according to claim 1, further comprising means for storing the answer history of the learner in a database.

3. 2. The system according to claim 1, further comprising means for adjusting the difficulty level of the original questions in accordance with the level of the learner.

4. 10. The system of claim 1, further comprising means for accepting feedback from a learner and adjusting the question generation algorithm based on the feedback.

5. 2. The system according to claim 1, further comprising means for providing a customized learning plan to the learner based on the analysis results.

Images