System

Abstract

A system is provided.SOLUTION: A system comprising: means for importing textbook data and past examination question data; means for analyzing the imported data and converting the data into text data; means for generating a new question using generative artificial intelligence; means for collecting user's answer data and determining whether the answer data is correct or incorrect; means for analyzing a user's learning tendency and extracting a specific weak area; means for giving a question intensively based on the weak area; and means for continuously monitoring the user's learning progress and providing feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's exam preparation, repeated practice of calculation problems and memorization subjects is essential, but storing numerous reference books and past exam questions on paper is not practical due to space and cost considerations. There is also a need for efficient management of each study material and optimization of study methods. However, existing study tools lack the functionality to automatically analyze users' learning trends and identify and focus on weak areas. Therefore, there is a need for a means to enable users to effectively overcome their weaknesses and study efficiently. [Means for solving the problem]

[0005] This invention provides a means for importing textbook data and past exam question data, a means for analyzing the imported data and converting it into text data, a means for generating new questions using generative artificial intelligence, and a means for collecting user answer data and determining whether they are correct or incorrect. It also includes a means for analyzing the user's learning tendencies and identifying specific areas of weakness, and a means for providing questions focused on those areas of weakness. This allows for the creation of a system that continuously monitors the user's learning progress and provides feedback. This system also includes a means for selecting and delivering questions tailored to specific subjects and levels of difficulty based on the user's requests, and a means for analyzing and digitally saving special data such as mathematical formulas and graphs.

[0006] "Textbook data" refers to information stored in digital format containing the contents of teaching materials used in educational institutions.

[0007] "Past exam question data" refers to information stored in digital format about exam questions that have been asked in the past.

[0008] "Means of import" refers to the functions and processes for inputting textbook data and past exam question data into the system.

[0009] "Means for analyzing and converting to text data" refers to the ability to convert captured data into text format using optical character recognition (OCR) or other technology.

[0010] "Generative AI" refers to AI that automatically generates new problems using machine learning and natural language processing.

[0011] "Means of generation" refers to the processes and functions for creating new questions based on textbook data and past test question data.

[0012] "Answer data" refers to the answer information entered by the user in response to a question.

[0013] "Means for determining whether the answer is correct" refers to a function for comparing the user's answer with the correct answer and determining whether it is correct or not.

[0014] "Means for analyzing learning trends and extracting specific areas of weakness" refers to a function that analyzes a user's learning data and identifies areas in which the user is weak.

[0015] "Measures to focus on" refers to a function that presents users with many questions related to the extracted weak areas.

[0016] "Means for monitoring and providing feedback" refers to the function of continuously observing the user's learning progress and providing advice and information for improvement.

[0017] "Means for selecting and delivering questions that are appropriate for a particular subject or level of difficulty" refers to the function of selecting questions of an appropriate subject and level of difficulty based on a user's request and sending them to the user's device.

[0018] "Means for analyzing special data such as mathematical formulas and graphs and storing them in a digital format" refers to the ability to analyze mathematical notation and visual information and store it in the system as digital data. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] System Overview

[0041] This invention is a learning support system for exam preparation. This system imports textbook data and past exam question data, analyzes that data, converts it into text format, and automatically generates new questions using a generation AI. Users answer the questions, and the answer data is collected and analyzed on a server, which identifies the user's weak areas and presents focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback.

[0042] Importing and analyzing textbook data and past exam data

[0043] Importing textbook data

[0044] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can upload "2023 Mathematics Past Questions" to the app.

[0045] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[0046] Question generation and distribution

[0047] Problem Generation

[0048] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[0049] Delivery in response to user requests

[0050] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[0051] Server: In response to a user request, selects appropriate problems and delivers them to the user's device. For example, it selects intermediate level math problems and sends them to the user's device.

[0052] Answer and learning trend analysis

[0053] Collecting and assessing answers

[0054] User's device: The user answers the delivered questions and sends the answer data to the server. For example, the user answers "x = -1, -2" to the question "Find the solution to y = x^2 + 3x + 2."

[0055] Server: The submitted answers are evaluated by an automatic scoring algorithm to determine whether they are correct or incorrect. The analysis results are stored in a database.

[0056] Analysis of learning trends

[0057] Server: Analyzes the user's answer data and identifies areas in which the user has particular difficulty. For example, it may be discovered that the user has difficulty with quadratic equations.

[0058] Focused questions and feedback on weak areas

[0059] Focused questions on weak areas

[0060] Server: Based on the user's learning tendencies, it generates new problems that are specific to weak areas. For example, it generates a large number of problems related to "quadratic equations."

[0061] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[0062] Providing Feedback

[0063] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0064] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[0065] Case Study

[0066] For example, if a user has difficulty with quadratic equations in mathematics, the system will analyze the user's answer data, generate and prioritize problems related to quadratic equations, and monitor the user's progress, providing feedback on how to improve their weaknesses. In this way, users can efficiently overcome their weaknesses and maximize their learning outcomes.

[0067] In the above-described form, the present invention provides a system that supports efficient learning for learners and enables them to effectively prepare for exams.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0071] Step 2:

[0072] Server: Receives the captured data and converts it into text data using OCR (Optical Character Recognition) technology, for example, analyzing text and mathematical formulas in images and converting them into digital text.

[0073] Step 3:

[0074] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[0075] Step 4:

[0076] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[0077] Step 5:

[0078] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[0079] Step 6:

[0080] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0081] Step 7:

[0082] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[0083] Step 8:

[0084] User's device: Sends the answer to the server.

[0085] Step 9:

[0086] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[0087] Step 10:

[0088] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[0089] Step 11:

[0090] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[0091] Step 12:

[0092] On the user's device: Questions related to weak areas are highlighted, and the user is encouraged to retry those questions.

[0093] Step 13:

[0094] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0095] Step 14:

[0096] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional learning support systems have struggled to efficiently analyze users' learning progress and weak areas and provide appropriate questions. They also lack the ability to accurately analyze special data such as mathematical formulas and graphs to generate and deliver questions tailored to the subjects and level of difficulty desired by users. Furthermore, they lacked a means to effectively collect users' answer data and provide continuous feedback.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for importing textbook data and past exam question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions that are focused on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for the user to request questions by specifying a specific subject or level of difficulty, and means for selecting and delivering appropriate questions in response to the user's request. This makes it possible to provide appropriate questions in accordance with the user's learning tendencies and to provide learning support through continuous feedback.

[0102] "Textbook data" refers to data that includes text information of learning materials used in learning.

[0103] "Past exam question data" refers to data that records the content of questions from exams that have been administered in the past.

[0104] "Analysis" is the process of deciphering captured data and converting it into text data or digital format.

[0105] "Text data" is data that represents character information in digital form.

[0106] "Generative AI" is an AI technology for automatically generating new problems.

[0107] "Means for generating questions" refers to technology that creates new study questions based on textbook data and past exam question data.

[0108] "User answer data" is data that includes the answers to questions answered by the user.

[0109] The "means for determining whether an answer is correct or incorrect" is a technique for determining whether an answer is correct or incorrect based on the user's answer data.

[0110] "Study trends" is information on learning progress and weak areas analyzed based on the user's question answer patterns and grades.

[0111] "Weak areas" refer to areas of study or types of questions that the user is particularly weak at.

[0112] "Feedback" is information about improvements and advice provided based on the user's learning progress.

[0113] "Means for requesting" is a function that allows the user to request questions by specifying a specific subject or level of difficulty.

[0114] "Distribution means" refers to the technology for transmitting generated questions and feedback to the user's terminal.

[0115] MODE FOR CARRYING OUT THE INVENTION

[0116] The present invention relates to a learning support system for exam preparation. This system imports textbook data and past exam question data uploaded by users, analyzes the data, converts it into text format, and generates new questions. The generated questions are distributed upon user request, allowing the user to answer them. The user's answer data is collected, and correct answers are determined and their learning trends are analyzed. The system then presents questions that focus on the user's weak areas, and continuously monitors the user's learning progress to provide appropriate feedback. Specific embodiments are described below.

[0117] Importing and analyzing textbook data and past exam data

[0118] User device: The user uses the application to upload textbook and past exam question data. For example, by selecting a PDF file of "2023 Mathematics Past Questions" and pressing the upload button, the data is sent to the server.

[0119] Server: The server receives the textbook data uploaded from the user's device. The received data is converted into text data using OCR (Optical Character Recognition) technology. During this process, special data such as mathematical formulas and graphs are also recognized and saved as text or figures.

[0120] Question generation and distribution

[0121] Server: The server generates new questions using a generative AI (artificial intelligence) model based on the analyzed textbook data. For example, the generative AI model generates a question asking about the solution to a quadratic equation.

[0122] On the user's device: The user specifies the subject and difficulty level they want to study through the application. For example, they may request "Intermediate level math problems."

[0123] Server: The server receives the user's request, selects appropriate problems from the generated questions, and delivers them to the user's device. For example, a set of intermediate level math problems is sent to the user's device.

[0124] Collecting and assessing answer data

[0125] User's device: The user answers the received question and sends the answer data to the server via the application. For example, in response to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the user answers "x = 2, x = 3."

[0126] Server: The server analyzes the received answer data using an automatic scoring algorithm and determines whether the answer is correct. If the answer is correct, the server presents the next question, and if the answer is incorrect, it displays an explanation.

[0127] Learning trend analysis and feedback

[0128] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendency. For example, it discovers that the user's average correct answer rate for quadratic equation problems is 50%.

[0129] Server: The server generates feedback for the user based on the analysis results, for example, a message saying "You need to practice more to improve your accuracy on quadratic equation problems."

[0130] User's device: The user's device receives and displays the feedback sent from the server, for example, a notification saying, "Try some extra practice to improve your accuracy on quadratic equation problems."

[0131] Focused questions on weak areas

[0132] Server: Based on the analysis results, the server generates new problems that are specific to the user's weak areas. For example, it generates a large number of problems related to "quadratic equations."

[0133] On the user's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed.

[0134] Examples of concrete examples and prompts

[0135] As a concrete example, consider the case where a user has difficulty with quadratic equations in mathematics. The user uploads "2023 Math Past Papers" to the app, which is then received and analyzed by the server. Then, a generation AI is used to generate "quadratic equation problems" and deliver them as the "intermediate level math problems" requested by the user.

[0136] An example prompt is:

[0137] "Generate new quadratic equation problems from past math questions for 2023. A user has requested intermediate level math problems."

[0138] This allows the system to provide an environment in which users can study efficiently.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Program processing flow

[0141] Step 1:

[0142] User's device: The user uses the learning support system's application to upload textbook and past exam question data. For example, they select a PDF file of "2023 Mathematics Past Questions" and press the upload button. This input data (textbook data) is sent to the server.

[0143] Step 2:

[0144] Server: The server receives textbook data uploaded from the user's device. This received data is converted into text data using OCR (optical character recognition) technology. Specifically, it analyzes images and text within the PDF and converts them into text format. Special data such as mathematical formulas and graphs are also converted into digital format as text or figures and saved. The output data is text data.

[0145] Step 3:

[0146] Server: The server uses a generative AI model to generate new problems based on the analyzed text data. For example, the generative AI model generates a "problem asking about the solution to a quadratic equation." Specifically, it takes as input a prompt statement that specifies the difficulty and content of the problem to be generated, and outputs the generated new problem data. The output data is the generated problem set.

[0147] Step 4:

[0148] User's device: The user requests questions through the application, specifying the subject and level of difficulty they want to study. For example, they might request "Please give me intermediate level math questions." The input data is the user's request.

[0149] Step 5:

[0150] Server: The server receives the user's request, selects the appropriate problem from the generated problem set, and delivers it to the Yuga terminal. Specifically, it searches the problem database based on the request content, selects the appropriate problem, and delivers it. The output data is the result of delivering the problem to the user terminal.

[0151] Step 6:

[0152] User's device: The user answers the delivered questions and sends the answer data to the server through the application. For example, to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the answer is "x = 2, x = 3." The input data is the user's answer.

[0153] Step 7:

[0154] Server: The server uses an automatic scoring algorithm to evaluate the received answer data and confirm whether it is correct or not. Specifically, it compares the answer data with a correct answer database and determines whether it is correct. The output data is the result of the evaluation.

[0155] Step 8:

[0156] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendencies. Specifically, it analyzes data such as the correct answer rate and the time it takes to answer, and identifies specific weak areas. The output data is the learning tendency analysis results.

[0157] Step 9:

[0158] Server: The server generates feedback for the user based on the analysis results. For example, it creates a message saying, "You need to practice more to improve your accuracy in solving quadratic equation problems." Specifically, it takes the learning trend analysis results as input, generates a feedback message, and sends it. The output data is the feedback message.

[0159] Step 10:

[0160] User's device: The user's device receives and displays the feedback sent from the server. For example, a notification such as "Please practice more to improve your accuracy on quadratic equation problems" is displayed. The input data is the feedback message.

[0161] Step 11:

[0162] Server: Based on the analysis results, the server generates new questions that are specific to the user's weak areas. Specifically, it uses information about the weak areas as input and provides prompts to the generative AI model to create new questions. The output data is the newly generated questions.

[0163] Step 12:

[0164] User's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed. The input data is the newly generated problem.

[0165] (Application example 1)

[0166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] Conventional learning support systems limit students to a passive learning experience where they simply solve problems, making it difficult to improve learning efficiency. Furthermore, they often lack the ability to select questions tailored to specific subjects or levels of difficulty, or to analyze learning trends based on users' answer data, making it difficult to provide effective feedback or prioritized questions. Furthermore, they lack real-time learning progress monitoring or learning support utilizing virtual spaces, making it difficult to maintain users' motivation to learn.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0169] In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether the answers are correct, means for analyzing the user's learning tendencies and identifying specific areas of weakness, means for providing focused questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, and means for supporting learning in a virtual space using a head-mounted display or smart glasses. This allows users to learn interactively in a virtual space, breaking away from passive learning and significantly improving learning efficiency and motivation.

[0170] "Textbook data" refers to digital data of learning materials and textbooks used in learning.

[0171] "Past exam question data" refers to digitized data of exam questions that have been administered in the past.

[0172] "Capturing means" refers to the method or technology by which the subject data is entered or uploaded into the system.

[0173] "Means for analyzing and converting to text data" refers to techniques or methods for analyzing the captured data and converting it into a readable text format.

[0174] "Generative AI" refers to systems or algorithms that use artificial intelligence techniques to generate new data and information.

[0175] "Means of generating questions" refers to techniques and methods for creating new questions based on textbook data and past exam question data.

[0176] "User answer data" refers to data of the answers entered by the user in response to the questions posed.

[0177] The "means for determining whether an answer is correct" refers to a technique or method for analyzing the user's answer data and determining whether the answer is correct or incorrect.

[0178] "Means for analyzing learning trends" refers to techniques and methods for analyzing a user's learning patterns and trends based on the user's answer data.

[0179] "Means for extracting weak areas" refers to techniques or methods for analyzing a user's learning data and identifying areas in which the user is particularly weak.

[0180] A "method of providing focused questions" is a technique or method of providing focused questions based on the user's areas of weakness.

[0181] "Means for continuously monitoring learning progress" refers to techniques and methods for constantly monitoring a user's learning status and tracking progress.

[0182] "Means for providing feedback" refers to techniques and methods for providing advice and guidance according to the user's learning situation.

[0183] A "head-mounted display" is a display device that displays a virtual space when worn by a user on the head.

[0184] "Smart glasses" are glasses-type devices that can display augmented reality and virtual reality.

[0185] "Means to support learning in virtual spaces" are technologies and methods that use virtual reality or augmented reality technology to help users learn in virtual spaces.

[0186] This invention provides a system that allows users to study effectively in a virtual space. This system imports textbook data and past exam question data, converts the data into text format, and generates new questions using a generative artificial intelligence (generative AI model). It also collects and analyzes user answer data, extracts specific areas of weakness, and provides focused questions and feedback.

[0187] Importing and analyzing textbook data and past exam data

[0188] The system receives textbook data and past exam question data from users' devices. Users can take photos of this data or upload it as a PDF or Doc file. This data is received by the server and converted into text data using OCR technology (e.g., Tesseract). Special data (such as mathematical formulas and graphs) is also analyzed and saved in digital format.

[0189] Question generation and distribution

[0190] The server generates new questions using a generative AI model (e.g., OpenAI GPT-4) based on textbook data and past exam question data. The generated questions are delivered in response to user requests. The user specifies a specific subject and level of difficulty, and the server selects appropriate questions based on that request and delivers them to the user's device.

[0191] Answer and learning trend analysis

[0192] Users answer the questions and send the answers to the server. The server then uses an automated scoring algorithm (e.g., Python and Scikit-Learn) to evaluate the answers and assess whether they are correct. The analysis results are stored in a database, and specific areas of weakness are identified by analyzing the user's learning patterns.

[0193] Learning support in virtual space

[0194] Users use a head-mounted display (HMD) or smart glasses to enter a virtual space and study. Within the virtual space, textbook data and generated questions are displayed as 3D objects, which users can manipulate interactively. In the virtual classroom, users answer questions using a virtual pen in their hand. Answer data is collected and analyzed in real time, and a virtual assistant provides appropriate feedback.

[0195] Examples and prompts

[0196] For example, if a user is learning quadratic equations in a virtual space, the learning progresses through the following steps.

[0197] Example: Quadratic equation problem: "Find the solution to y = x^2 + 3x + 2"

[0198] User's solution: The user writes "x = -1, -2" on the whiteboard in the virtual space with a virtual pen.

[0199] feedback:

[0200] A virtual assistant will appear and provide feedback, saying, "That's right! Try the next one," and then a more difficult question will be displayed.

[0201] Example prompt sentence:

[0202] A user is solving a quadratic equation problem in a virtual classroom. The answer is "x = -1, -2." Please provide appropriate feedback for this correct answer.

[0203] In this way, users can learn interactively in a virtual space, moving away from passive learning and significantly improving their learning efficiency and motivation.

[0204] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0205] Step 1:

[0206] Importing textbook data and past exam question data

[0207] User's device: Users can take photos of textbook data and past exam question data or upload them as PDF or Doc files.

[0208] Input: Image data, PDF files, Doc files

[0209] Output: Raw data sent to the server

[0210] What happens: A user opens the application, selects a textbook or exam question file, and clicks the upload button. The file is then sent to the server.

[0211] Step 2:

[0212] Data analysis and conversion to text data

[0213] Server: Converts received data into text using OCR technology (such as Tesseract). Analyzes special data such as mathematical formulas and graphs and saves them in digital format.

[0214] Input: Raw data (image, PDF, Doc)

[0215] Output: Text data, special data in digital format

[0216] How it works: The server analyzes the received files and uses OCR technology to convert the contents of textbooks and exam questions into text data. It also recognizes mathematical formulas and graphs and saves them as digital data.

[0217] Step 3:

[0218] Problem Generation

[0219] Server: Generates new questions using a generative AI model (such as OpenAI GPT-4) based on textbook data and past exam question data.

[0220] Input: Text data, past exam question data

[0221] Output: The newly generated problem data

[0222] How it works: The server analyzes the text data and provides it as input to the generative AI model, which then generates new questions and stores them in a database.

[0223] Step 4:

[0224] Problem delivery based on user requests

[0225] Server: Accepts user requests for questions based on specific subjects and difficulty levels, selects appropriate questions based on those requests, and delivers them to the user's device.

[0226] Input: User's problem request (subject, difficulty level)

[0227] Output: Selected question data, delivered to the terminal

[0228] Specific operation: The user selects the subject and difficulty level on the app and requests questions. The server receives the request, selects appropriate questions, and delivers them to the user's device.

[0229] Step 5:

[0230] Collecting answer data and determining correctness

[0231] User's device: The user answers the questions delivered and sends the answer data to the server.

[0232] Server: Determines whether the answer is correct or incorrect based on an algorithm and stores the analysis results in a database.

[0233] Input: User's answer data

[0234] Output: Correct / incorrect result, saved in database

[0235] Specific operation: The user answers questions in the virtual space, inputs the answers, and sends them to the server. The server analyzes the answers, determines whether they are correct, and saves the results.

[0236] Step 6:

[0237] Analysis of learning trends and identification of weak areas

[0238] Server: Analyzes user answer data, identifies learning trends, and identifies specific areas of weakness.

[0239] Input: Correct / incorrect results, past answer data

[0240] Output: User learning tendency data, weak area data

[0241] Specific operation: The server runs an algorithm that analyzes learning trends based on the user's answer data stored in the database and extracts specific areas of weakness.

[0242] Step 7:

[0243] Focused questions and feedback

[0244] Server: Generates new questions based on the extracted weak areas and presents them to the user in a focused manner. In addition, it monitors the learning progress and provides appropriate feedback.

[0245] Input: Weakness Area Data

[0246] Output: New problem data, feedback message

[0247] Specific operation: The server generates new questions related to the weak areas and delivers them to the user's device. It also generates and provides feedback messages to the user based on their learning progress.

[0248] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0249] System Overview

[0250] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[0251] Importing and analyzing textbook data and past exam data

[0252] Importing textbook data

[0253] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0254] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[0255] Question generation and distribution

[0256] Problem Generation

[0257] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[0258] Delivery in response to user requests

[0259] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[0260] Server: In response to a user request, selects an appropriate problem and delivers it to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0261] Answer and learning trend analysis

[0262] Collecting and assessing answers

[0263] User's device: The user answers the delivered problem and sends the answer data to the server. For example, the user answers the problem "Find the solution to y = x^2 + 3x + 2" as "x = -1, -2".

[0264] Server: The submitted answer is evaluated by an automatic scoring algorithm to determine whether it is correct or not. For example, it determines that the user's answer "x = -1, -2" is correct.

[0265] Analysis of learning trends

[0266] Server: Analyzes the user's answer data and identifies areas of particular difficulty. For example, analyzes how well the user answers quadratic equation problems.

[0267] Focused questions and feedback on weak areas

[0268] Focused questions on weak areas

[0269] Server: Based on the user's learning tendencies, generate new problems that are specific to weak areas. For example, generate additional problems related to "quadratic equations."

[0270] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[0271] Providing Feedback

[0272] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0273] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[0274] Incorporating an emotion engine

[0275] emotion recognition

[0276] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[0277] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[0278] Emotion-based problem adjustment

[0279] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[0280] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[0281] Providing emotion-based feedback

[0282] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[0283] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[0284] In the above-described manner, the present invention constructs a system that takes into account the emotional state of the learner and can provide a more personalized learning experience.

[0285] The processing flow will be explained below.

[0286] Step 1:

[0287] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0288] Step 2:

[0289] Server: Receives the captured data. The received data is converted into text data using OCR technology. Special data such as formulas and graphs are also analyzed and saved in digital format.

[0290] Step 3:

[0291] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[0292] Step 4:

[0293] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[0294] Step 5:

[0295] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[0296] Step 6:

[0297] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0298] Step 7:

[0299] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[0300] Step 8:

[0301] User's device: Sends the answer to the server.

[0302] Step 9:

[0303] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[0304] Step 10:

[0305] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[0306] Step 11:

[0307] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[0308] Step 12:

[0309] On the user's device: Questions related to weak areas are displayed with emphasis, and the user is encouraged to retry those questions.

[0310] Step 13:

[0311] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0312] Step 14:

[0313] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[0314] Step 15:

[0315] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[0316] Step 16:

[0317] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[0318] Step 17:

[0319] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[0320] Step 18:

[0321] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[0322] Step 19:

[0323] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[0324] Step 20:

[0325] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[0326] Example 2

[0327] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0328] Conventional learning support systems provide questions based on the user's learning progress and weak areas, but they lack the flexibility to consider the user's emotional state. This can lead to a decrease in the user's concentration and stress, potentially reducing the effectiveness of their learning. Furthermore, their ability to provide questions based on the user's specific requests is limited, leaving a need for more effective learning support.

[0329] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0330] In this invention, the server includes a means for importing textbook data and past exam question data, a means for analyzing the imported data and converting it into text data, a means for generating new questions using generative artificial intelligence, a means for collecting user answer data and determining whether the answers are correct, a means for analyzing the user's learning tendencies and identifying specific areas of weakness, a means for providing questions focused on the areas of weakness, a means for continuously monitoring the user's learning progress and providing feedback, and a means for recognizing the user's emotional state and reflecting it in learning activities. This enables personalized question provision and feedback that takes the user's emotional state into account, improving learning effectiveness. Furthermore, questions of specific subjects and levels of difficulty can be provided in response to the user's requests, supporting efficient learning.

[0331] "Textbook data" refers to digital data such as text information, mathematical formulas, and graphs contained in textbooks.

[0332] "Past exam question data" refers to information on exam questions that have been asked in the past, stored in digital format.

[0333] "Generative AI" is an AI technology that has the ability to generate new information and solutions based on input data.

[0334] "User answer data" refers to answer information entered by the user in response to the questions provided.

[0335] "Learning trends" refers to data that indicates specific patterns or progress observed in the user's learning process.

[0336] "Weakness areas" are areas of learning where the user does not fully understand or is less accurate in their answers.

[0337] "Feedback" refers to messages of encouragement and suggestions for improvement provided based on the user's learning progress and answer results.

[0338] "Emotional state" refers to the user's current mental and psychological state, such as whether they are focused, tired, or stressed.

[0339] A "camera" is a device that optically captures information and stores and transmits it as digital data.

[0340] A "microphone" is a device that captures sound and stores and transmits it as digital data.

[0341] MODE FOR CARRYING OUT THE INVENTION

[0342] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[0343] Importing and analyzing textbook data and past exam data

[0344] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The textbook page captured as an image file is saved in the app.

[0345] Examples:

[0346] The user takes a photo of a textbook page using their smartphone camera and saves it in the app.

[0347] The user uploads past exam questions in PDF format.

[0348] The server receives the textbook data sent from the user's device, converts it into text using OCR technology (such as Google's Tesseract OCR), analyzes special data such as formulas and graphs, and saves it in digital format.

[0349] Examples:

[0350] The server receives textbook images and converts them into text data using Tesseract OCR.

[0351] Special formulas and graphs are analyzed and saved using libraries such as MathJax.

[0352] Question generation and distribution

[0353] The server generates new questions using a generative AI (e.g., OpenAI's GPT-4) based on textbook data and past exam question data. It generates questions by inputting question generation prompts into the AI ​​model.

[0354] Examples:

[0355] The AI ​​is instructed to generate a problem based on the formula for solving a quadratic equation, and the generated problem is saved in text format.

[0356] The user's device requests questions by specifying a specific subject and difficulty level. The user selects the "Mathematics" subject and the "Intermediate" level in the app interface.

[0357] Examples:

[0358] The user selects and requests "Intermediate level math problems."

[0359] The server selects appropriate questions in response to a user's request and delivers them to the user's terminal.

[0360] Examples:

[0361] The server searches for "intermediate level quadratic equation problems" and sends the corresponding problems to the user's terminal.

[0362] Answer and learning trend analysis

[0363] The user answers the questions delivered to the user's terminal and transmits the answer data to the server.

[0364] Examples:

[0365] The user answers the question "Find the solution to y = x^2 + 3x + 2" with "x = -1, -2", and the app collects the answer and sends it to the server.

[0366] The server uses an automatic scoring algorithm to evaluate the submitted answers and determine whether they are correct or not.

[0367] Examples:

[0368] The server receives the user's answer "x = -1, -2" and uses an automatic scoring algorithm to determine that this is the correct answer.

[0369] The server analyzes the user's answer data and identifies areas in which the user is particularly weak.

[0370] Examples:

[0371] The server analyzes the user's answer history and identifies a low rate of correct answers for problems involving quadratic equations.

[0372] Focused questions and feedback on weak areas

[0373] The server generates new questions that are specific to the user's weak areas based on the user's learning tendencies.

[0374] Examples:

[0375] The server inputs a new problem related to "quadratic equations" as a prompt to the generation AI and sends the generated problem to the user's device.

[0376] The user's device will display many questions related to areas in which they are weak, and they can try those questions again.

[0377] Examples:

[0378] Many additional practice problems related to "quadratic equations" will appear on the user's smartphone.

[0379] The server continuously monitors the user's learning progress and provides appropriate feedback.

[0380] Examples:

[0381] The server generates a message such as "Your quadratic equation accuracy rate has improved to 80%."

[0382] The user's device receives feedback from the server and adjusts the learning plan.

[0383] Examples:

[0384] The user reads the feedback message and sets a new goal, such as "strengthen memorization."

[0385] Incorporating an emotion engine

[0386] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and transmits them to the emotion engine.

[0387] Examples:

[0388] While the user is solving the problem, the device captures the user's face with a camera and also records their voice.

[0389] The server analyzes the received data using an emotion engine to identify the user's emotional state.

[0390] Examples:

[0391] The server analyzes the received data and determines whether the user is "concentrated" or "tired."

[0392] The server adjusts the difficulty and quantity of questions based on the recognized user emotions.

[0393] Examples:

[0394] If the user is determined to be "tired," the server will present more slightly easier questions.

[0395] The user's device is presented with adjusted questions, allowing the user to continue learning in a comfortable environment.

[0396] Examples:

[0397] Easy questions are displayed so that users can relax and work through the questions.

[0398] Based on the user's emotional data, the server generates messages encouraging them to take a break when their concentration is low and encouraging them when they are feeling positive.

[0399] Examples:

[0400] The server generates a message such as "You seem to be feeling a little tired. Let's take a 5-minute break" and sends it to the user's device.

[0401] The user's device receives emotion-based feedback and prompts to take breaks, allowing the user to adjust the pace of their learning as needed.

[0402] Examples:

[0403] The user checks the feedback message and takes a few minutes to regain focus.

[0404] Examples of prompt statements

[0405] 1. Upload a PDF file and generate a quadratic equation problem:

[0406] Prompt: "Generate three quadratic equation problems based on Chapter 3 of this PDF file."

[0407] 2. Generate additional questions based on the user's weak areas:

[0408] Prompt: "Based on your answer history, you are having difficulty with problems involving quadratic equations. Please generate another practice problem."

[0409] 3. Generating encouraging messages based on emotion data:

[0410] Prompt: "The user is feeling fatigued. Please generate a message encouraging them to take a break."

[0411] This allows the present invention to provide a tailored learning experience that takes into account the learner's emotional state and learning progress.

[0412] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0413] Step 1: Importing textbook data

[0414] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The app receives image files and PDF files as input and saves the data in the app. Specific operations include the user taking a photo of a textbook page with the smartphone camera and importing the image file into the app.

[0415] Input: Textbook image file or PDF file

[0416] Output: Textbook data saved in the app

[0417] Step 2: Analyzing textbook data

[0418] The server receives textbook data sent from the user's device. It converts the received data into text data using OCR technology (for example, Google's Tesseract OCR), and analyzes special data such as formulas and graphs before saving them in digital format. The input is image data or PDF files sent from the user's device, and the output is text data.

[0419] Input: Image data or PDF file from the device

[0420] Output: Text data stored on the server

[0421] Step 3: Generate the problem

[0422] The server generates new questions using a generative AI (for example, OpenAI's GPT-4) based on textbook data and past exam question data. It receives prompt text and analyzed text data as input and generates new questions based on that. Specifically, the server sends a prompt for a "problem based on the formula for solving a quadratic equation" to the generative AI and saves the generated problem in text format.

[0423] Input: prompt and parsed text data

[0424] Output: Text data of the generated problem

[0425] Step 4: Deliver in response to user requests

[0426] The user's device requests a problem by specifying a specific subject and difficulty level. The server receives the user's request information as input and sends it to the server. The server selects appropriate problems based on the user's request and delivers them to the user's device. As a specific example, the user selects and requests "intermediate level math problems," and the server searches for and sends "intermediate level quadratic equation problems."

[0427] Input: User request information

[0428] Output: The problem delivered to the user's device

[0429] Step 5: Collect and assess answers

[0430] The user answers the questions delivered to the user's device and sends the answer data to the server. The server receives the user's answer data as input and sends it to the server. The server uses an automatic scoring algorithm to evaluate the submitted answer and determine whether it is correct or not. For example, a user searches for the solution to y = x^2 + 3x + 2 and answers "x = -1, -2", and the data is sent to the server.

[0431] Input: User's answer

[0432] Output: Correct / incorrect result

[0433] Step 6: Analyze learning trends

[0434] The server analyzes the user's answer data and identifies areas in which the user is particularly weak. As input, it receives the user's past answer history and analyzes it to identify areas in which the user is weak. Specifically, the server calculates the correct answer rate for questions related to quadratic equations from the user's answer history and identifies areas in which the user is weak.

[0435] Input: User's past answer history

[0436] Output: Identification of user's weak areas

[0437] Step 7: Focused questions and feedback on weak areas

[0438] The server generates new problems specific to the user's weak areas based on the user's learning tendencies. It receives the user's weak areas as input and sends prompts for new problems to the generation AI based on that information. For example, it generates additional problems related to "quadratic equations" and sends them to the user's device.

[0439] Input: User's weak areas information

[0440] Output: New problem delivered to user's device

[0441] The server continuously monitors the user's learning progress and provides appropriate feedback. It receives the user's answer history and learning progress information as input, and generates a feedback message based on that information. For example, it generates a feedback message such as "Your rate of correct answers to quadratic equations has improved to 80%" and sends it to the user's device.

[0442] Input: User's answer history and learning progress information

[0443] Output: Feedback sent to the user's device

[0444] Step 8: Obtaining and analyzing sentiment data

[0445] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and recognize their emotional state. The user's facial expressions and voice data are acquired as input and sent to the server. The server then analyzes the received data using an emotion engine to identify the user's emotional state. Specifically, the server determines the user's emotional state, such as "concentrated" or "tired."

[0446] Input: facial expressions and voice data

[0447] Output: Emotional state judgment result

[0448] Step 9: Emotionally Based Problem Adjustment

[0449] The server adjusts the difficulty and quantity of questions based on the user's recognized emotions. It receives the user's emotional state as input and sends question prompts to the generation AI based on that. For example, if the user is determined to be "tired," the server will present more slightly easier questions.

[0450] Input: User's emotional state

[0451] Output: Adjusted problem

[0452] Step 10: Provide emotional feedback

[0453] The server generates and provides messages encouraging or encouraging a break based on the user's emotional data. It receives the user's emotional data as input and generates messages based on it. For example, it generates a message such as "You seem to be feeling a little tired. Let's take a five-minute break" and sends it to the user's device. The user's device receives the emotional feedback and the prompt to take a break, allowing the user to adjust the pace of their learning as needed.

[0454] Input: User emotion data

[0455] Output: Message to encourage break and adjustment of user behavior

[0456] This allows the present invention to provide a more effective and personalized learning experience by taking into account the learner's emotional state and learning progress.

[0457] (Application example 2)

[0458] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0459] Conventional learning support systems have difficulty considering the user's emotional state, and are therefore unable to provide an optimal learning environment that responds to emotional changes.Furthermore, food delivery services do not offer personalized menu suggestions based on the user's emotions, making it difficult to improve user satisfaction.

[0460] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for recognizing emotions by acquiring facial expressions and voice data, means for adjusting the content of questions and feedback based on the recognized emotions, and means for generating customized menus based on emotions and past data. This allows for optimal learning and service provision that takes user emotions into consideration and responds to changes in those emotions.

[0461] "Textbook data" refers to digital data of text materials used in education.

[0462] "Past exam question data" refers to digital data of questions and answers from exams previously administered.

[0463] The "means for importing" refers to a method or device for acquiring textbook data and past exam question data from a user's terminal to a server.

[0464] An "analyzing means" is a method or device that processes captured data and converts it into an understandable form.

[0465] "Means for converting to text data" refers to technology that extracts text information from image data or PDFs and converts it into text format.

[0466] "Generative AI" is a technology that uses machine learning and natural language processing techniques to automatically generate new questions and text.

[0467] "Means for collecting" refers to a method or device for collecting answer data from users and transmitting it to the server.

[0468] The "means for determining whether a question is correct or incorrect" refers to an algorithm or device that analyzes collected answer data and automatically determines whether it is correct or incorrect.

[0469] "Means for analyzing learning trends" refers to technology that analyzes a user's answer data and identifies their learning patterns and weaknesses.

[0470] The "means for extracting weak areas" refers to a method or device for identifying areas of study that are particularly weak for a user based on an analysis of learning trends.

[0471] The "means of providing focused questions" refers to a method or device that provides questions intensively in the extracted weak areas.

[0472] "Monitoring means" refers to techniques for continuously observing a user's learning progress and collecting data.

[0473] A "means for providing feedback" is a method or device that provides advice or encouragement to the user based on collected data.

[0474] "Means for acquiring facial expression and voice data" refers to technology that uses the user's camera and microphone to capture facial expressions and voice and collect that data.

[0475] "Means for recognizing emotions" refers to technology that analyzes acquired facial expressions and voice data to identify the user's emotions.

[0476] The "means for adjusting the content of questions" refers to a method or device for changing the difficulty or amount of learning content based on the recognized emotions.

[0477] The "means for generating customized menus" is a technology that generates optimal food menus and learning content for users based on emotions and past data.

[0478] System Overview

[0479] This invention combines technologies to improve learning support systems and food delivery services. The system is equipped with an emotion engine that recognizes user emotions, aiming to provide a more personalized learning and service experience. The system of this invention has the ability to input textbook data and past exam question data, analyze them, and generate new questions. It also has the ability to collect and analyze user answer data, identify learning trends, and provide individualized feedback.

[0480] Importing and analyzing textbook data and past exam data

[0481] Users can photograph or upload textbook data and past exam question data using devices such as smartphones. The server converts the received data into text data using OCR technology, and analyzes special data such as formulas and graphs before saving them in digital format. The software used includes Python and OpenCV.

[0482] Question generation and distribution

[0483] The server generates new questions using a generative AI model based on imported textbook data and past exam question data. This generative AI model utilizes natural language processing technology. Users can request questions based on specific subjects and difficulty levels from their devices, and the server selects appropriate questions and delivers them to the user's device.

[0484] Answer and learning trend analysis

[0485] When a user answers a question, the answer data is sent to the server. The server uses an automatic scoring algorithm to judge the answer data and record whether it is correct or incorrect. The server also analyzes the user's answer data to identify areas in which the user is particularly weak. This allows the system to understand the user's learning trends.

[0486] Focused questions and feedback on weak areas

[0487] The server generates new questions based on the user's learning trends, focusing on areas of weakness. It also continuously monitors the user's learning progress and provides appropriate feedback. For example, it generates a message saying, "Your rate of correct answers to quadratic equations has improved to 80%."

[0488] Incorporating an emotion engine

[0489] The user's device captures the user's facial expressions and voice using a built-in camera and microphone and sends them to the server. The server then analyzes the received data with an emotion engine to identify the user's emotional state. For example, it determines whether the user is focused, tired, or stressed. Based on the recognized emotion, the server adjusts the difficulty and quantity of questions and provides feedback.

[0490] Emotion-Based Feedback

[0491] The server uses the user's emotional data to generate messages that encourage a break if the user's concentration is low and encourages the user if the user's emotions are positive. For example, a message such as "You seem to be feeling a little tired. Let's take a five-minute break" is displayed.

[0492] Customized Menu Generation

[0493] Based on emotions and past data, the system provides users with optimal learning content and services. For example, in a food delivery service, a generative AI model suggests individually customized food menus based on emotions and past order data. An example prompt might be, "When your emotions are positive, the recommended menu is chocolate cake."

[0494] In this way, the present invention is a system that takes into account the emotional state of the user and can provide a more personalized learning experience and service.

[0495] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0496] Step 1: Import textbook data and past exam question data

[0497] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[0498] Specific operation: Users use devices such as smartphones or PCs to take photos of textbook pages or exam questions and upload them as PDF files. This data is then sent from the device to the server.

[0499] Output: Educational data files imported to the server

[0500] Step 2: Analyze and convert data to text

[0501] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[0502] How it works: The server uses OCR technology to analyze imported image data and PDFs and convert them into text data. It also uses Python and OpenCV to analyze special data such as formulas and graphs.

[0503] Output: Text data and analyzed special data (formulas, graphs, etc.)

[0504] Step 3: Generate a new problem

[0505] Input: Text data, analyzed past exam question data

[0506] How it works: The server uses a generative AI model to automatically generate new questions. It uses natural language processing technology to generate questions that correspond to specific themes and levels of difficulty.

[0507] Output: The new problem set generated

[0508] Step 4: Question distribution based on user requests

[0509] Input: User request (specific subject, difficulty level)

[0510] Specific operation: A user requests a question by specifying a specific subject and difficulty level using a terminal. The server selects appropriate questions based on the request and delivers them to the user's terminal.

[0511] Output: The problem delivered to the user's device

[0512] Step 5: Collecting answer data and determining whether the answers are correct or incorrect

[0513] Input: User's answer data

[0514] Specific operation: The user solves the problem and sends the answer data from the device to the server. The server analyzes the answer data using an automatic scoring algorithm and determines whether the answer is correct or incorrect.

[0515] Output: Record of correct / incorrect judgment results and answer data

[0516] Step 6: Analyze learning trends and identify weak areas

[0517] Input: User's answer data, correct / incorrect result

[0518] Specific operation: The server analyzes the user's past answer data and identifies areas of weakness, thereby understanding the user's learning trends.

[0519] Output: Analysis results (user's weak areas)

[0520] Step 7: Focus questions and provide feedback

[0521] Input: Analysis results (weak areas), learning progress data

[0522] What it does: The server generates new problems based on the identified weaknesses and delivers them to the user. It also monitors learning progress and provides appropriate feedback, such as a message saying, "Your success rate in solving quadratic equations has improved to 80%."

[0523] Output: Focused questions, feedback messages

[0524] Step 8: Emotion recognition by acquiring facial expressions and voice data

[0525] Input: User's facial expression data, voice data

[0526] How it works: The user's device uses its built-in camera and microphone to capture facial expressions and voice data. This data is then sent from the device to the server, which then uses an emotion engine to identify the user's emotional state.

[0527] Output: Perceived emotional state

[0528] Step 9: Emotional content adjustment and feedback

[0529] Input: Recognized emotional state, learning tendency data

[0530] Specific behavior: The server adjusts the difficulty and quantity of questions based on the recognized emotion. It also generates a message encouraging the user to take a break if the user's concentration is low, and provides an encouraging message if the user's emotion is positive. For example, it generates a message saying, "You seem to be feeling a little tired. Let's take a five-minute break."

[0531] Output: Adjusted questions and feedback messages

[0532] Step 10: Generate a customized menu

[0533] Input: Recognized emotional state, historical data

[0534] Specific operation: Based on emotions and past data, the server uses a generative AI model to provide optimal learning content and services to users. For example, a food delivery service might suggest a menu item such as, "When you're feeling positive, the recommended menu is chocolate cake."

[0535] Output: Customized learning content or food menu suggestions

[0536] In this way, the specific actions and inputs / outputs at each step are clearly defined, optimizing the user learning experience and service delivery.

[0537] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0538] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0539] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0540] [Second embodiment]

[0541] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0542] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0543] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0544] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0545] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0546] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0547] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0548] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0549] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0550] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0551] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0552] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0553] System Overview

[0554] This invention is a learning support system for exam preparation. This system imports textbook data and past exam question data, analyzes that data, converts it into text format, and automatically generates new questions using a generation AI. Users answer the questions, and the answer data is collected and analyzed on a server, which identifies the user's weak areas and presents focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback.

[0555] Importing and analyzing textbook data and past exam data

[0556] Importing textbook data

[0557] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can upload "2023 Mathematics Past Questions" to the app.

[0558] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[0559] Question generation and distribution

[0560] Problem Generation

[0561] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[0562] Delivery in response to user requests

[0563] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[0564] Server: In response to a user request, selects appropriate problems and delivers them to the user's device. For example, it selects intermediate level math problems and sends them to the user's device.

[0565] Answer and learning trend analysis

[0566] Collecting and assessing answers

[0567] User's device: The user answers the delivered questions and sends the answer data to the server. For example, the user answers "x = -1, -2" to the question "Find the solution to y = x^2 + 3x + 2."

[0568] Server: The submitted answers are evaluated by an automatic scoring algorithm to determine whether they are correct or incorrect. The analysis results are stored in a database.

[0569] Analysis of learning trends

[0570] Server: Analyzes the user's answer data and identifies areas in which the user has particular difficulty. For example, it may be discovered that the user has difficulty with quadratic equations.

[0571] Focused questions and feedback on weak areas

[0572] Focused questions on weak areas

[0573] Server: Based on the user's learning tendencies, it generates new problems that are specific to weak areas. For example, it generates a large number of problems related to "quadratic equations."

[0574] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[0575] Providing Feedback

[0576] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0577] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[0578] Case Study

[0579] For example, if a user has difficulty with quadratic equations in mathematics, the system will analyze the user's answer data, generate and prioritize problems related to quadratic equations, and monitor the user's progress, providing feedback on how to improve their weaknesses. In this way, users can efficiently overcome their weaknesses and maximize their learning outcomes.

[0580] In the above-described form, the present invention provides a system that supports efficient learning for learners and enables them to effectively prepare for exams.

[0581] The processing flow will be explained below.

[0582] Step 1:

[0583] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0584] Step 2:

[0585] Server: Receives the captured data and converts it into text data using OCR (Optical Character Recognition) technology, for example, analyzing text and mathematical formulas in images and converting them into digital text.

[0586] Step 3:

[0587] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[0588] Step 4:

[0589] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[0590] Step 5:

[0591] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[0592] Step 6:

[0593] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0594] Step 7:

[0595] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[0596] Step 8:

[0597] User's device: Sends the answer to the server.

[0598] Step 9:

[0599] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[0600] Step 10:

[0601] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[0602] Step 11:

[0603] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[0604] Step 12:

[0605] On the user's device: Questions related to weak areas are highlighted, and the user is encouraged to retry those questions.

[0606] Step 13:

[0607] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0608] Step 14:

[0609] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[0610] Example 1

[0611] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] Conventional learning support systems have struggled to efficiently analyze users' learning progress and weak areas and provide appropriate questions. They also lack the ability to accurately analyze special data such as mathematical formulas and graphs to generate and deliver questions tailored to the subjects and level of difficulty desired by users. Furthermore, they lacked a means to effectively collect users' answer data and provide continuous feedback.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0614] In this invention, the server includes means for importing textbook data and past exam question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions that are focused on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for the user to request questions by specifying a specific subject or level of difficulty, and means for selecting and delivering appropriate questions in response to the user's request. This makes it possible to provide appropriate questions in accordance with the user's learning tendencies and to provide learning support through continuous feedback.

[0615] "Textbook data" refers to data that includes text information of learning materials used in learning.

[0616] "Past exam question data" refers to data that records the content of questions from exams that have been administered in the past.

[0617] "Analysis" is the process of deciphering captured data and converting it into text data or digital format.

[0618] "Text data" is data that represents character information in digital form.

[0619] "Generative AI" is an AI technology for automatically generating new problems.

[0620] "Means for generating questions" refers to technology that creates new study questions based on textbook data and past exam question data.

[0621] "User answer data" is data that includes the answers to questions answered by the user.

[0622] The "means for determining whether an answer is correct or incorrect" is a technique for determining whether an answer is correct or incorrect based on the user's answer data.

[0623] "Study trends" is information on learning progress and weak areas analyzed based on the user's question answer patterns and grades.

[0624] "Weak areas" refer to areas of study or types of questions that the user is particularly weak at.

[0625] "Feedback" is information about improvements and advice provided based on the user's learning progress.

[0626] "Means for requesting" is a function that allows the user to request questions by specifying a specific subject or level of difficulty.

[0627] "Distribution means" refers to the technology for transmitting generated questions and feedback to the user's terminal.

[0628] MODE FOR CARRYING OUT THE INVENTION

[0629] The present invention relates to a learning support system for exam preparation. This system imports textbook data and past exam question data uploaded by users, analyzes the data, converts it into text format, and generates new questions. The generated questions are distributed upon user request, allowing the user to answer them. The user's answer data is collected, and correct answers are determined and their learning trends are analyzed. The system then presents questions that focus on the user's weak areas, and continuously monitors the user's learning progress to provide appropriate feedback. Specific embodiments are described below.

[0630] Importing and analyzing textbook data and past exam data

[0631] User device: The user uses the application to upload textbook and past exam question data. For example, by selecting a PDF file of "2023 Mathematics Past Questions" and pressing the upload button, the data is sent to the server.

[0632] Server: The server receives the textbook data uploaded from the user's device. The received data is converted into text data using OCR (Optical Character Recognition) technology. During this process, special data such as mathematical formulas and graphs are also recognized and saved as text or figures.

[0633] Question generation and distribution

[0634] Server: The server generates new questions using a generative AI (artificial intelligence) model based on the analyzed textbook data. For example, the generative AI model generates a question asking about the solution to a quadratic equation.

[0635] On the user's device: The user specifies the subject and difficulty level they want to study through the application. For example, they may request "Intermediate level math problems."

[0636] Server: The server receives the user's request, selects appropriate problems from the generated questions, and delivers them to the user's device. For example, a set of intermediate level math problems is sent to the user's device.

[0637] Collecting and assessing answer data

[0638] User's device: The user answers the received question and sends the answer data to the server via the application. For example, in response to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the user answers "x = 2, x = 3."

[0639] Server: The server analyzes the received answer data using an automatic scoring algorithm and determines whether the answer is correct. If the answer is correct, the server presents the next question, and if the answer is incorrect, it displays an explanation.

[0640] Learning trend analysis and feedback

[0641] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendency. For example, it discovers that the user's average correct answer rate for quadratic equation problems is 50%.

[0642] Server: The server generates feedback for the user based on the analysis results, for example, a message saying "You need to practice more to improve your accuracy on quadratic equation problems."

[0643] User's device: The user's device receives and displays the feedback sent from the server, for example, a notification saying, "Try some extra practice to improve your accuracy on quadratic equation problems."

[0644] Focused questions on weak areas

[0645] Server: Based on the analysis results, the server generates new problems that are specific to the user's weak areas. For example, it generates a large number of problems related to "quadratic equations."

[0646] On the user's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed.

[0647] Examples of concrete examples and prompts

[0648] As a concrete example, consider the case where a user has difficulty with quadratic equations in mathematics. The user uploads "2023 Math Past Papers" to the app, which is then received and analyzed by the server. Then, a generation AI is used to generate "quadratic equation problems" and deliver them as the "intermediate level math problems" requested by the user.

[0649] An example prompt is:

[0650] "Generate new quadratic equation problems from past math questions for 2023. A user has requested intermediate level math problems."

[0651] This allows the system to provide an environment in which users can study efficiently.

[0652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0653] Program processing flow

[0654] Step 1:

[0655] User's device: The user uses the learning support system's application to upload textbook and past exam question data. For example, they select a PDF file of "2023 Mathematics Past Questions" and press the upload button. This input data (textbook data) is sent to the server.

[0656] Step 2:

[0657] Server: The server receives textbook data uploaded from the user's device. This received data is converted into text data using OCR (optical character recognition) technology. Specifically, it analyzes images and text within the PDF and converts them into text format. Special data such as mathematical formulas and graphs are also converted into digital format as text or figures and saved. The output data is text data.

[0658] Step 3:

[0659] Server: The server uses a generative AI model to generate new problems based on the analyzed text data. For example, the generative AI model generates a "problem asking about the solution to a quadratic equation." Specifically, it takes as input a prompt statement that specifies the difficulty and content of the problem to be generated, and outputs the generated new problem data. The output data is the generated problem set.

[0660] Step 4:

[0661] User's device: The user requests questions through the application, specifying the subject and level of difficulty they want to study. For example, they might request "Please give me intermediate level math questions." The input data is the user's request.

[0662] Step 5:

[0663] Server: The server receives the user's request, selects the appropriate problem from the generated problem set, and delivers it to the Yuga terminal. Specifically, it searches the problem database based on the request content, selects the appropriate problem, and delivers it. The output data is the result of delivering the problem to the user terminal.

[0664] Step 6:

[0665] User's device: The user answers the delivered questions and sends the answer data to the server through the application. For example, to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the answer is "x = 2, x = 3." The input data is the user's answer.

[0666] Step 7:

[0667] Server: The server uses an automatic scoring algorithm to evaluate the received answer data and confirm whether it is correct or not. Specifically, it compares the answer data with a correct answer database and determines whether it is correct. The output data is the result of the evaluation.

[0668] Step 8:

[0669] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendencies. Specifically, it analyzes data such as the correct answer rate and the time it takes to answer, and identifies specific weak areas. The output data is the learning tendency analysis results.

[0670] Step 9:

[0671] Server: The server generates feedback for the user based on the analysis results. For example, it creates a message saying, "You need to practice more to improve your accuracy in solving quadratic equation problems." Specifically, it takes the learning trend analysis results as input, generates a feedback message, and sends it. The output data is the feedback message.

[0672] Step 10:

[0673] User's device: The user's device receives and displays the feedback sent from the server. For example, a notification such as "Please practice more to improve your accuracy on quadratic equation problems" is displayed. The input data is the feedback message.

[0674] Step 11:

[0675] Server: Based on the analysis results, the server generates new questions that are specific to the user's weak areas. Specifically, it uses information about the weak areas as input and provides prompts to the generative AI model to create new questions. The output data is the newly generated questions.

[0676] Step 12:

[0677] User's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed. The input data is the newly generated problem.

[0678] (Application example 1)

[0679] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0680] Conventional learning support systems limit students to a passive learning experience where they simply solve problems, making it difficult to improve learning efficiency. Furthermore, they often lack the ability to select questions tailored to specific subjects or levels of difficulty, or to analyze learning trends based on users' answer data, making it difficult to provide effective feedback or prioritized questions. Furthermore, they lack real-time learning progress monitoring or learning support utilizing virtual spaces, making it difficult to maintain users' motivation to learn.

[0681] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0682] In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether the answers are correct, means for analyzing the user's learning tendencies and identifying specific areas of weakness, means for providing focused questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, and means for supporting learning in a virtual space using a head-mounted display or smart glasses. This allows users to learn interactively in a virtual space, breaking away from passive learning and significantly improving learning efficiency and motivation.

[0683] "Textbook data" refers to digital data of learning materials and textbooks used in learning.

[0684] "Past exam question data" refers to digitized data of exam questions that have been administered in the past.

[0685] "Capturing means" refers to the method or technology by which the subject data is entered or uploaded into the system.

[0686] "Means for analyzing and converting to text data" refers to techniques or methods for analyzing the captured data and converting it into a readable text format.

[0687] "Generative AI" refers to systems or algorithms that use artificial intelligence techniques to generate new data and information.

[0688] "Means of generating questions" refers to techniques and methods for creating new questions based on textbook data and past exam question data.

[0689] "User answer data" refers to data of the answers entered by the user in response to the questions posed.

[0690] The "means for determining whether an answer is correct" refers to a technique or method for analyzing the user's answer data and determining whether the answer is correct or incorrect.

[0691] "Means for analyzing learning trends" refers to techniques and methods for analyzing a user's learning patterns and trends based on the user's answer data.

[0692] "Means for extracting weak areas" refers to techniques or methods for analyzing a user's learning data and identifying areas in which the user is particularly weak.

[0693] A "method of providing focused questions" is a technique or method of providing focused questions based on the user's areas of weakness.

[0694] "Means for continuously monitoring learning progress" refers to techniques and methods for constantly monitoring a user's learning status and tracking progress.

[0695] "Means for providing feedback" refers to techniques and methods for providing advice and guidance according to the user's learning situation.

[0696] A "head-mounted display" is a display device that displays a virtual space when worn by a user on the head.

[0697] "Smart glasses" are glasses-type devices that can display augmented reality and virtual reality.

[0698] "Means to support learning in virtual spaces" are technologies and methods that use virtual reality or augmented reality technology to help users learn in virtual spaces.

[0699] This invention provides a system that allows users to study effectively in a virtual space. This system imports textbook data and past exam question data, converts the data into text format, and generates new questions using a generative artificial intelligence (generative AI model). It also collects and analyzes user answer data, extracts specific areas of weakness, and provides focused questions and feedback.

[0700] Importing and analyzing textbook data and past exam data

[0701] The system receives textbook data and past exam question data from users' devices. Users can take photos of this data or upload it as a PDF or Doc file. This data is received by the server and converted into text data using OCR technology (e.g., Tesseract). Special data (such as mathematical formulas and graphs) is also analyzed and saved in digital format.

[0702] Question generation and distribution

[0703] The server generates new questions using a generative AI model (e.g., OpenAI GPT-4) based on textbook data and past exam question data. The generated questions are delivered in response to user requests. The user specifies a specific subject and level of difficulty, and the server selects appropriate questions based on that request and delivers them to the user's device.

[0704] Answer and learning trend analysis

[0705] Users answer the questions and send the answers to the server. The server then uses an automated scoring algorithm (e.g., Python and Scikit-Learn) to evaluate the answers and assess whether they are correct. The analysis results are stored in a database, and specific areas of weakness are identified by analyzing the user's learning patterns.

[0706] Learning support in virtual space

[0707] Users use a head-mounted display (HMD) or smart glasses to enter a virtual space and study. Within the virtual space, textbook data and generated questions are displayed as 3D objects, which users can manipulate interactively. In the virtual classroom, users answer questions using a virtual pen in their hand. Answer data is collected and analyzed in real time, and a virtual assistant provides appropriate feedback.

[0708] Examples and prompts

[0709] For example, if a user is learning quadratic equations in a virtual space, the learning progresses through the following steps.

[0710] Example: Quadratic equation problem: "Find the solution to y = x^2 + 3x + 2"

[0711] User's solution: The user writes "x = -1, -2" on the whiteboard in the virtual space with a virtual pen.

[0712] feedback:

[0713] A virtual assistant will appear and provide feedback, saying, "That's right! Try the next one," and then a more difficult question will be displayed.

[0714] Example prompt sentence:

[0715] A user is solving a quadratic equation problem in a virtual classroom. The answer is "x = -1, -2." Please provide appropriate feedback for this correct answer.

[0716] In this way, users can learn interactively in a virtual space, moving away from passive learning and significantly improving their learning efficiency and motivation.

[0717] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0718] Step 1:

[0719] Importing textbook data and past exam question data

[0720] User's device: Users can take photos of textbook data and past exam question data or upload them as PDF or Doc files.

[0721] Input: Image data, PDF files, Doc files

[0722] Output: Raw data sent to the server

[0723] What happens: A user opens the application, selects a textbook or exam question file, and clicks the upload button. The file is then sent to the server.

[0724] Step 2:

[0725] Data analysis and conversion to text data

[0726] Server: Converts received data into text using OCR technology (such as Tesseract). Analyzes special data such as mathematical formulas and graphs and saves them in digital format.

[0727] Input: Raw data (image, PDF, Doc)

[0728] Output: Text data, special data in digital format

[0729] How it works: The server analyzes the received files and uses OCR technology to convert the contents of textbooks and exam questions into text data. It also recognizes mathematical formulas and graphs and saves them as digital data.

[0730] Step 3:

[0731] Problem Generation

[0732] Server: Generates new questions using a generative AI model (such as OpenAI GPT-4) based on textbook data and past exam question data.

[0733] Input: Text data, past exam question data

[0734] Output: The newly generated problem data

[0735] How it works: The server analyzes the text data and provides it as input to the generative AI model, which then generates new questions and stores them in a database.

[0736] Step 4:

[0737] Problem delivery based on user requests

[0738] Server: Accepts user requests for questions based on specific subjects and difficulty levels, selects appropriate questions based on those requests, and delivers them to the user's device.

[0739] Input: User's problem request (subject, difficulty level)

[0740] Output: Selected question data, delivered to the terminal

[0741] Specific operation: The user selects the subject and difficulty level on the app and requests questions. The server receives the request, selects appropriate questions, and delivers them to the user's device.

[0742] Step 5:

[0743] Collecting answer data and determining correctness

[0744] User's device: The user answers the questions delivered and sends the answer data to the server.

[0745] Server: Determines whether the answer is correct or incorrect based on an algorithm and stores the analysis results in a database.

[0746] Input: User's answer data

[0747] Output: Correct / incorrect result, saved in database

[0748] Specific operation: The user answers questions in the virtual space, inputs the answers, and sends them to the server. The server analyzes the answers, determines whether they are correct, and saves the results.

[0749] Step 6:

[0750] Analysis of learning trends and identification of weak areas

[0751] Server: Analyzes user answer data, identifies learning trends, and identifies specific areas of weakness.

[0752] Input: Correct / incorrect results, past answer data

[0753] Output: User learning tendency data, weak area data

[0754] Specific operation: The server runs an algorithm that analyzes learning trends based on the user's answer data stored in the database and extracts specific areas of weakness.

[0755] Step 7:

[0756] Focused questions and feedback

[0757] Server: Generates new questions based on the extracted weak areas and presents them to the user in a focused manner. In addition, it monitors the learning progress and provides appropriate feedback.

[0758] Input: Weakness Area Data

[0759] Output: New problem data, feedback message

[0760] Specific operation: The server generates new questions related to the weak areas and delivers them to the user's device. It also generates and provides feedback messages to the user based on their learning progress.

[0761] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0762] System Overview

[0763] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[0764] Importing and analyzing textbook data and past exam data

[0765] Importing textbook data

[0766] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0767] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[0768] Question generation and distribution

[0769] Problem Generation

[0770] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[0771] Delivery in response to user requests

[0772] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[0773] Server: In response to a user request, selects an appropriate problem and delivers it to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0774] Answer and learning trend analysis

[0775] Collecting and assessing answers

[0776] User's device: The user answers the delivered problem and sends the answer data to the server. For example, the user answers the problem "Find the solution to y = x^2 + 3x + 2" as "x = -1, -2".

[0777] Server: The submitted answer is evaluated by an automatic scoring algorithm to determine whether it is correct or not. For example, it determines that the user's answer "x = -1, -2" is correct.

[0778] Analysis of learning trends

[0779] Server: Analyzes the user's answer data and identifies areas of particular difficulty. For example, analyzes how well the user answers quadratic equation problems.

[0780] Focused questions and feedback on weak areas

[0781] Focused questions on weak areas

[0782] Server: Based on the user's learning tendencies, generate new problems that are specific to weak areas. For example, generate additional problems related to "quadratic equations."

[0783] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[0784] Providing Feedback

[0785] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0786] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[0787] Incorporating an emotion engine

[0788] emotion recognition

[0789] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[0790] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[0791] Emotion-based problem adjustment

[0792] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[0793] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[0794] Providing emotion-based feedback

[0795] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[0796] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[0797] In the above-described manner, the present invention constructs a system that takes into account the emotional state of the learner and can provide a more personalized learning experience.

[0798] The processing flow will be explained below.

[0799] Step 1:

[0800] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[0801] Step 2:

[0802] Server: Receives the captured data. The received data is converted into text data using OCR technology. Special data such as formulas and graphs are also analyzed and saved in digital format.

[0803] Step 3:

[0804] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[0805] Step 4:

[0806] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[0807] Step 5:

[0808] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[0809] Step 6:

[0810] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[0811] Step 7:

[0812] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[0813] Step 8:

[0814] User's device: Sends the answer to the server.

[0815] Step 9:

[0816] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[0817] Step 10:

[0818] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[0819] Step 11:

[0820] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[0821] Step 12:

[0822] On the user's device: Questions related to weak areas are displayed with emphasis, and the user is encouraged to retry those questions.

[0823] Step 13:

[0824] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[0825] Step 14:

[0826] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[0827] Step 15:

[0828] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[0829] Step 16:

[0830] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[0831] Step 17:

[0832] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[0833] Step 18:

[0834] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[0835] Step 19:

[0836] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[0837] Step 20:

[0838] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[0839] Example 2

[0840] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] Conventional learning support systems provide questions based on the user's learning progress and weak areas, but they lack the flexibility to consider the user's emotional state. This can lead to a decrease in the user's concentration and stress, potentially reducing the effectiveness of their learning. Furthermore, their ability to provide questions based on the user's specific requests is limited, leaving a need for more effective learning support.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0843] In this invention, the server includes a means for importing textbook data and past exam question data, a means for analyzing the imported data and converting it into text data, a means for generating new questions using generative artificial intelligence, a means for collecting user answer data and determining whether the answers are correct, a means for analyzing the user's learning tendencies and identifying specific areas of weakness, a means for providing questions focused on the areas of weakness, a means for continuously monitoring the user's learning progress and providing feedback, and a means for recognizing the user's emotional state and reflecting it in learning activities. This enables personalized question provision and feedback that takes the user's emotional state into account, improving learning effectiveness. Furthermore, questions of specific subjects and levels of difficulty can be provided in response to the user's requests, supporting efficient learning.

[0844] "Textbook data" refers to digital data such as text information, mathematical formulas, and graphs contained in textbooks.

[0845] "Past exam question data" refers to information on exam questions that have been asked in the past, stored in digital format.

[0846] "Generative AI" is an AI technology that has the ability to generate new information and solutions based on input data.

[0847] "User answer data" refers to answer information entered by the user in response to the questions provided.

[0848] "Learning trends" refers to data that indicates specific patterns or progress observed in the user's learning process.

[0849] "Weakness areas" are areas of learning where the user does not fully understand or is less accurate in their answers.

[0850] "Feedback" refers to messages of encouragement and suggestions for improvement provided based on the user's learning progress and answer results.

[0851] "Emotional state" refers to the user's current mental and psychological state, such as whether they are focused, tired, or stressed.

[0852] A "camera" is a device that optically captures information and stores and transmits it as digital data.

[0853] A "microphone" is a device that captures sound and stores and transmits it as digital data.

[0854] MODE FOR CARRYING OUT THE INVENTION

[0855] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[0856] Importing and analyzing textbook data and past exam data

[0857] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The textbook page captured as an image file is saved in the app.

[0858] Examples:

[0859] The user takes a photo of a textbook page using their smartphone camera and saves it in the app.

[0860] The user uploads past exam questions in PDF format.

[0861] The server receives the textbook data sent from the user's device, converts it into text using OCR technology (such as Google's Tesseract OCR), analyzes special data such as formulas and graphs, and saves it in digital format.

[0862] Examples:

[0863] The server receives textbook images and converts them into text data using Tesseract OCR.

[0864] Special formulas and graphs are analyzed and saved using libraries such as MathJax.

[0865] Question generation and distribution

[0866] The server generates new questions using a generative AI (e.g., OpenAI's GPT-4) based on textbook data and past exam question data. It generates questions by inputting question generation prompts into the AI ​​model.

[0867] Examples:

[0868] The AI ​​is instructed to generate a problem based on the formula for solving a quadratic equation, and the generated problem is saved in text format.

[0869] The user's device requests questions by specifying a specific subject and difficulty level. The user selects the "Mathematics" subject and the "Intermediate" level in the app interface.

[0870] Examples:

[0871] The user selects and requests "Intermediate level math problems."

[0872] The server selects appropriate questions in response to a user's request and delivers them to the user's terminal.

[0873] Examples:

[0874] The server searches for "intermediate level quadratic equation problems" and sends the corresponding problems to the user's terminal.

[0875] Answer and learning trend analysis

[0876] The user answers the questions delivered to the user's terminal and transmits the answer data to the server.

[0877] Examples:

[0878] The user answers the question "Find the solution to y = x^2 + 3x + 2" with "x = -1, -2", and the app collects the answer and sends it to the server.

[0879] The server uses an automatic scoring algorithm to evaluate the submitted answers and determine whether they are correct or not.

[0880] Examples:

[0881] The server receives the user's answer "x = -1, -2" and uses an automatic scoring algorithm to determine that this is the correct answer.

[0882] The server analyzes the user's answer data and identifies areas in which the user is particularly weak.

[0883] Examples:

[0884] The server analyzes the user's answer history and identifies a low rate of correct answers for problems involving quadratic equations.

[0885] Focused questions and feedback on weak areas

[0886] The server generates new questions that are specific to the user's weak areas based on the user's learning tendencies.

[0887] Examples:

[0888] The server inputs a new problem related to "quadratic equations" as a prompt to the generation AI and sends the generated problem to the user's device.

[0889] The user's device will display many questions related to areas in which they are weak, and they can try those questions again.

[0890] Examples:

[0891] Many additional practice problems related to "quadratic equations" will appear on the user's smartphone.

[0892] The server continuously monitors the user's learning progress and provides appropriate feedback.

[0893] Examples:

[0894] The server generates a message such as "Your quadratic equation accuracy rate has improved to 80%."

[0895] The user's device receives feedback from the server and adjusts the learning plan.

[0896] Examples:

[0897] The user reads the feedback message and sets a new goal, such as "strengthen memorization."

[0898] Incorporating an emotion engine

[0899] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and transmits them to the emotion engine.

[0900] Examples:

[0901] While the user is solving the problem, the device captures the user's face with a camera and also records their voice.

[0902] The server analyzes the received data using an emotion engine to identify the user's emotional state.

[0903] Examples:

[0904] The server analyzes the received data and determines whether the user is "concentrated" or "tired."

[0905] The server adjusts the difficulty and quantity of questions based on the recognized user emotions.

[0906] Examples:

[0907] If the user is determined to be "tired," the server will present more slightly easier questions.

[0908] The user's device is presented with adjusted questions, allowing the user to continue learning in a comfortable environment.

[0909] Examples:

[0910] Easy questions are displayed so that users can relax and work through the questions.

[0911] Based on the user's emotional data, the server generates messages encouraging them to take a break when their concentration is low and encouraging them when they are feeling positive.

[0912] Examples:

[0913] The server generates a message such as "You seem to be feeling a little tired. Let's take a 5-minute break" and sends it to the user's device.

[0914] The user's device receives emotion-based feedback and prompts to take breaks, allowing the user to adjust the pace of their learning as needed.

[0915] Examples:

[0916] The user checks the feedback message and takes a few minutes to regain focus.

[0917] Examples of prompt statements

[0918] 1. Upload a PDF file and generate a quadratic equation problem:

[0919] Prompt: "Generate three quadratic equation problems based on Chapter 3 of this PDF file."

[0920] 2. Generate additional questions based on the user's weak areas:

[0921] Prompt: "Based on your answer history, you are having difficulty with problems involving quadratic equations. Please generate another practice problem."

[0922] 3. Generating encouraging messages based on emotion data:

[0923] Prompt: "The user is feeling fatigued. Please generate a message encouraging them to take a break."

[0924] This allows the present invention to provide a tailored learning experience that takes into account the learner's emotional state and learning progress.

[0925] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0926] Step 1: Importing textbook data

[0927] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The app receives image files and PDF files as input and saves the data in the app. Specific operations include the user taking a photo of a textbook page with the smartphone camera and importing the image file into the app.

[0928] Input: Textbook image file or PDF file

[0929] Output: Textbook data saved in the app

[0930] Step 2: Analyzing textbook data

[0931] The server receives textbook data sent from the user's device. It converts the received data into text data using OCR technology (for example, Google's Tesseract OCR), and analyzes special data such as formulas and graphs before saving them in digital format. The input is image data or PDF files sent from the user's device, and the output is text data.

[0932] Input: Image data or PDF file from the device

[0933] Output: Text data stored on the server

[0934] Step 3: Generate the problem

[0935] The server generates new questions using a generative AI (for example, OpenAI's GPT-4) based on textbook data and past exam question data. It receives prompt text and analyzed text data as input and generates new questions based on that. Specifically, the server sends a prompt for a "problem based on the formula for solving a quadratic equation" to the generative AI and saves the generated problem in text format.

[0936] Input: prompt and parsed text data

[0937] Output: Text data of the generated problem

[0938] Step 4: Deliver in response to user requests

[0939] The user's device requests a problem by specifying a specific subject and difficulty level. The server receives the user's request information as input and sends it to the server. The server selects appropriate problems based on the user's request and delivers them to the user's device. As a specific example, the user selects and requests "intermediate level math problems," and the server searches for and sends "intermediate level quadratic equation problems."

[0940] Input: User request information

[0941] Output: The problem delivered to the user's device

[0942] Step 5: Collect and assess answers

[0943] The user answers the questions delivered to the user's device and sends the answer data to the server. The server receives the user's answer data as input and sends it to the server. The server uses an automatic scoring algorithm to evaluate the submitted answer and determine whether it is correct or not. For example, a user searches for the solution to y = x^2 + 3x + 2 and answers "x = -1, -2", and the data is sent to the server.

[0944] Input: User's answer

[0945] Output: Correct / incorrect result

[0946] Step 6: Analyze learning trends

[0947] The server analyzes the user's answer data and identifies areas in which the user is particularly weak. As input, it receives the user's past answer history and analyzes it to identify areas in which the user is weak. Specifically, the server calculates the correct answer rate for questions related to quadratic equations from the user's answer history and identifies areas in which the user is weak.

[0948] Input: User's past answer history

[0949] Output: Identification of user's weak areas

[0950] Step 7: Focused questions and feedback on weak areas

[0951] The server generates new problems specific to the user's weak areas based on the user's learning tendencies. It receives the user's weak areas as input and sends prompts for new problems to the generation AI based on that information. For example, it generates additional problems related to "quadratic equations" and sends them to the user's device.

[0952] Input: User's weak areas information

[0953] Output: New problem delivered to user's device

[0954] The server continuously monitors the user's learning progress and provides appropriate feedback. It receives the user's answer history and learning progress information as input, and generates a feedback message based on that information. For example, it generates a feedback message such as "Your rate of correct answers to quadratic equations has improved to 80%" and sends it to the user's device.

[0955] Input: User's answer history and learning progress information

[0956] Output: Feedback sent to the user's device

[0957] Step 8: Obtaining and analyzing sentiment data

[0958] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and recognize their emotional state. The user's facial expressions and voice data are acquired as input and sent to the server. The server then analyzes the received data using an emotion engine to identify the user's emotional state. Specifically, the server determines the user's emotional state, such as "concentrated" or "tired."

[0959] Input: facial expressions and voice data

[0960] Output: Emotional state judgment result

[0961] Step 9: Emotionally Based Problem Adjustment

[0962] The server adjusts the difficulty and quantity of questions based on the user's recognized emotions. It receives the user's emotional state as input and sends question prompts to the generation AI based on that. For example, if the user is determined to be "tired," the server will present more slightly easier questions.

[0963] Input: User's emotional state

[0964] Output: Adjusted problem

[0965] Step 10: Provide emotional feedback

[0966] The server generates and provides messages encouraging or encouraging a break based on the user's emotional data. It receives the user's emotional data as input and generates messages based on it. For example, it generates a message such as "You seem to be feeling a little tired. Let's take a five-minute break" and sends it to the user's device. The user's device receives the emotional feedback and the prompt to take a break, allowing the user to adjust the pace of their learning as needed.

[0967] Input: User emotion data

[0968] Output: Message to encourage break and adjustment of user behavior

[0969] This allows the present invention to provide a more effective and personalized learning experience by taking into account the learner's emotional state and learning progress.

[0970] (Application example 2)

[0971] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0972] Conventional learning support systems have difficulty considering the user's emotional state, and are therefore unable to provide an optimal learning environment that responds to emotional changes.Furthermore, food delivery services do not offer personalized menu suggestions based on the user's emotions, making it difficult to improve user satisfaction.

[0973] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for recognizing emotions by acquiring facial expressions and voice data, means for adjusting the content of questions and feedback based on the recognized emotions, and means for generating customized menus based on emotions and past data. This allows for optimal learning and service provision that takes user emotions into consideration and responds to changes in those emotions.

[0974] "Textbook data" refers to digital data of text materials used in education.

[0975] "Past exam question data" refers to digital data of questions and answers from exams previously administered.

[0976] The "means for importing" refers to a method or device for acquiring textbook data and past exam question data from a user's terminal to a server.

[0977] An "analyzing means" is a method or device that processes captured data and converts it into an understandable form.

[0978] "Means for converting to text data" refers to technology that extracts text information from image data or PDFs and converts it into text format.

[0979] "Generative AI" is a technology that uses machine learning and natural language processing techniques to automatically generate new questions and text.

[0980] "Means for collecting" refers to a method or device for collecting answer data from users and transmitting it to the server.

[0981] The "means for determining whether a question is correct or incorrect" refers to an algorithm or device that analyzes collected answer data and automatically determines whether it is correct or incorrect.

[0982] "Means for analyzing learning trends" refers to technology that analyzes a user's answer data and identifies their learning patterns and weaknesses.

[0983] The "means for extracting weak areas" refers to a method or device for identifying areas of study that are particularly weak for a user based on an analysis of learning trends.

[0984] The "means of providing focused questions" refers to a method or device that provides questions intensively in the extracted weak areas.

[0985] "Monitoring means" refers to techniques for continuously observing a user's learning progress and collecting data.

[0986] A "means for providing feedback" is a method or device that provides advice or encouragement to the user based on collected data.

[0987] "Means for acquiring facial expression and voice data" refers to technology that uses the user's camera and microphone to capture facial expressions and voice and collect that data.

[0988] "Means for recognizing emotions" refers to technology that analyzes acquired facial expressions and voice data to identify the user's emotions.

[0989] The "means for adjusting the content of questions" refers to a method or device for changing the difficulty or amount of learning content based on the recognized emotions.

[0990] The "means for generating customized menus" is a technology that generates optimal food menus and learning content for users based on emotions and past data.

[0991] System Overview

[0992] This invention combines technologies to improve learning support systems and food delivery services. The system is equipped with an emotion engine that recognizes user emotions, aiming to provide a more personalized learning and service experience. The system of this invention has the ability to input textbook data and past exam question data, analyze them, and generate new questions. It also has the ability to collect and analyze user answer data, identify learning trends, and provide individualized feedback.

[0993] Importing and analyzing textbook data and past exam data

[0994] Users can photograph or upload textbook data and past exam question data using devices such as smartphones. The server converts the received data into text data using OCR technology, and analyzes special data such as formulas and graphs before saving them in digital format. The software used includes Python and OpenCV.

[0995] Question generation and distribution

[0996] The server generates new questions using a generative AI model based on imported textbook data and past exam question data. This generative AI model utilizes natural language processing technology. Users can request questions based on specific subjects and difficulty levels from their devices, and the server selects appropriate questions and delivers them to the user's device.

[0997] Answer and learning trend analysis

[0998] When a user answers a question, the answer data is sent to the server. The server uses an automatic scoring algorithm to judge the answer data and record whether it is correct or incorrect. The server also analyzes the user's answer data to identify areas in which the user is particularly weak. This allows the system to understand the user's learning trends.

[0999] Focused questions and feedback on weak areas

[1000] The server generates new questions based on the user's learning trends, focusing on areas of weakness. It also continuously monitors the user's learning progress and provides appropriate feedback. For example, it generates a message saying, "Your rate of correct answers to quadratic equations has improved to 80%."

[1001] Incorporating an emotion engine

[1002] The user's device captures the user's facial expressions and voice using a built-in camera and microphone and sends them to the server. The server then analyzes the received data with an emotion engine to identify the user's emotional state. For example, it determines whether the user is focused, tired, or stressed. Based on the recognized emotion, the server adjusts the difficulty and quantity of questions and provides feedback.

[1003] Emotion-Based Feedback

[1004] The server uses the user's emotional data to generate messages that encourage a break if the user's concentration is low and encourages the user if the user's emotions are positive. For example, a message such as "You seem to be feeling a little tired. Let's take a five-minute break" is displayed.

[1005] Customized Menu Generation

[1006] Based on emotions and past data, the system provides users with optimal learning content and services. For example, in a food delivery service, a generative AI model suggests individually customized food menus based on emotions and past order data. An example prompt might be, "When your emotions are positive, the recommended menu is chocolate cake."

[1007] In this way, the present invention is a system that takes into account the emotional state of the user and can provide a more personalized learning experience and service.

[1008] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1009] Step 1: Import textbook data and past exam question data

[1010] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[1011] Specific operation: Users use devices such as smartphones or PCs to take photos of textbook pages or exam questions and upload them as PDF files. This data is then sent from the device to the server.

[1012] Output: Educational data files imported to the server

[1013] Step 2: Analyze and convert data to text

[1014] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[1015] How it works: The server uses OCR technology to analyze imported image data and PDFs and convert them into text data. It also uses Python and OpenCV to analyze special data such as formulas and graphs.

[1016] Output: Text data and analyzed special data (formulas, graphs, etc.)

[1017] Step 3: Generate a new problem

[1018] Input: Text data, analyzed past exam question data

[1019] How it works: The server uses a generative AI model to automatically generate new questions. It uses natural language processing technology to generate questions that correspond to specific themes and levels of difficulty.

[1020] Output: The new problem set generated

[1021] Step 4: Question distribution based on user requests

[1022] Input: User request (specific subject, difficulty level)

[1023] Specific operation: A user requests a question by specifying a specific subject and difficulty level using a terminal. The server selects appropriate questions based on the request and delivers them to the user's terminal.

[1024] Output: The problem delivered to the user's device

[1025] Step 5: Collecting answer data and determining whether the answers are correct or incorrect

[1026] Input: User's answer data

[1027] Specific operation: The user solves the problem and sends the answer data from the device to the server. The server analyzes the answer data using an automatic scoring algorithm and determines whether the answer is correct or incorrect.

[1028] Output: Record of correct / incorrect judgment results and answer data

[1029] Step 6: Analyze learning trends and identify weak areas

[1030] Input: User's answer data, correct / incorrect result

[1031] Specific operation: The server analyzes the user's past answer data and identifies areas of weakness, thereby understanding the user's learning trends.

[1032] Output: Analysis results (user's weak areas)

[1033] Step 7: Focus questions and provide feedback

[1034] Input: Analysis results (weak areas), learning progress data

[1035] What it does: The server generates new problems based on the identified weaknesses and delivers them to the user. It also monitors learning progress and provides appropriate feedback, such as a message saying, "Your success rate in solving quadratic equations has improved to 80%."

[1036] Output: Focused questions, feedback messages

[1037] Step 8: Emotion recognition by acquiring facial expressions and voice data

[1038] Input: User's facial expression data, voice data

[1039] How it works: The user's device uses its built-in camera and microphone to capture facial expressions and voice data. This data is then sent from the device to the server, which then uses an emotion engine to identify the user's emotional state.

[1040] Output: Perceived emotional state

[1041] Step 9: Emotional content adjustment and feedback

[1042] Input: Recognized emotional state, learning tendency data

[1043] Specific behavior: The server adjusts the difficulty and quantity of questions based on the recognized emotion. It also generates a message encouraging the user to take a break if the user's concentration is low, and provides an encouraging message if the user's emotion is positive. For example, it generates a message saying, "You seem to be feeling a little tired. Let's take a five-minute break."

[1044] Output: Adjusted questions and feedback messages

[1045] Step 10: Generate a customized menu

[1046] Input: Recognized emotional state, historical data

[1047] Specific operation: Based on emotions and past data, the server uses a generative AI model to provide optimal learning content and services to users. For example, a food delivery service might suggest a menu item such as, "When you're feeling positive, the recommended menu is chocolate cake."

[1048] Output: Customized learning content or food menu suggestions

[1049] In this way, the specific actions and inputs / outputs at each step are clearly defined, optimizing the user learning experience and service delivery.

[1050] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1051] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1052] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1053] [Third embodiment]

[1054] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1055] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1056] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1057] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1058] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1059] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1060] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1061] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1062] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1063] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1064] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1065] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1066] System Overview

[1067] This invention is a learning support system for exam preparation. This system imports textbook data and past exam question data, analyzes that data, converts it into text format, and automatically generates new questions using a generation AI. Users answer the questions, and the answer data is collected and analyzed on a server, which identifies the user's weak areas and presents focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback.

[1068] Importing and analyzing textbook data and past exam data

[1069] Importing textbook data

[1070] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can upload "2023 Mathematics Past Questions" to the app.

[1071] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[1072] Question generation and distribution

[1073] Problem Generation

[1074] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[1075] Delivery in response to user requests

[1076] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[1077] Server: In response to a user request, selects appropriate problems and delivers them to the user's device. For example, it selects intermediate level math problems and sends them to the user's device.

[1078] Answer and learning trend analysis

[1079] Collecting and assessing answers

[1080] User's device: The user answers the delivered questions and sends the answer data to the server. For example, the user answers "x = -1, -2" to the question "Find the solution to y = x^2 + 3x + 2."

[1081] Server: The submitted answers are evaluated by an automatic scoring algorithm to determine whether they are correct or incorrect. The analysis results are stored in a database.

[1082] Analysis of learning trends

[1083] Server: Analyzes the user's answer data and identifies areas in which the user has particular difficulty. For example, it may be discovered that the user has difficulty with quadratic equations.

[1084] Focused questions and feedback on weak areas

[1085] Focused questions on weak areas

[1086] Server: Based on the user's learning tendencies, it generates new problems that are specific to weak areas. For example, it generates a large number of problems related to "quadratic equations."

[1087] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[1088] Providing Feedback

[1089] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1090] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[1091] Case Study

[1092] For example, if a user has difficulty with quadratic equations in mathematics, the system will analyze the user's answer data, generate and prioritize problems related to quadratic equations, and monitor the user's progress, providing feedback on how to improve their weaknesses. In this way, users can efficiently overcome their weaknesses and maximize their learning outcomes.

[1093] In the above-described form, the present invention provides a system that supports efficient learning for learners and enables them to effectively prepare for exams.

[1094] The processing flow will be explained below.

[1095] Step 1:

[1096] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1097] Step 2:

[1098] Server: Receives the captured data and converts it into text data using OCR (Optical Character Recognition) technology, for example, analyzing text and mathematical formulas in images and converting them into digital text.

[1099] Step 3:

[1100] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[1101] Step 4:

[1102] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[1103] Step 5:

[1104] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[1105] Step 6:

[1106] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1107] Step 7:

[1108] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[1109] Step 8:

[1110] User's device: Sends the answer to the server.

[1111] Step 9:

[1112] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[1113] Step 10:

[1114] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[1115] Step 11:

[1116] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[1117] Step 12:

[1118] On the user's device: Questions related to weak areas are highlighted, and the user is encouraged to retry those questions.

[1119] Step 13:

[1120] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1121] Step 14:

[1122] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[1123] Example 1

[1124] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1125] Conventional learning support systems have struggled to efficiently analyze users' learning progress and weak areas and provide appropriate questions. They also lack the ability to accurately analyze special data such as mathematical formulas and graphs to generate and deliver questions tailored to the subjects and level of difficulty desired by users. Furthermore, they lacked a means to effectively collect users' answer data and provide continuous feedback.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1127] In this invention, the server includes means for importing textbook data and past exam question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions that are focused on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for the user to request questions by specifying a specific subject or level of difficulty, and means for selecting and delivering appropriate questions in response to the user's request. This makes it possible to provide appropriate questions in accordance with the user's learning tendencies and to provide learning support through continuous feedback.

[1128] "Textbook data" refers to data that includes text information of learning materials used in learning.

[1129] "Past exam question data" refers to data that records the content of questions from exams that have been administered in the past.

[1130] "Analysis" is the process of deciphering captured data and converting it into text data or digital format.

[1131] "Text data" is data that represents character information in digital form.

[1132] "Generative AI" is an AI technology for automatically generating new problems.

[1133] "Means for generating questions" refers to technology that creates new study questions based on textbook data and past exam question data.

[1134] "User answer data" is data that includes the answers to questions answered by the user.

[1135] The "means for determining whether an answer is correct or incorrect" is a technique for determining whether an answer is correct or incorrect based on the user's answer data.

[1136] "Study trends" is information on learning progress and weak areas analyzed based on the user's question answer patterns and grades.

[1137] "Weak areas" refer to areas of study or types of questions that the user is particularly weak at.

[1138] "Feedback" is information about improvements and advice provided based on the user's learning progress.

[1139] "Means for requesting" is a function that allows the user to request questions by specifying a specific subject or level of difficulty.

[1140] "Distribution means" refers to the technology for transmitting generated questions and feedback to the user's terminal.

[1141] MODE FOR CARRYING OUT THE INVENTION

[1142] The present invention relates to a learning support system for exam preparation. This system imports textbook data and past exam question data uploaded by users, analyzes the data, converts it into text format, and generates new questions. The generated questions are distributed upon user request, allowing the user to answer them. The user's answer data is collected, and correct answers are determined and their learning trends are analyzed. The system then presents questions that focus on the user's weak areas, and continuously monitors the user's learning progress to provide appropriate feedback. Specific embodiments are described below.

[1143] Importing and analyzing textbook data and past exam data

[1144] User device: The user uses the application to upload textbook and past exam question data. For example, by selecting a PDF file of "2023 Mathematics Past Questions" and pressing the upload button, the data is sent to the server.

[1145] Server: The server receives the textbook data uploaded from the user's device. The received data is converted into text data using OCR (Optical Character Recognition) technology. During this process, special data such as mathematical formulas and graphs are also recognized and saved as text or figures.

[1146] Question generation and distribution

[1147] Server: The server generates new questions using a generative AI (artificial intelligence) model based on the analyzed textbook data. For example, the generative AI model generates a question asking about the solution to a quadratic equation.

[1148] On the user's device: The user specifies the subject and difficulty level they want to study through the application. For example, they may request "Intermediate level math problems."

[1149] Server: The server receives the user's request, selects appropriate problems from the generated questions, and delivers them to the user's device. For example, a set of intermediate level math problems is sent to the user's device.

[1150] Collecting and assessing answer data

[1151] User's device: The user answers the received question and sends the answer data to the server via the application. For example, in response to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the user answers "x = 2, x = 3."

[1152] Server: The server analyzes the received answer data using an automatic scoring algorithm and determines whether the answer is correct. If the answer is correct, the server presents the next question, and if the answer is incorrect, it displays an explanation.

[1153] Learning trend analysis and feedback

[1154] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendency. For example, it discovers that the user's average correct answer rate for quadratic equation problems is 50%.

[1155] Server: The server generates feedback for the user based on the analysis results, for example, a message saying "You need to practice more to improve your accuracy on quadratic equation problems."

[1156] User's device: The user's device receives and displays the feedback sent from the server, for example, a notification saying, "Try some extra practice to improve your accuracy on quadratic equation problems."

[1157] Focused questions on weak areas

[1158] Server: Based on the analysis results, the server generates new problems that are specific to the user's weak areas. For example, it generates a large number of problems related to "quadratic equations."

[1159] On the user's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed.

[1160] Examples of concrete examples and prompts

[1161] As a concrete example, consider the case where a user has difficulty with quadratic equations in mathematics. The user uploads "2023 Math Past Papers" to the app, which is then received and analyzed by the server. Then, a generation AI is used to generate "quadratic equation problems" and deliver them as the "intermediate level math problems" requested by the user.

[1162] An example prompt is:

[1163] "Generate new quadratic equation problems from past math questions for 2023. A user has requested intermediate level math problems."

[1164] This allows the system to provide an environment in which users can study efficiently.

[1165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1166] Program processing flow

[1167] Step 1:

[1168] User's device: The user uses the learning support system's application to upload textbook and past exam question data. For example, they select a PDF file of "2023 Mathematics Past Questions" and press the upload button. This input data (textbook data) is sent to the server.

[1169] Step 2:

[1170] Server: The server receives textbook data uploaded from the user's device. This received data is converted into text data using OCR (optical character recognition) technology. Specifically, it analyzes images and text within the PDF and converts them into text format. Special data such as mathematical formulas and graphs are also converted into digital format as text or figures and saved. The output data is text data.

[1171] Step 3:

[1172] Server: The server uses a generative AI model to generate new problems based on the analyzed text data. For example, the generative AI model generates a "problem asking about the solution to a quadratic equation." Specifically, it takes as input a prompt statement that specifies the difficulty and content of the problem to be generated, and outputs the generated new problem data. The output data is the generated problem set.

[1173] Step 4:

[1174] User's device: The user requests questions through the application, specifying the subject and level of difficulty they want to study. For example, they might request "Please give me intermediate level math questions." The input data is the user's request.

[1175] Step 5:

[1176] Server: The server receives the user's request, selects the appropriate problem from the generated problem set, and delivers it to the Yuga terminal. Specifically, it searches the problem database based on the request content, selects the appropriate problem, and delivers it. The output data is the result of delivering the problem to the user terminal.

[1177] Step 6:

[1178] User's device: The user answers the delivered questions and sends the answer data to the server through the application. For example, to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the answer is "x = 2, x = 3." The input data is the user's answer.

[1179] Step 7:

[1180] Server: The server uses an automatic scoring algorithm to evaluate the received answer data and confirm whether it is correct or not. Specifically, it compares the answer data with a correct answer database and determines whether it is correct. The output data is the result of the evaluation.

[1181] Step 8:

[1182] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendencies. Specifically, it analyzes data such as the correct answer rate and the time it takes to answer, and identifies specific weak areas. The output data is the learning tendency analysis results.

[1183] Step 9:

[1184] Server: The server generates feedback for the user based on the analysis results. For example, it creates a message saying, "You need to practice more to improve your accuracy in solving quadratic equation problems." Specifically, it takes the learning trend analysis results as input, generates a feedback message, and sends it. The output data is the feedback message.

[1185] Step 10:

[1186] User's device: The user's device receives and displays the feedback sent from the server. For example, a notification such as "Please practice more to improve your accuracy on quadratic equation problems" is displayed. The input data is the feedback message.

[1187] Step 11:

[1188] Server: Based on the analysis results, the server generates new questions that are specific to the user's weak areas. Specifically, it uses information about the weak areas as input and provides prompts to the generative AI model to create new questions. The output data is the newly generated questions.

[1189] Step 12:

[1190] User's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed. The input data is the newly generated problem.

[1191] (Application example 1)

[1192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1193] Conventional learning support systems limit students to a passive learning experience where they simply solve problems, making it difficult to improve learning efficiency. Furthermore, they often lack the ability to select questions tailored to specific subjects or levels of difficulty, or to analyze learning trends based on users' answer data, making it difficult to provide effective feedback or prioritized questions. Furthermore, they lack real-time learning progress monitoring or learning support utilizing virtual spaces, making it difficult to maintain users' motivation to learn.

[1194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1195] In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether the answers are correct, means for analyzing the user's learning tendencies and identifying specific areas of weakness, means for providing focused questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, and means for supporting learning in a virtual space using a head-mounted display or smart glasses. This allows users to learn interactively in a virtual space, breaking away from passive learning and significantly improving learning efficiency and motivation.

[1196] "Textbook data" refers to digital data of learning materials and textbooks used in learning.

[1197] "Past exam question data" refers to digitized data of exam questions that have been administered in the past.

[1198] "Capturing means" refers to the method or technology by which the subject data is entered or uploaded into the system.

[1199] "Means for analyzing and converting to text data" refers to techniques or methods for analyzing the captured data and converting it into a readable text format.

[1200] "Generative AI" refers to systems or algorithms that use artificial intelligence techniques to generate new data and information.

[1201] "Means of generating questions" refers to techniques and methods for creating new questions based on textbook data and past exam question data.

[1202] "User answer data" refers to data of the answers entered by the user in response to the questions posed.

[1203] The "means for determining whether an answer is correct" refers to a technique or method for analyzing the user's answer data and determining whether the answer is correct or incorrect.

[1204] "Means for analyzing learning trends" refers to techniques and methods for analyzing a user's learning patterns and trends based on the user's answer data.

[1205] "Means for extracting weak areas" refers to techniques or methods for analyzing a user's learning data and identifying areas in which the user is particularly weak.

[1206] A "method of providing focused questions" is a technique or method of providing focused questions based on the user's areas of weakness.

[1207] "Means for continuously monitoring learning progress" refers to techniques and methods for constantly monitoring a user's learning status and tracking progress.

[1208] "Means for providing feedback" refers to techniques and methods for providing advice and guidance according to the user's learning situation.

[1209] A "head-mounted display" is a display device that displays a virtual space when worn by a user on the head.

[1210] "Smart glasses" are glasses-type devices that can display augmented reality and virtual reality.

[1211] "Means to support learning in virtual spaces" are technologies and methods that use virtual reality or augmented reality technology to help users learn in virtual spaces.

[1212] This invention provides a system that allows users to study effectively in a virtual space. This system imports textbook data and past exam question data, converts the data into text format, and generates new questions using a generative artificial intelligence (generative AI model). It also collects and analyzes user answer data, extracts specific areas of weakness, and provides focused questions and feedback.

[1213] Importing and analyzing textbook data and past exam data

[1214] The system receives textbook data and past exam question data from users' devices. Users can take photos of this data or upload it as a PDF or Doc file. This data is received by the server and converted into text data using OCR technology (e.g., Tesseract). Special data (such as mathematical formulas and graphs) is also analyzed and saved in digital format.

[1215] Question generation and distribution

[1216] The server generates new questions using a generative AI model (e.g., OpenAI GPT-4) based on textbook data and past exam question data. The generated questions are delivered in response to user requests. The user specifies a specific subject and level of difficulty, and the server selects appropriate questions based on that request and delivers them to the user's device.

[1217] Answer and learning trend analysis

[1218] Users answer the questions and send the answers to the server. The server then uses an automated scoring algorithm (e.g., Python and Scikit-Learn) to evaluate the answers and assess whether they are correct. The analysis results are stored in a database, and specific areas of weakness are identified by analyzing the user's learning patterns.

[1219] Learning support in virtual space

[1220] Users use a head-mounted display (HMD) or smart glasses to enter a virtual space and study. Within the virtual space, textbook data and generated questions are displayed as 3D objects, which users can manipulate interactively. In the virtual classroom, users answer questions using a virtual pen in their hand. Answer data is collected and analyzed in real time, and a virtual assistant provides appropriate feedback.

[1221] Examples and prompts

[1222] For example, if a user is learning quadratic equations in a virtual space, the learning progresses through the following steps.

[1223] Example: Quadratic equation problem: "Find the solution to y = x^2 + 3x + 2"

[1224] User's solution: The user writes "x = -1, -2" on the whiteboard in the virtual space with a virtual pen.

[1225] feedback:

[1226] A virtual assistant will appear and provide feedback, saying, "That's right! Try the next one," and then a more difficult question will be displayed.

[1227] Example prompt sentence:

[1228] A user is solving a quadratic equation problem in a virtual classroom. The answer is "x = -1, -2." Please provide appropriate feedback for this correct answer.

[1229] In this way, users can learn interactively in a virtual space, moving away from passive learning and significantly improving their learning efficiency and motivation.

[1230] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1231] Step 1:

[1232] Importing textbook data and past exam question data

[1233] User's device: Users can take photos of textbook data and past exam question data or upload them as PDF or Doc files.

[1234] Input: Image data, PDF files, Doc files

[1235] Output: Raw data sent to the server

[1236] What happens: A user opens the application, selects a textbook or exam question file, and clicks the upload button. The file is then sent to the server.

[1237] Step 2:

[1238] Data analysis and conversion to text data

[1239] Server: Converts received data into text using OCR technology (such as Tesseract). Analyzes special data such as mathematical formulas and graphs and saves them in digital format.

[1240] Input: Raw data (image, PDF, Doc)

[1241] Output: Text data, special data in digital format

[1242] How it works: The server analyzes the received files and uses OCR technology to convert the contents of textbooks and exam questions into text data. It also recognizes mathematical formulas and graphs and saves them as digital data.

[1243] Step 3:

[1244] Problem Generation

[1245] Server: Generates new questions using a generative AI model (such as OpenAI GPT-4) based on textbook data and past exam question data.

[1246] Input: Text data, past exam question data

[1247] Output: The newly generated problem data

[1248] How it works: The server analyzes the text data and provides it as input to the generative AI model, which then generates new questions and stores them in a database.

[1249] Step 4:

[1250] Problem delivery based on user requests

[1251] Server: Accepts user requests for questions based on specific subjects and difficulty levels, selects appropriate questions based on those requests, and delivers them to the user's device.

[1252] Input: User's problem request (subject, difficulty level)

[1253] Output: Selected question data, delivered to the terminal

[1254] Specific operation: The user selects the subject and difficulty level on the app and requests questions. The server receives the request, selects appropriate questions, and delivers them to the user's device.

[1255] Step 5:

[1256] Collecting answer data and determining correctness

[1257] User's device: The user answers the questions delivered and sends the answer data to the server.

[1258] Server: Determines whether the answer is correct or incorrect based on an algorithm and stores the analysis results in a database.

[1259] Input: User's answer data

[1260] Output: Correct / incorrect result, saved in database

[1261] Specific operation: The user answers questions in the virtual space, inputs the answers, and sends them to the server. The server analyzes the answers, determines whether they are correct, and saves the results.

[1262] Step 6:

[1263] Analysis of learning trends and identification of weak areas

[1264] Server: Analyzes user answer data, identifies learning trends, and identifies specific areas of weakness.

[1265] Input: Correct / incorrect results, past answer data

[1266] Output: User learning tendency data, weak area data

[1267] Specific operation: The server runs an algorithm that analyzes learning trends based on the user's answer data stored in the database and extracts specific areas of weakness.

[1268] Step 7:

[1269] Focused questions and feedback

[1270] Server: Generates new questions based on the extracted weak areas and presents them to the user in a focused manner. In addition, it monitors the learning progress and provides appropriate feedback.

[1271] Input: Weakness Area Data

[1272] Output: New problem data, feedback message

[1273] Specific operation: The server generates new questions related to the weak areas and delivers them to the user's device. It also generates and provides feedback messages to the user based on their learning progress.

[1274] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1275] System Overview

[1276] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[1277] Importing and analyzing textbook data and past exam data

[1278] Importing textbook data

[1279] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1280] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[1281] Question generation and distribution

[1282] Problem Generation

[1283] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[1284] Delivery in response to user requests

[1285] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[1286] Server: In response to a user request, selects an appropriate problem and delivers it to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1287] Answer and learning trend analysis

[1288] Collecting and assessing answers

[1289] User's device: The user answers the delivered problem and sends the answer data to the server. For example, the user answers the problem "Find the solution to y = x^2 + 3x + 2" as "x = -1, -2".

[1290] Server: The submitted answer is evaluated by an automatic scoring algorithm to determine whether it is correct or not. For example, it determines that the user's answer "x = -1, -2" is correct.

[1291] Analysis of learning trends

[1292] Server: Analyzes the user's answer data and identifies areas of particular difficulty. For example, analyzes how well the user answers quadratic equation problems.

[1293] Focused questions and feedback on weak areas

[1294] Focused questions on weak areas

[1295] Server: Based on the user's learning tendencies, generate new problems that are specific to weak areas. For example, generate additional problems related to "quadratic equations."

[1296] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[1297] Providing Feedback

[1298] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1299] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[1300] Incorporating an emotion engine

[1301] emotion recognition

[1302] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[1303] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[1304] Emotion-based problem adjustment

[1305] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[1306] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[1307] Providing emotion-based feedback

[1308] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[1309] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[1310] In the above-described manner, the present invention constructs a system that takes into account the emotional state of the learner and can provide a more personalized learning experience.

[1311] The processing flow will be explained below.

[1312] Step 1:

[1313] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1314] Step 2:

[1315] Server: Receives the captured data. The received data is converted into text data using OCR technology. Special data such as formulas and graphs are also analyzed and saved in digital format.

[1316] Step 3:

[1317] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[1318] Step 4:

[1319] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[1320] Step 5:

[1321] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[1322] Step 6:

[1323] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1324] Step 7:

[1325] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[1326] Step 8:

[1327] User's device: Sends the answer to the server.

[1328] Step 9:

[1329] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[1330] Step 10:

[1331] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[1332] Step 11:

[1333] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[1334] Step 12:

[1335] On the user's device: Questions related to weak areas are displayed with emphasis, and the user is encouraged to retry those questions.

[1336] Step 13:

[1337] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1338] Step 14:

[1339] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[1340] Step 15:

[1341] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[1342] Step 16:

[1343] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[1344] Step 17:

[1345] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[1346] Step 18:

[1347] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[1348] Step 19:

[1349] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[1350] Step 20:

[1351] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[1352] Example 2

[1353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1354] Conventional learning support systems provide questions based on the user's learning progress and weak areas, but they lack the flexibility to consider the user's emotional state. This can lead to a decrease in the user's concentration and stress, potentially reducing the effectiveness of their learning. Furthermore, their ability to provide questions based on the user's specific requests is limited, leaving a need for more effective learning support.

[1355] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1356] In this invention, the server includes a means for importing textbook data and past exam question data, a means for analyzing the imported data and converting it into text data, a means for generating new questions using generative artificial intelligence, a means for collecting user answer data and determining whether the answers are correct, a means for analyzing the user's learning tendencies and identifying specific areas of weakness, a means for providing questions focused on the areas of weakness, a means for continuously monitoring the user's learning progress and providing feedback, and a means for recognizing the user's emotional state and reflecting it in learning activities. This enables personalized question provision and feedback that takes the user's emotional state into account, improving learning effectiveness. Furthermore, questions of specific subjects and levels of difficulty can be provided in response to the user's requests, supporting efficient learning.

[1357] "Textbook data" refers to digital data such as text information, mathematical formulas, and graphs contained in textbooks.

[1358] "Past exam question data" refers to information on exam questions that have been asked in the past, stored in digital format.

[1359] "Generative AI" is an AI technology that has the ability to generate new information and solutions based on input data.

[1360] "User answer data" refers to answer information entered by the user in response to the questions provided.

[1361] "Learning trends" refers to data that indicates specific patterns or progress observed in the user's learning process.

[1362] "Weakness areas" are areas of learning where the user does not fully understand or is less accurate in their answers.

[1363] "Feedback" refers to messages of encouragement and suggestions for improvement provided based on the user's learning progress and answer results.

[1364] "Emotional state" refers to the user's current mental and psychological state, such as whether they are focused, tired, or stressed.

[1365] A "camera" is a device that optically captures information and stores and transmits it as digital data.

[1366] A "microphone" is a device that captures sound and stores and transmits it as digital data.

[1367] MODE FOR CARRYING OUT THE INVENTION

[1368] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[1369] Importing and analyzing textbook data and past exam data

[1370] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The textbook page captured as an image file is saved in the app.

[1371] Examples:

[1372] The user takes a photo of a textbook page using their smartphone camera and saves it in the app.

[1373] The user uploads past exam questions in PDF format.

[1374] The server receives the textbook data sent from the user's device, converts it into text using OCR technology (such as Google's Tesseract OCR), analyzes special data such as formulas and graphs, and saves it in digital format.

[1375] Examples:

[1376] The server receives textbook images and converts them into text data using Tesseract OCR.

[1377] Special formulas and graphs are analyzed and saved using libraries such as MathJax.

[1378] Question generation and distribution

[1379] The server generates new questions using a generative AI (e.g., OpenAI's GPT-4) based on textbook data and past exam question data. It generates questions by inputting question generation prompts into the AI ​​model.

[1380] Examples:

[1381] The AI ​​is instructed to generate a problem based on the formula for solving a quadratic equation, and the generated problem is saved in text format.

[1382] The user's device requests questions by specifying a specific subject and difficulty level. The user selects the "Mathematics" subject and the "Intermediate" level in the app interface.

[1383] Examples:

[1384] The user selects and requests "Intermediate level math problems."

[1385] The server selects appropriate questions in response to a user's request and delivers them to the user's terminal.

[1386] Examples:

[1387] The server searches for "intermediate level quadratic equation problems" and sends the corresponding problems to the user's terminal.

[1388] Answer and learning trend analysis

[1389] The user answers the questions delivered to the user's terminal and transmits the answer data to the server.

[1390] Examples:

[1391] The user answers the question "Find the solution to y = x^2 + 3x + 2" with "x = -1, -2", and the app collects the answer and sends it to the server.

[1392] The server uses an automatic scoring algorithm to evaluate the submitted answers and determine whether they are correct or not.

[1393] Examples:

[1394] The server receives the user's answer "x = -1, -2" and uses an automatic scoring algorithm to determine that this is the correct answer.

[1395] The server analyzes the user's answer data and identifies areas in which the user is particularly weak.

[1396] Examples:

[1397] The server analyzes the user's answer history and identifies a low rate of correct answers for problems involving quadratic equations.

[1398] Focused questions and feedback on weak areas

[1399] The server generates new questions that are specific to the user's weak areas based on the user's learning tendencies.

[1400] Examples:

[1401] The server inputs a new problem related to "quadratic equations" as a prompt to the generation AI and sends the generated problem to the user's device.

[1402] The user's device will display many questions related to areas in which they are weak, and they can try those questions again.

[1403] Examples:

[1404] Many additional practice problems related to "quadratic equations" will appear on the user's smartphone.

[1405] The server continuously monitors the user's learning progress and provides appropriate feedback.

[1406] Examples:

[1407] The server generates a message such as "Your quadratic equation accuracy rate has improved to 80%."

[1408] The user's device receives feedback from the server and adjusts the learning plan.

[1409] Examples:

[1410] The user reads the feedback message and sets a new goal, such as "strengthen memorization."

[1411] Incorporating an emotion engine

[1412] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and transmits them to the emotion engine.

[1413] Examples:

[1414] While the user is solving the problem, the device captures the user's face with a camera and also records their voice.

[1415] The server analyzes the received data using an emotion engine to identify the user's emotional state.

[1416] Examples:

[1417] The server analyzes the received data and determines whether the user is "concentrated" or "tired."

[1418] The server adjusts the difficulty and quantity of questions based on the recognized user emotions.

[1419] Examples:

[1420] If the user is determined to be "tired," the server will present more slightly easier questions.

[1421] The user's device is presented with adjusted questions, allowing the user to continue learning in a comfortable environment.

[1422] Examples:

[1423] Easy questions are displayed so that users can relax and work through the questions.

[1424] Based on the user's emotional data, the server generates messages encouraging them to take a break when their concentration is low and encouraging them when they are feeling positive.

[1425] Examples:

[1426] The server generates a message such as "You seem to be feeling a little tired. Let's take a 5-minute break" and sends it to the user's device.

[1427] The user's device receives emotion-based feedback and prompts to take breaks, allowing the user to adjust the pace of their learning as needed.

[1428] Examples:

[1429] The user checks the feedback message and takes a few minutes to regain focus.

[1430] Examples of prompt statements

[1431] 1. Upload a PDF file and generate a quadratic equation problem:

[1432] Prompt: "Generate three quadratic equation problems based on Chapter 3 of this PDF file."

[1433] 2. Generate additional questions based on the user's weak areas:

[1434] Prompt: "Based on your answer history, you are having difficulty with problems involving quadratic equations. Please generate another practice problem."

[1435] 3. Generating encouraging messages based on emotion data:

[1436] Prompt: "The user is feeling fatigued. Please generate a message encouraging them to take a break."

[1437] This allows the present invention to provide a tailored learning experience that takes into account the learner's emotional state and learning progress.

[1438] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1439] Step 1: Importing textbook data

[1440] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The app receives image files and PDF files as input and saves the data in the app. Specific operations include the user taking a photo of a textbook page with the smartphone camera and importing the image file into the app.

[1441] Input: Textbook image file or PDF file

[1442] Output: Textbook data saved in the app

[1443] Step 2: Analyzing textbook data

[1444] The server receives textbook data sent from the user's device. It converts the received data into text data using OCR technology (for example, Google's Tesseract OCR), and analyzes special data such as formulas and graphs before saving them in digital format. The input is image data or PDF files sent from the user's device, and the output is text data.

[1445] Input: Image data or PDF file from the device

[1446] Output: Text data stored on the server

[1447] Step 3: Generate the problem

[1448] The server generates new questions using a generative AI (for example, OpenAI's GPT-4) based on textbook data and past exam question data. It receives prompt text and analyzed text data as input and generates new questions based on that. Specifically, the server sends a prompt for a "problem based on the formula for solving a quadratic equation" to the generative AI and saves the generated problem in text format.

[1449] Input: prompt and parsed text data

[1450] Output: Text data of the generated problem

[1451] Step 4: Deliver in response to user requests

[1452] The user's device requests a problem by specifying a specific subject and difficulty level. The server receives the user's request information as input and sends it to the server. The server selects appropriate problems based on the user's request and delivers them to the user's device. As a specific example, the user selects and requests "intermediate level math problems," and the server searches for and sends "intermediate level quadratic equation problems."

[1453] Input: User request information

[1454] Output: The problem delivered to the user's device

[1455] Step 5: Collect and assess answers

[1456] The user answers the questions delivered to the user's device and sends the answer data to the server. The server receives the user's answer data as input and sends it to the server. The server uses an automatic scoring algorithm to evaluate the submitted answer and determine whether it is correct or not. For example, a user searches for the solution to y = x^2 + 3x + 2 and answers "x = -1, -2", and the data is sent to the server.

[1457] Input: User's answer

[1458] Output: Correct / incorrect result

[1459] Step 6: Analyze learning trends

[1460] The server analyzes the user's answer data and identifies areas in which the user is particularly weak. As input, it receives the user's past answer history and analyzes it to identify areas in which the user is weak. Specifically, the server calculates the correct answer rate for questions related to quadratic equations from the user's answer history and identifies areas in which the user is weak.

[1461] Input: User's past answer history

[1462] Output: Identification of user's weak areas

[1463] Step 7: Focused questions and feedback on weak areas

[1464] The server generates new problems specific to the user's weak areas based on the user's learning tendencies. It receives the user's weak areas as input and sends prompts for new problems to the generation AI based on that information. For example, it generates additional problems related to "quadratic equations" and sends them to the user's device.

[1465] Input: User's weak areas information

[1466] Output: New problem delivered to user's device

[1467] The server continuously monitors the user's learning progress and provides appropriate feedback. It receives the user's answer history and learning progress information as input, and generates a feedback message based on that information. For example, it generates a feedback message such as "Your rate of correct answers to quadratic equations has improved to 80%" and sends it to the user's device.

[1468] Input: User's answer history and learning progress information

[1469] Output: Feedback sent to the user's device

[1470] Step 8: Obtaining and analyzing sentiment data

[1471] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and recognize their emotional state. The user's facial expressions and voice data are acquired as input and sent to the server. The server then analyzes the received data using an emotion engine to identify the user's emotional state. Specifically, the server determines the user's emotional state, such as "concentrated" or "tired."

[1472] Input: facial expressions and voice data

[1473] Output: Emotional state judgment result

[1474] Step 9: Emotionally Based Problem Adjustment

[1475] The server adjusts the difficulty and quantity of questions based on the user's recognized emotions. It receives the user's emotional state as input and sends question prompts to the generation AI based on that. For example, if the user is determined to be "tired," the server will present more slightly easier questions.

[1476] Input: User's emotional state

[1477] Output: Adjusted problem

[1478] Step 10: Provide emotional feedback

[1479] The server generates and provides messages encouraging or encouraging a break based on the user's emotional data. It receives the user's emotional data as input and generates messages based on it. For example, it generates a message such as "You seem to be feeling a little tired. Let's take a five-minute break" and sends it to the user's device. The user's device receives the emotional feedback and the prompt to take a break, allowing the user to adjust the pace of their learning as needed.

[1480] Input: User emotion data

[1481] Output: Message to encourage break and adjustment of user behavior

[1482] This allows the present invention to provide a more effective and personalized learning experience by taking into account the learner's emotional state and learning progress.

[1483] (Application example 2)

[1484] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1485] Conventional learning support systems have difficulty considering the user's emotional state, and are therefore unable to provide an optimal learning environment that responds to emotional changes.Furthermore, food delivery services do not offer personalized menu suggestions based on the user's emotions, making it difficult to improve user satisfaction.

[1486] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for recognizing emotions by acquiring facial expressions and voice data, means for adjusting the content of questions and feedback based on the recognized emotions, and means for generating customized menus based on emotions and past data. This allows for optimal learning and service provision that takes user emotions into consideration and responds to changes in those emotions.

[1487] "Textbook data" refers to digital data of text materials used in education.

[1488] "Past exam question data" refers to digital data of questions and answers from exams previously administered.

[1489] The "means for importing" refers to a method or device for acquiring textbook data and past exam question data from a user's terminal to a server.

[1490] An "analyzing means" is a method or device that processes captured data and converts it into an understandable form.

[1491] "Means for converting to text data" refers to technology that extracts text information from image data or PDFs and converts it into text format.

[1492] "Generative AI" is a technology that uses machine learning and natural language processing techniques to automatically generate new questions and text.

[1493] "Means for collecting" refers to a method or device for collecting answer data from users and transmitting it to the server.

[1494] The "means for determining whether a question is correct or incorrect" refers to an algorithm or device that analyzes collected answer data and automatically determines whether it is correct or incorrect.

[1495] "Means for analyzing learning trends" refers to technology that analyzes a user's answer data and identifies their learning patterns and weaknesses.

[1496] The "means for extracting weak areas" refers to a method or device for identifying areas of study that are particularly weak for a user based on an analysis of learning trends.

[1497] The "means of providing focused questions" refers to a method or device that provides questions intensively in the extracted weak areas.

[1498] "Monitoring means" refers to techniques for continuously observing a user's learning progress and collecting data.

[1499] A "means for providing feedback" is a method or device that provides advice or encouragement to the user based on collected data.

[1500] "Means for acquiring facial expression and voice data" refers to technology that uses the user's camera and microphone to capture facial expressions and voice and collect that data.

[1501] "Means for recognizing emotions" refers to technology that analyzes acquired facial expressions and voice data to identify the user's emotions.

[1502] The "means for adjusting the content of questions" refers to a method or device for changing the difficulty or amount of learning content based on the recognized emotions.

[1503] The "means for generating customized menus" is a technology that generates optimal food menus and learning content for users based on emotions and past data.

[1504] System Overview

[1505] This invention combines technologies to improve learning support systems and food delivery services. The system is equipped with an emotion engine that recognizes user emotions, aiming to provide a more personalized learning and service experience. The system of this invention has the ability to input textbook data and past exam question data, analyze them, and generate new questions. It also has the ability to collect and analyze user answer data, identify learning trends, and provide individualized feedback.

[1506] Importing and analyzing textbook data and past exam data

[1507] Users can photograph or upload textbook data and past exam question data using devices such as smartphones. The server converts the received data into text data using OCR technology, and analyzes special data such as formulas and graphs before saving them in digital format. The software used includes Python and OpenCV.

[1508] Question generation and distribution

[1509] The server generates new questions using a generative AI model based on imported textbook data and past exam question data. This generative AI model utilizes natural language processing technology. Users can request questions based on specific subjects and difficulty levels from their devices, and the server selects appropriate questions and delivers them to the user's device.

[1510] Answer and learning trend analysis

[1511] When a user answers a question, the answer data is sent to the server. The server uses an automatic scoring algorithm to judge the answer data and record whether it is correct or incorrect. The server also analyzes the user's answer data to identify areas in which the user is particularly weak. This allows the system to understand the user's learning trends.

[1512] Focused questions and feedback on weak areas

[1513] The server generates new questions based on the user's learning trends, focusing on areas of weakness. It also continuously monitors the user's learning progress and provides appropriate feedback. For example, it generates a message saying, "Your rate of correct answers to quadratic equations has improved to 80%."

[1514] Incorporating an emotion engine

[1515] The user's device captures the user's facial expressions and voice using a built-in camera and microphone and sends them to the server. The server then analyzes the received data with an emotion engine to identify the user's emotional state. For example, it determines whether the user is focused, tired, or stressed. Based on the recognized emotion, the server adjusts the difficulty and quantity of questions and provides feedback.

[1516] Emotion-Based Feedback

[1517] The server uses the user's emotional data to generate messages that encourage a break if the user's concentration is low and encourages the user if the user's emotions are positive. For example, a message such as "You seem to be feeling a little tired. Let's take a five-minute break" is displayed.

[1518] Customized Menu Generation

[1519] Based on emotions and past data, the system provides users with optimal learning content and services. For example, in a food delivery service, a generative AI model suggests individually customized food menus based on emotions and past order data. An example prompt might be, "When your emotions are positive, the recommended menu is chocolate cake."

[1520] In this way, the present invention is a system that takes into account the emotional state of the user and can provide a more personalized learning experience and service.

[1521] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1522] Step 1: Import textbook data and past exam question data

[1523] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[1524] Specific operation: Users use devices such as smartphones or PCs to take photos of textbook pages or exam questions and upload them as PDF files. This data is then sent from the device to the server.

[1525] Output: Educational data files imported to the server

[1526] Step 2: Analyze and convert data to text

[1527] Input: Textbook data, past exam question data (image files, PDF files, etc.)

[1528] How it works: The server uses OCR technology to analyze imported image data and PDFs and convert them into text data. It also uses Python and OpenCV to analyze special data such as formulas and graphs.

[1529] Output: Text data and analyzed special data (formulas, graphs, etc.)

[1530] Step 3: Generate a new problem

[1531] Input: Text data, analyzed past exam question data

[1532] How it works: The server uses a generative AI model to automatically generate new questions. It uses natural language processing technology to generate questions that correspond to specific themes and levels of difficulty.

[1533] Output: The new problem set generated

[1534] Step 4: Question distribution based on user requests

[1535] Input: User request (specific subject, difficulty level)

[1536] Specific operation: A user requests a question by specifying a specific subject and difficulty level using a terminal. The server selects appropriate questions based on the request and delivers them to the user's terminal.

[1537] Output: The problem delivered to the user's device

[1538] Step 5: Collecting answer data and determining whether the answers are correct or incorrect

[1539] Input: User's answer data

[1540] Specific operation: The user solves the problem and sends the answer data from the device to the server. The server analyzes the answer data using an automatic scoring algorithm and determines whether the answer is correct or incorrect.

[1541] Output: Record of correct / incorrect judgment results and answer data

[1542] Step 6: Analyze learning trends and identify weak areas

[1543] Input: User's answer data, correct / incorrect result

[1544] Specific operation: The server analyzes the user's past answer data and identifies areas of weakness, thereby understanding the user's learning trends.

[1545] Output: Analysis results (user's weak areas)

[1546] Step 7: Focus questions and provide feedback

[1547] Input: Analysis results (weak areas), learning progress data

[1548] What it does: The server generates new problems based on the identified weaknesses and delivers them to the user. It also monitors learning progress and provides appropriate feedback, such as a message saying, "Your success rate in solving quadratic equations has improved to 80%."

[1549] Output: Focused questions, feedback messages

[1550] Step 8: Emotion recognition by acquiring facial expressions and voice data

[1551] Input: User's facial expression data, voice data

[1552] How it works: The user's device uses its built-in camera and microphone to capture facial expressions and voice data. This data is then sent from the device to the server, which then uses an emotion engine to identify the user's emotional state.

[1553] Output: Perceived emotional state

[1554] Step 9: Emotional content adjustment and feedback

[1555] Input: Recognized emotional state, learning tendency data

[1556] Specific behavior: The server adjusts the difficulty and quantity of questions based on the recognized emotion. It also generates a message encouraging the user to take a break if the user's concentration is low, and provides an encouraging message if the user's emotion is positive. For example, it generates a message saying, "You seem to be feeling a little tired. Let's take a five-minute break."

[1557] Output: Adjusted questions and feedback messages

[1558] Step 10: Generate a customized menu

[1559] Input: Recognized emotional state, historical data

[1560] Specific operation: Based on emotions and past data, the server uses a generative AI model to provide optimal learning content and services to users. For example, a food delivery service might suggest a menu item such as, "When you're feeling positive, the recommended menu is chocolate cake."

[1561] Output: Customized learning content or food menu suggestions

[1562] In this way, the specific actions and inputs / outputs at each step are clearly defined, optimizing the user learning experience and service delivery.

[1563] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1564] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1565] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1566] [Fourth embodiment]

[1567] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1568] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1569] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1570] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1571] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1572] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1573] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1574] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1575] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1576] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1577] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1578] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1579] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1580] System Overview

[1581] This invention is a learning support system for exam preparation. This system imports textbook data and past exam question data, analyzes that data, converts it into text format, and automatically generates new questions using a generation AI. Users answer the questions, and the answer data is collected and analyzed on a server, which identifies the user's weak areas and presents focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback.

[1582] Importing and analyzing textbook data and past exam data

[1583] Importing textbook data

[1584] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can upload "2023 Mathematics Past Questions" to the app.

[1585] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[1586] Question generation and distribution

[1587] Problem Generation

[1588] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[1589] Delivery in response to user requests

[1590] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[1591] Server: In response to a user request, selects appropriate problems and delivers them to the user's device. For example, it selects intermediate level math problems and sends them to the user's device.

[1592] Answer and learning trend analysis

[1593] Collecting and assessing answers

[1594] User's device: The user answers the delivered questions and sends the answer data to the server. For example, the user answers "x = -1, -2" to the question "Find the solution to y = x^2 + 3x + 2."

[1595] Server: The submitted answers are evaluated by an automatic scoring algorithm to determine whether they are correct or incorrect. The analysis results are stored in a database.

[1596] Analysis of learning trends

[1597] Server: Analyzes the user's answer data and identifies areas in which the user has particular difficulty. For example, it may be discovered that the user has difficulty with quadratic equations.

[1598] Focused questions and feedback on weak areas

[1599] Focused questions on weak areas

[1600] Server: Based on the user's learning tendencies, it generates new problems that are specific to weak areas. For example, it generates a large number of problems related to "quadratic equations."

[1601] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[1602] Providing Feedback

[1603] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1604] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[1605] Case Study

[1606] For example, if a user has difficulty with quadratic equations in mathematics, the system will analyze the user's answer data, generate and prioritize problems related to quadratic equations, and monitor the user's progress, providing feedback on how to improve their weaknesses. In this way, users can efficiently overcome their weaknesses and maximize their learning outcomes.

[1607] In the above-described form, the present invention provides a system that supports efficient learning for learners and enables them to effectively prepare for exams.

[1608] The processing flow will be explained below.

[1609] Step 1:

[1610] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1611] Step 2:

[1612] Server: Receives the captured data and converts it into text data using OCR (Optical Character Recognition) technology, for example, analyzing text and mathematical formulas in images and converting them into digital text.

[1613] Step 3:

[1614] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[1615] Step 4:

[1616] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[1617] Step 5:

[1618] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[1619] Step 6:

[1620] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1621] Step 7:

[1622] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[1623] Step 8:

[1624] User's device: Sends the answer to the server.

[1625] Step 9:

[1626] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[1627] Step 10:

[1628] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[1629] Step 11:

[1630] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[1631] Step 12:

[1632] On the user's device: Questions related to weak areas are highlighted, and the user is encouraged to retry those questions.

[1633] Step 13:

[1634] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1635] Step 14:

[1636] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[1637] Example 1

[1638] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1639] Conventional learning support systems have struggled to efficiently analyze users' learning progress and weak areas and provide appropriate questions. They also lack the ability to accurately analyze special data such as mathematical formulas and graphs to generate and deliver questions tailored to the subjects and level of difficulty desired by users. Furthermore, they lacked a means to effectively collect users' answer data and provide continuous feedback.

[1640] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1641] In this invention, the server includes means for importing textbook data and past exam question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether they are correct, means for analyzing the user's learning tendencies and extracting specific areas of weakness, means for providing questions that are focused on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, means for the user to request questions by specifying a specific subject or level of difficulty, and means for selecting and delivering appropriate questions in response to the user's request. This makes it possible to provide appropriate questions in accordance with the user's learning tendencies and to provide learning support through continuous feedback.

[1642] "Textbook data" refers to data that includes text information of learning materials used in learning.

[1643] "Past exam question data" refers to data that records the content of questions from exams that have been administered in the past.

[1644] "Analysis" is the process of deciphering captured data and converting it into text data or digital format.

[1645] "Text data" is data that represents character information in digital form.

[1646] "Generative AI" is an AI technology for automatically generating new problems.

[1647] "Means for generating questions" refers to technology that creates new study questions based on textbook data and past exam question data.

[1648] "User answer data" is data that includes the answers to questions answered by the user.

[1649] The "means for determining whether an answer is correct or incorrect" is a technique for determining whether an answer is correct or incorrect based on the user's answer data.

[1650] "Study trends" is information on learning progress and weak areas analyzed based on the user's question answer patterns and grades.

[1651] "Weak areas" refer to areas of study or types of questions that the user is particularly weak at.

[1652] "Feedback" is information about improvements and advice provided based on the user's learning progress.

[1653] "Means for requesting" is a function that allows the user to request questions by specifying a specific subject or level of difficulty.

[1654] "Distribution means" refers to the technology for transmitting generated questions and feedback to the user's terminal.

[1655] MODE FOR CARRYING OUT THE INVENTION

[1656] The present invention relates to a learning support system for exam preparation. This system imports textbook data and past exam question data uploaded by users, analyzes the data, converts it into text format, and generates new questions. The generated questions are distributed upon user request, allowing the user to answer them. The user's answer data is collected, and correct answers are determined and their learning trends are analyzed. The system then presents questions that focus on the user's weak areas, and continuously monitors the user's learning progress to provide appropriate feedback. Specific embodiments are described below.

[1657] Importing and analyzing textbook data and past exam data

[1658] User device: The user uses the application to upload textbook and past exam question data. For example, by selecting a PDF file of "2023 Mathematics Past Questions" and pressing the upload button, the data is sent to the server.

[1659] Server: The server receives the textbook data uploaded from the user's device. The received data is converted into text data using OCR (Optical Character Recognition) technology. During this process, special data such as mathematical formulas and graphs are also recognized and saved as text or figures.

[1660] Question generation and distribution

[1661] Server: The server generates new questions using a generative AI (artificial intelligence) model based on the analyzed textbook data. For example, the generative AI model generates a question asking about the solution to a quadratic equation.

[1662] On the user's device: The user specifies the subject and difficulty level they want to study through the application. For example, they may request "Intermediate level math problems."

[1663] Server: The server receives the user's request, selects appropriate problems from the generated questions, and delivers them to the user's device. For example, a set of intermediate level math problems is sent to the user's device.

[1664] Collecting and assessing answer data

[1665] User's device: The user answers the received question and sends the answer data to the server via the application. For example, in response to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the user answers "x = 2, x = 3."

[1666] Server: The server analyzes the received answer data using an automatic scoring algorithm and determines whether the answer is correct. If the answer is correct, the server presents the next question, and if the answer is incorrect, it displays an explanation.

[1667] Learning trend analysis and feedback

[1668] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendency. For example, it discovers that the user's average correct answer rate for quadratic equation problems is 50%.

[1669] Server: The server generates feedback for the user based on the analysis results, for example, a message saying "You need to practice more to improve your accuracy on quadratic equation problems."

[1670] User's device: The user's device receives and displays the feedback sent from the server, for example, a notification saying, "Try some extra practice to improve your accuracy on quadratic equation problems."

[1671] Focused questions on weak areas

[1672] Server: Based on the analysis results, the server generates new problems that are specific to the user's weak areas. For example, it generates a large number of problems related to "quadratic equations."

[1673] On the user's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed.

[1674] Examples of concrete examples and prompts

[1675] As a concrete example, consider the case where a user has difficulty with quadratic equations in mathematics. The user uploads "2023 Math Past Papers" to the app, which is then received and analyzed by the server. Then, a generation AI is used to generate "quadratic equation problems" and deliver them as the "intermediate level math problems" requested by the user.

[1676] An example prompt is:

[1677] "Generate new quadratic equation problems from past math questions for 2023. A user has requested intermediate level math problems."

[1678] This allows the system to provide an environment in which users can study efficiently.

[1679] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1680] Program processing flow

[1681] Step 1:

[1682] User's device: The user uses the learning support system's application to upload textbook and past exam question data. For example, they select a PDF file of "2023 Mathematics Past Questions" and press the upload button. This input data (textbook data) is sent to the server.

[1683] Step 2:

[1684] Server: The server receives textbook data uploaded from the user's device. This received data is converted into text data using OCR (optical character recognition) technology. Specifically, it analyzes images and text within the PDF and converts them into text format. Special data such as mathematical formulas and graphs are also converted into digital format as text or figures and saved. The output data is text data.

[1685] Step 3:

[1686] Server: The server uses a generative AI model to generate new problems based on the analyzed text data. For example, the generative AI model generates a "problem asking about the solution to a quadratic equation." Specifically, it takes as input a prompt statement that specifies the difficulty and content of the problem to be generated, and outputs the generated new problem data. The output data is the generated problem set.

[1687] Step 4:

[1688] User's device: The user requests questions through the application, specifying the subject and level of difficulty they want to study. For example, they might request "Please give me intermediate level math questions." The input data is the user's request.

[1689] Step 5:

[1690] Server: The server receives the user's request, selects the appropriate problem from the generated problem set, and delivers it to the Yuga terminal. Specifically, it searches the problem database based on the request content, selects the appropriate problem, and delivers it. The output data is the result of delivering the problem to the user terminal.

[1691] Step 6:

[1692] User's device: The user answers the delivered questions and sends the answer data to the server through the application. For example, to the question "What is the solution to the quadratic equation x^2 - 5x + 6 = 0?", the answer is "x = 2, x = 3." The input data is the user's answer.

[1693] Step 7:

[1694] Server: The server uses an automatic scoring algorithm to evaluate the received answer data and confirm whether it is correct or not. Specifically, it compares the answer data with a correct answer database and determines whether it is correct. The output data is the result of the evaluation.

[1695] Step 8:

[1696] Server: The server accumulates the user's answer data and uses an analysis algorithm to analyze the user's learning tendencies. Specifically, it analyzes data such as the correct answer rate and the time it takes to answer, and identifies specific weak areas. The output data is the learning tendency analysis results.

[1697] Step 9:

[1698] Server: The server generates feedback for the user based on the analysis results. For example, it creates a message saying, "You need to practice more to improve your accuracy in solving quadratic equation problems." Specifically, it takes the learning trend analysis results as input, generates a feedback message, and sends it. The output data is the feedback message.

[1699] Step 10:

[1700] User's device: The user's device receives and displays the feedback sent from the server. For example, a notification such as "Please practice more to improve your accuracy on quadratic equation problems" is displayed. The input data is the feedback message.

[1701] Step 11:

[1702] Server: Based on the analysis results, the server generates new questions that are specific to the user's weak areas. Specifically, it uses information about the weak areas as input and provides prompts to the generative AI model to create new questions. The output data is the newly generated questions.

[1703] Step 12:

[1704] User's device: New problems related to weak areas are delivered to the user's device, and the user can try them again. For example, a problem such as "Find the solution to the following quadratic equation: x^2 - 4x + 3 = 0" is displayed. The input data is the newly generated problem.

[1705] (Application example 1)

[1706] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1707] Conventional learning support systems limit students to a passive learning experience where they simply solve problems, making it difficult to improve learning efficiency. Furthermore, they often lack the ability to select questions tailored to specific subjects or levels of difficulty, or to analyze learning trends based on users' answer data, making it difficult to provide effective feedback or prioritized questions. Furthermore, they lack real-time learning progress monitoring or learning support utilizing virtual spaces, making it difficult to maintain users' motivation to learn.

[1708] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1709] In this invention, the server includes means for importing textbook data and past test question data, means for analyzing the imported data and converting it into text data, means for generating new questions using generative artificial intelligence, means for collecting user answer data and determining whether the answers are correct, means for analyzing the user's learning tendencies and identifying specific areas of weakness, means for providing focused questions based on the areas of weakness, means for continuously monitoring the user's learning progress and providing feedback, and means for supporting learning in a virtual space using a head-mounted display or smart glasses. This allows users to learn interactively in a virtual space, breaking away from passive learning and significantly improving learning efficiency and motivation.

[1710] "Textbook data" refers to digital data of learning materials and textbooks used in learning.

[1711] "Past exam question data" refers to digitized data of exam questions that have been administered in the past.

[1712] "Capturing means" refers to the method or technology by which the subject data is entered or uploaded into the system.

[1713] "Means for analyzing and converting to text data" refers to techniques or methods for analyzing the captured data and converting it into a readable text format.

[1714] "Generative AI" refers to systems or algorithms that use artificial intelligence techniques to generate new data and information.

[1715] "Means of generating questions" refers to techniques and methods for creating new questions based on textbook data and past exam question data.

[1716] "User answer data" refers to data of the answers entered by the user in response to the questions posed.

[1717] The "means for determining whether an answer is correct" refers to a technique or method for analyzing the user's answer data and determining whether the answer is correct or incorrect.

[1718] "Means for analyzing learning trends" refers to techniques and methods for analyzing a user's learning patterns and trends based on the user's answer data.

[1719] "Means for extracting weak areas" refers to techniques or methods for analyzing a user's learning data and identifying areas in which the user is particularly weak.

[1720] A "method of providing focused questions" is a technique or method of providing focused questions based on the user's areas of weakness.

[1721] "Means for continuously monitoring learning progress" refers to techniques and methods for constantly monitoring a user's learning status and tracking progress.

[1722] "Means for providing feedback" refers to techniques and methods for providing advice and guidance according to the user's learning situation.

[1723] A "head-mounted display" is a display device that displays a virtual space when worn by a user on the head.

[1724] "Smart glasses" are glasses-type devices that can display augmented reality and virtual reality.

[1725] "Means to support learning in virtual spaces" are technologies and methods that use virtual reality or augmented reality technology to help users learn in virtual spaces.

[1726] This invention provides a system that allows users to study effectively in a virtual space. This system imports textbook data and past exam question data, converts the data into text format, and generates new questions using a generative artificial intelligence (generative AI model). It also collects and analyzes user answer data, extracts specific areas of weakness, and provides focused questions and feedback.

[1727] Importing and analyzing textbook data and past exam data

[1728] The system receives textbook data and past exam question data from users' devices. Users can take photos of this data or upload it as a PDF or Doc file. This data is received by the server and converted into text data using OCR technology (e.g., Tesseract). Special data (such as mathematical formulas and graphs) is also analyzed and saved in digital format.

[1729] Question generation and distribution

[1730] The server generates new questions using a generative AI model (e.g., OpenAI GPT-4) based on textbook data and past exam question data. The generated questions are delivered in response to user requests. The user specifies a specific subject and level of difficulty, and the server selects appropriate questions based on that request and delivers them to the user's device.

[1731] Answer and learning trend analysis

[1732] Users answer the questions and send the answers to the server. The server then uses an automated scoring algorithm (e.g., Python and Scikit-Learn) to evaluate the answers and assess whether they are correct. The analysis results are stored in a database, and specific areas of weakness are identified by analyzing the user's learning patterns.

[1733] Learning support in virtual space

[1734] Users use a head-mounted display (HMD) or smart glasses to enter a virtual space and study. Within the virtual space, textbook data and generated questions are displayed as 3D objects, which users can manipulate interactively. In the virtual classroom, users answer questions using a virtual pen in their hand. Answer data is collected and analyzed in real time, and a virtual assistant provides appropriate feedback.

[1735] Examples and prompts

[1736] For example, if a user is learning quadratic equations in a virtual space, the learning progresses through the following steps.

[1737] Example: Quadratic equation problem: "Find the solution to y = x^2 + 3x + 2"

[1738] User's solution: The user writes "x = -1, -2" on the whiteboard in the virtual space with a virtual pen.

[1739] feedback:

[1740] A virtual assistant will appear and provide feedback, saying, "That's right! Try the next one," and then a more difficult question will be displayed.

[1741] Example prompt sentence:

[1742] A user is solving a quadratic equation problem in a virtual classroom. The answer is "x = -1, -2." Please provide appropriate feedback for this correct answer.

[1743] In this way, users can learn interactively in a virtual space, moving away from passive learning and significantly improving their learning efficiency and motivation.

[1744] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1745] Step 1:

[1746] Importing textbook data and past exam question data

[1747] User's device: Users can take photos of textbook data and past exam question data or upload them as PDF or Doc files.

[1748] Input: Image data, PDF files, Doc files

[1749] Output: Raw data sent to the server

[1750] What happens: A user opens the application, selects a textbook or exam question file, and clicks the upload button. The file is then sent to the server.

[1751] Step 2:

[1752] Data analysis and conversion to text data

[1753] Server: Converts received data into text using OCR technology (such as Tesseract). Analyzes special data such as mathematical formulas and graphs and saves them in digital format.

[1754] Input: Raw data (image, PDF, Doc)

[1755] Output: Text data, special data in digital format

[1756] How it works: The server analyzes the received files and uses OCR technology to convert the contents of textbooks and exam questions into text data. It also recognizes mathematical formulas and graphs and saves them as digital data.

[1757] Step 3:

[1758] Problem Generation

[1759] Server: Generates new questions using a generative AI model (such as OpenAI GPT-4) based on textbook data and past exam question data.

[1760] Input: Text data, past exam question data

[1761] Output: The newly generated problem data

[1762] How it works: The server analyzes the text data and provides it as input to the generative AI model, which then generates new questions and stores them in a database.

[1763] Step 4:

[1764] Problem delivery based on user requests

[1765] Server: Accepts user requests for questions based on specific subjects and difficulty levels, selects appropriate questions based on those requests, and delivers them to the user's device.

[1766] Input: User's problem request (subject, difficulty level)

[1767] Output: Selected question data, delivered to the terminal

[1768] Specific operation: The user selects the subject and difficulty level on the app and requests questions. The server receives the request, selects appropriate questions, and delivers them to the user's device.

[1769] Step 5:

[1770] Collecting answer data and determining correctness

[1771] User's device: The user answers the questions delivered and sends the answer data to the server.

[1772] Server: Determines whether the answer is correct or incorrect based on an algorithm and stores the analysis results in a database.

[1773] Input: User's answer data

[1774] Output: Correct / incorrect result, saved in database

[1775] Specific operation: The user answers questions in the virtual space, inputs the answers, and sends them to the server. The server analyzes the answers, determines whether they are correct, and saves the results.

[1776] Step 6:

[1777] Analysis of learning trends and identification of weak areas

[1778] Server: Analyzes user answer data, identifies learning trends, and identifies specific areas of weakness.

[1779] Input: Correct / incorrect results, past answer data

[1780] Output: User learning tendency data, weak area data

[1781] Specific operation: The server runs an algorithm that analyzes learning trends based on the user's answer data stored in the database and extracts specific areas of weakness.

[1782] Step 7:

[1783] Focused questions and feedback

[1784] Server: Generates new questions based on the extracted weak areas and presents them to the user in a focused manner. In addition, it monitors the learning progress and provides appropriate feedback.

[1785] Input: Weakness Area Data

[1786] Output: New problem data, feedback message

[1787] Specific operation: The server generates new questions related to the weak areas and delivers them to the user's device. It also generates and provides feedback messages to the user based on their learning progress.

[1788] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1789] System Overview

[1790] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[1791] Importing and analyzing textbook data and past exam data

[1792] Importing textbook data

[1793] On the user's device: Textbook data and past exam question data can be photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1794] Server: Receives the imported textbook data. The received data is converted into text data using OCR technology. Special data such as mathematical formulas and graphs are also analyzed and saved in digital format.

[1795] Question generation and distribution

[1796] Problem Generation

[1797] Server: A generation AI generates new questions based on textbook data and past exam question data. For example, it generates questions based on the formula for solving quadratic equations.

[1798] Delivery in response to user requests

[1799] On the user's device: The user requests questions by specifying a specific subject and difficulty level, for example, selecting "Intermediate level math questions."

[1800] Server: In response to a user request, selects an appropriate problem and delivers it to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1801] Answer and learning trend analysis

[1802] Collecting and assessing answers

[1803] User's device: The user answers the delivered problem and sends the answer data to the server. For example, the user answers the problem "Find the solution to y = x^2 + 3x + 2" as "x = -1, -2".

[1804] Server: The submitted answer is evaluated by an automatic scoring algorithm to determine whether it is correct or not. For example, it determines that the user's answer "x = -1, -2" is correct.

[1805] Analysis of learning trends

[1806] Server: Analyzes the user's answer data and identifies areas of particular difficulty. For example, analyzes how well the user answers quadratic equation problems.

[1807] Focused questions and feedback on weak areas

[1808] Focused questions on weak areas

[1809] Server: Based on the user's learning tendencies, generate new problems that are specific to weak areas. For example, generate additional problems related to "quadratic equations."

[1810] On the user's device: Many questions related to weak areas are displayed, and the user can retry those questions.

[1811] Providing Feedback

[1812] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1813] User's device: Receives feedback from the server and adjusts the learning plan. For example, the user can set a new goal such as "strengthen memorization."

[1814] Incorporating an emotion engine

[1815] emotion recognition

[1816] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[1817] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[1818] Emotion-based problem adjustment

[1819] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[1820] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[1821] Providing emotion-based feedback

[1822] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[1823] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[1824] In the above-described manner, the present invention constructs a system that takes into account the emotional state of the learner and can provide a more personalized learning experience.

[1825] The processing flow will be explained below.

[1826] Step 1:

[1827] On the user's device: Textbook data and past exam question data are photographed or uploaded as a PDF or Doc file. For example, a user can take a photo of a textbook page with their smartphone and import it into the app as an image file.

[1828] Step 2:

[1829] Server: Receives the captured data. The received data is converted into text data using OCR technology. Special data such as formulas and graphs are also analyzed and saved in digital format.

[1830] Step 3:

[1831] Server: Analyzes the text data and extracts important keywords and themes, for example, keywords related to quadratic equations.

[1832] Step 4:

[1833] Server: Uses generative AI to generate new problems based on the extracted keywords and themes. For example, it generates a problem to find the solution to a quadratic equation.

[1834] Step 5:

[1835] On the user's device: The user selects a specific subject and difficulty level and requests a problem. For example, they select "Intermediate level math."

[1836] Step 6:

[1837] Server: Based on the user's request, selects appropriate problems and delivers them to the user's device. For example, select an intermediate level quadratic equation problem and send it to the user's device.

[1838] Step 7:

[1839] User's device: The delivered problem is displayed on the screen and the user answers it. For example, to the question "Find the solution to y = x^2 + 3x + 2," the user answers "x = -1, -2."

[1840] Step 8:

[1841] User's device: Sends the answer to the server.

[1842] Step 9:

[1843] Server: The server uses an automatic scoring algorithm to score the received answer and determine whether it is correct. For example, it determines that the user's answer "x = -1, -2" is correct.

[1844] Step 10:

[1845] Server: Collects user answer data and analyzes it to identify areas of weakness. For example, analyzes how well a user answers quadratic equation problems.

[1846] Step 11:

[1847] Server: Identifies the user's weak areas and generates new problems specific to those areas. For example, it generates additional problems related to quadratic equations.

[1848] Step 12:

[1849] On the user's device: Questions related to weak areas are displayed with emphasis, and the user is encouraged to retry those questions.

[1850] Step 13:

[1851] Server: Continuously monitors the user's learning progress and provides appropriate feedback, such as generating a message like "Your rate of correct answers to quadratic equations has improved to 80%."

[1852] Step 14:

[1853] On the user's device: Receives feedback from the server and adjusts the learning plan, for example, setting a new goal of "further improving in this area."

[1854] Step 15:

[1855] On the user's device: The built-in camera and microphone are used to capture the user's facial expressions and voice, and send them to the emotion engine. For example, the facial expressions and tone of voice are captured while the user is solving a problem.

[1856] Step 16:

[1857] Server: The emotion engine analyzes the received data and identifies the user's emotional state, for example, whether the user is focused, tired, or stressed.

[1858] Step 17:

[1859] Server: Adjust the difficulty and quantity of questions based on the user's perceived emotions. For example, if the user is tired, present more easy questions.

[1860] Step 18:

[1861] On the user's device: Tailored questions are presented, allowing the user to continue learning in a comfortable environment.

[1862] Step 19:

[1863] Server: Based on the user's emotional data, the server generates messages to encourage a break when concentration is low and to encourage the user when the user is feeling positive. For example, it displays a message such as, "You seem to be feeling a little tired. Let's take a five-minute break."

[1864] Step 20:

[1865] On the user's device: Receive emotion-based feedback and break prompts, allowing users to pace their learning as needed.

[1866] Example 2

[1867] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1868] Conventional learning support systems provide questions based on the user's learning progress and weak areas, but they lack the flexibility to consider the user's emotional state. This can lead to a decrease in the user's concentration and stress, potentially reducing the effectiveness of their learning. Furthermore, their ability to provide questions based on the user's specific requests is limited, leaving a need for more effective learning support.

[1869] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1870] In this invention, the server includes a means for importing textbook data and past exam question data, a means for analyzing the imported data and converting it into text data, a means for generating new questions using generative artificial intelligence, a means for collecting user answer data and determining whether the answers are correct, a means for analyzing the user's learning tendencies and identifying specific areas of weakness, a means for providing questions focused on the areas of weakness, a means for continuously monitoring the user's learning progress and providing feedback, and a means for recognizing the user's emotional state and reflecting it in learning activities. This enables personalized question provision and feedback that takes the user's emotional state into account, improving learning effectiveness. Furthermore, questions of specific subjects and levels of difficulty can be provided in response to the user's requests, supporting efficient learning.

[1871] "Textbook data" refers to digital data such as text information, mathematical formulas, and graphs contained in textbooks.

[1872] "Past exam question data" refers to information on exam questions that have been asked in the past, stored in digital format.

[1873] "Generative AI" is an AI technology that has the ability to generate new information and solutions based on input data.

[1874] "User answer data" refers to answer information entered by the user in response to the questions provided.

[1875] "Learning trends" refers to data that indicates specific patterns or progress observed in the user's learning process.

[1876] "Weakness areas" are areas of learning where the user does not fully understand or is less accurate in their answers.

[1877] "Feedback" refers to messages of encouragement and suggestions for improvement provided based on the user's learning progress and answer results.

[1878] "Emotional state" refers to the user's current mental and psychological state, such as whether they are focused, tired, or stressed.

[1879] A "camera" is a device that optically captures information and stores and transmits it as digital data.

[1880] A "microphone" is a device that captures sound and stores and transmits it as digital data.

[1881] MODE FOR CARRYING OUT THE INVENTION

[1882] This invention combines a learning support system for exam preparation with an emotion engine that recognizes the user's emotions. This system imports textbook data and past exam question data, analyzes it, converts it into text format, and automatically generates new questions using a generative AI. The user's answer data is collected and analyzed on a server, which identifies the user's weak areas and sets focused questions. The system also continuously monitors the user's learning progress and provides appropriate feedback. The emotion engine recognizes the user's emotions and reflects them in learning activities.

[1883] Importing and analyzing textbook data and past exam data

[1884] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The textbook page captured as an image file is saved in the app.

[1885] Examples:

[1886] The user takes a photo of a textbook page using their smartphone camera and saves it in the app.

[1887] The user uploads past exam questions in PDF format.

[1888] The server receives the textbook data sent from the user's device, converts it into text using OCR technology (such as Google's Tesseract OCR), analyzes special data such as formulas and graphs, and saves it in digital format.

[1889] Examples:

[1890] The server receives textbook images and converts them into text data using Tesseract OCR.

[1891] Special formulas and graphs are analyzed and saved using libraries such as MathJax.

[1892] Question generation and distribution

[1893] The server generates new questions using a generative AI (e.g., OpenAI's GPT-4) based on textbook data and past exam question data. It generates questions by inputting question generation prompts into the AI ​​model.

[1894] Examples:

[1895] The AI ​​is instructed to generate a problem based on the formula for solving a quadratic equation, and the generated problem is saved in text format.

[1896] The user's device requests questions by specifying a specific subject and difficulty level. The user selects the "Mathematics" subject and the "Intermediate" level in the app interface.

[1897] Examples:

[1898] The user selects and requests "Intermediate level math problems."

[1899] The server selects appropriate questions in response to a user's request and delivers them to the user's terminal.

[1900] Examples:

[1901] The server searches for "intermediate level quadratic equation problems" and sends the corresponding problems to the user's terminal.

[1902] Answer and learning trend analysis

[1903] The user answers the questions delivered to the user's terminal and transmits the answer data to the server.

[1904] Examples:

[1905] The user answers the question "Find the solution to y = x^2 + 3x + 2" with "x = -1, -2", and the app collects the answer and sends it to the server.

[1906] The server uses an automatic scoring algorithm to evaluate the submitted answers and determine whether they are correct or not.

[1907] Examples:

[1908] The server receives the user's answer "x = -1, -2" and uses an automatic scoring algorithm to determine that this is the correct answer.

[1909] The server analyzes the user's answer data and identifies areas in which the user is particularly weak.

[1910] Examples:

[1911] The server analyzes the user's answer history and identifies a low rate of correct answers for problems involving quadratic equations.

[1912] Focused questions and feedback on weak areas

[1913] The server generates new questions that are specific to the user's weak areas based on the user's learning tendencies.

[1914] Examples:

[1915] The server inputs a new problem related to "quadratic equations" as a prompt to the generation AI and sends the generated problem to the user's device.

[1916] The user's device will display many questions related to areas in which they are weak, and they can try those questions again.

[1917] Examples:

[1918] Many additional practice problems related to "quadratic equations" will appear on the user's smartphone.

[1919] The server continuously monitors the user's learning progress and provides appropriate feedback.

[1920] Examples:

[1921] The server generates a message such as "Your quadratic equation accuracy rate has improved to 80%."

[1922] The user's device receives feedback from the server and adjusts the learning plan.

[1923] Examples:

[1924] The user reads the feedback message and sets a new goal, such as "strengthen memorization."

[1925] Incorporating an emotion engine

[1926] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice and transmits them to the emotion engine.

[1927] Examples:

[1928] While the user is solving the problem, the device captures the user's face with a camera and also records their voice.

[1929] The server analyzes the received data using an emotion engine to identify the user's emotional state.

[1930] Examples:

[1931] The server analyzes the received data and determines whether the user is "concentrated" or "tired."

[1932] The server adjusts the difficulty and quantity of questions based on the recognized user emotions.

[1933] Examples:

[1934] If the user is determined to be "tired," the server will present more slightly easier questions.

[1935] The user's device is presented with adjusted questions, allowing the user to continue learning in a comfortable environment.

[1936] Examples:

[1937] Easy questions are displayed so that users can relax and work through the questions.

[1938] Based on the user's emotional data, the server generates messages encouraging them to take a break when their concentration is low and encouraging them when they are feeling positive.

[1939] Examples:

[1940] The server generates a message such as "You seem to be feeling a little tired. Let's take a 5-minute break" and sends it to the user's device.

[1941] The user's device receives emotion-based feedback and prompts to take breaks, allowing the user to adjust the pace of their learning as needed.

[1942] Examples:

[1943] The user checks the feedback message and takes a few minutes to regain focus.

[1944] Examples of prompt statements

[1945] 1. Upload a PDF file and generate a quadratic equation problem:

[1946] Prompt: "Generate three quadratic equation problems based on Chapter 3 of this PDF file."

[1947] 2. Generate additional questions based on the user's weak areas:

[1948] Prompt: "Based on your answer history, you are having difficulty with problems involving quadratic equations. Please generate another practice problem."

[1949] 3. Generating encouraging messages based on emotion data:

[1950] Prompt: "The user is feeling fatigued. Please generate a message encouraging them to take a break."

[1951] This allows the present invention to provide a tailored learning experience that takes into account the learner's emotional state and learning progress.

[1952] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1953] Step 1: Importing textbook data

[1954] The user's device acquires textbook data and past exam question data. This is achieved by the user taking a photo of a textbook page with the device's camera or uploading it as a PDF or Doc file. The app receives image files and PDF files as input and saves the data in the app. Specific operations include the user taking a photo of a textbook page with the smartphone camera and importing the image file into the app.

[1955] Input: Textbook image file or PDF file

[1956] Output: Textbook data saved in the app

[1957] Step 2: Analyzing textbook data

[1958] The server receives textbook data sent from the user's device. It converts the received data into text data using OCR technology (for example, Google's Tesseract OCR), and analyzes special data such as formulas and graphs before saving them in digital format. The input is image data or PDF files sent from the user's device, and the output is text data.

[1959] Input: Image data or PDF file from the device

[1960] Output: Text data stored on the server

[1961] Step 3: Generate the problem

[1962] The server generates new questions using a generative AI (for example, OpenAI's GPT-4) based on textbook data and past exam question data. It receives prompt text and analyzed text data as input and generates new questions based on that. Specifically, the server sends a prompt for a "problem based on the formula for solving a quadratic equation" to the generative AI and saves the gene...

Claims

1. A means of importing textbook data and past exam question data, A means for analyzing the captured data and converting it into text data; A means for generating new problems using generative artificial intelligence; A means for collecting user answer data and determining whether the answer is correct or incorrect; A means for analyzing a user's learning tendencies and extracting specific weak areas; A method to focus questions based on weak areas, A system that includes a means for continuously monitoring a user's learning progress and providing feedback.

2. 2. The system according to claim 1, further comprising means for selecting and distributing questions according to a specific subject or level of difficulty based on a user's request.

3. 10. The system of claim 1, further comprising means for analyzing and storing in digital form specialized data such as formulas and graphs.

Images