System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting homework content by a learner; means for analyzing the input homework content and classifying the homework content into a specific learning category; means for generating a game-form question according to the classified learning category and transmitting the game-form question to a terminal; means for displaying the game-form question received by the terminal and allowing the learner to interactively answer the question; means for evaluating the learner's answer and presenting a result; and means for recording the learner's progress and managing a learning history.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional homework and learning activities often do not interest learners, making it difficult for them to maintain concentration. Furthermore, the learning content may not stick in their heads, resulting in less effective learning. This often leads to concerns about a decline in motivation to learn and a decline in learning outcomes. Particularly for elementary and junior high school students, completing homework can be a pain, making it difficult for them to establish good study habits. [Means for solving the problem]

[0005] The present invention is a system that includes a means for a learner to input homework content, a means for analyzing the input homework content and classifying it into specific learning categories, a means for generating game-style questions according to the classified learning categories and sending them to a terminal, a means for displaying the game-style questions received by the terminal and allowing the learner to answer them interactively, a means for evaluating the learner's answers and presenting the results, and a means for recording the learner's progress and managing the learner's learning history. This allows the learner to enjoy working on their homework as if they were completing a game, making it easier for them to maintain concentration and effectively absorb the learning content. The system also suggests next learning content and questions based on the learner's answers, and provides rewards such as points and badges for correctly answered questions, further increasing the learner's motivation to learn.

[0006] "Learner" refers to an actor who is a child, student or other learner who engages in educational activities.

[0007] "Homework content" refers to the specific tasks, problems, or learning scope of the homework assigned to the learner.

[0008] "Input means" refers to the interface or device that the learner uses to input their homework content into the system, including, for example, a keyboard, touchscreen, or voice input.

[0009] "Means of analysis" refers to algorithms and programs, particularly technologies that use natural language processing, to analyze and understand the input homework content.

[0010] A "study category" is a group or category of various study contents (e.g., mathematics, geography, kanji, English, etc.) for classifying homework contents.

[0011] "Game-style questions" are question formats that present learning content in a way that incorporates game elements, and include quizzes, puzzles, flash cards, etc.

[0012] "Terminal" means a device used by a User to access the System and answer game-style questions, including a smartphone, tablet, or PC.

[0013] "Display means" refers to the technology and related interface that enables the terminal to visually present game-style questions to the learner.

[0014] "Solution tools" refers to interfaces or devices that allow learners to enter or select answers to game-style questions.

[0015] "Means of assessment" refers to the programs or algorithms used to judge learners' answers and determine whether they are correct or incorrect.

[0016] "Means of presenting results" refers to the technology and interface used to present learners with information such as whether their answers were correct or incorrect, their score, and the next question.

[0017] "Means for recording progress" refers to the technology and programs used to store and track a learner's learning status and answer history in a database.

[0018] "Means for managing learning history" refers to a system for organizing, storing, and analyzing the history of a learner's past learning activities and managing learning outcomes.

[0019] "Rewards" refers to incentives such as points, badges, and titles provided to learners based on correct answers and learning progress. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention is a system that allows learners to enjoyably complete their homework in a game-like manner. This system analyzes the learner's input, generates appropriate game-style questions based on that input, and supports the learner in the process of answering them. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[0042] Program processing

[0043] Students input and submit their homework

[0044] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0045] Analysis and classification of homework content by the server

[0046] The server then analyzes the received input, using natural language processing (NLP) techniques to analyze the text and classify the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.), and selects appropriate game-style problems based on the classification results.

[0047] Game-style question generation and submission

[0048] The server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[0049] Terminal display of questions and interactive solutions

[0050] The device displays the game-style questions received from the server on the screen. The learner looks at this screen and inputs the answer to each question. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[0051] Server-based answer evaluation and feedback

[0052] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0053] Track progress and provide rewards

[0054] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, rewards such as points and badges are provided via the device for correctly answered questions, which increases the learner's motivation.

[0055] Specific examples

[0056] 1. Math homework

[0057] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0058] 2. English homework

[0059] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[0060] In this way, the present invention is a system that converts homework into a game format, thereby increasing learners' concentration and motivation to study and promoting effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The user inputs the homework content.

[0064] The user uses the device to enter the homework content into the text box and clicks the "Convert homework to game" button, which then sends the input data to the server.

[0065] Step 2:

[0066] The server receives the homework content.

[0067] The server receives the homework content sent from the terminal and prepares it for analysis as text data.

[0068] Step 3:

[0069] The server analyzes the homework content and classifies it into categories.

[0070] The server uses natural language processing (NLP) technology to analyze the input text and, based on the analysis results, classifies the homework content into specific learning categories (e.g., math problems, geography problems, kanji problems, English problems, etc.).

[0071] Step 4:

[0072] The server will choose the appropriate game format.

[0073] The server selects an appropriate game format (e.g., quiz format, puzzle format, flash card format, etc.) from the database according to the classified learning category.

[0074] Step 5:

[0075] The server generates game-style questions.

[0076] The server generates a set of questions based on the selected game format. For example, if the game is a math calculation game, a quiz containing 10 arithmetic operations will be generated.

[0077] Step 6:

[0078] The server sends the generated game to the device.

[0079] The server sends the generated game data to the terminal, which receives this data.

[0080] Step 7:

[0081] The device displays a game-style question.

[0082] The device displays the received game data on the screen, presenting the user with questions and answer options.

[0083] Step 8:

[0084] The user answers the question.

[0085] The user inputs or selects an answer to the question displayed on the screen. For example, if it is a calculation question, the user inputs the answer in the input field.

[0086] Step 9:

[0087] The device sends the answer to the server.

[0088] The terminal transmits the user's answer to the server.

[0089] Step 10:

[0090] The server evaluates the answers.

[0091] The server compares the received answers with pre-set correct answer data, determines whether they are correct, and returns the evaluation results to the terminal.

[0092] Step 11:

[0093] The terminal displays the evaluation results.

[0094] The terminal displays the evaluation results received from the server to the user. If the answer is correct, the user has the option to proceed to the next question, and if the answer is incorrect, the user has the option to try again.

[0095] Step 12:

[0096] The server records the progress data.

[0097] The server records the user's answer data and progress data and updates the learning history.

[0098] Step 13:

[0099] The terminal provides the reward.

[0100] The device provides users with rewards such as points and badges for correctly answering questions.

[0101] Step 14:

[0102] The server suggests what to learn next.

[0103] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and sends them to the device, which then displays these suggestions to the user.

[0104] Through the above processing steps, the user can enjoyably complete their homework in a game-like format, and can effectively master the learning content.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Traditional homework learning methods are monotonous and burdensome for learners. This can lead to problems such as reduced motivation to learn and difficulty in effective learning. Furthermore, there is a lack of progress management of learning content and feedback tailored to individual needs, which can lead to reduced learning efficiency. Furthermore, since no rewards are offered, maintaining motivation is also an issue.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes: a means for a learner to input homework content using a terminal; a means for analyzing the input homework content using natural language processing technology and classifying it into specific learning categories; a means for generating game-style questions corresponding to the classified learning categories using a game engine and sending them to the terminal; a means for interactively displaying the game-style questions received by the terminal and allowing the learner to input answers; a means for comparing the learner's answers with a database to evaluate whether they are correct or incorrect and presenting the results; a means for recording the learner's answer data and progress data and managing the learning history; and a means for generating rewards for the learner and sending them to the terminal for display. This allows the learner to enjoyably complete their homework in a game-like format, improving their motivation to learn and promoting effective learning. Furthermore, managing the learning history and providing rewards can help maintain motivation to learn.

[0110] A "terminal" is an electronic device used by a learner to enter homework content, and examples include tablets and personal computers.

[0111] "Natural language processing technology" is a computer technology that analyzes input text data and understands specific meanings and contexts, and uses NLP libraries and algorithms.

[0112] A "learning category" is a category for classifying homework content into specific fields or types, and examples include calculation problems, Kanji, English, etc.

[0113] A "game engine" is a software platform for generating game-style problems, and examples include Unity and Unreal Engine.

[0114] "Interactive" refers to a function that allows learners to directly operate and input information through their terminal.

[0115] A "database" is a system that stores learners' answer data and learning history, and collates and manages the data as needed.

[0116] "Rewards" are incentives such as points or badges that are provided to motivate learners when they answer questions correctly.

[0117] "Progress data" is data used to record the results of questions answered by a learner and the answer status, and to track the progress of learning.

[0118] MODE FOR CARRYING OUT THE INVENTION

[0119] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner. The specific processing content of the program of this system and the hardware and software used will be described in detail below.

[0120] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. Specifically, the learner enters the homework content in the text box displayed on the device screen and clicks the "Convert homework into a game" button.

[0121] This input is sent from the device to a server, which then analyzes the received homework content using natural language processing (NLP) techniques. Suitable NLP libraries to use are "spaCy" and "NLTK." The text data is analyzed, and the homework content is classified into specific learning categories (e.g., math problems, Kanji, English, etc.).

[0122] The server then generates game-style questions based on the classified learning categories. The software used for this is a game engine such as Unity or Unreal Engine. For example, a quiz-style game data set containing 10 arithmetic calculation questions is generated.

[0123] The generated game data is sent from the server to the user's device. The device receives this game data and displays it on the screen in an interactive format. The learner can answer questions and enter their answers through the on-screen interface. The learner's answer data is then sent back to the server from the device.

[0124] The server checks the received answers against pre-set correct answer data using a database. For example, a database system such as MySQL is used. The evaluation results are sent back to the device and displayed to the learner. If the learner answers correctly, they can proceed to the next question, but if they answer incorrectly, they are given the option to try again.

[0125] Furthermore, the server records the learner's answer data and progress data, and uses BaaS (Backend as a Service) such as "Firebase" to update the learning history. This data is used when generating and suggesting questions for the next learning session. In addition, rewards such as points and badges are generated for correctly answered questions and sent to the device. The device displays reward information to the learner in real time, increasing their motivation to learn.

[0126] Specific examples

[0127] 1. Math homework

[0128] The learner types in "Today's math homework is a calculation problem." The server analyzes this and classifies it into the calculation problem category. The server generates a quiz-style game containing 10 arithmetic operation problems. The learner answers the questions displayed on the device one by one. The server evaluates the answers and displays the results on the device. When the learner answers all the questions, they are awarded points that are reflected in the next learning material.

[0129] Prompt Sentence Examples

[0130] Today's math homework is a calculation problem. Turn it into a game.

[0131] 2. English homework

[0132] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. The server then generates a puzzle-style game in which the learner answers the meaning of the word. The learner answers the puzzle displayed on the terminal by pairing words with their meanings. The server evaluates the answers, and if all pairs are correct, a reward is provided.

[0133] Prompt Sentence Examples

[0134] Today's English homework is to memorize the meanings of words. Turn this into a game.

[0135] In this way, the present invention automatically analyzes homework content and provides it in a game format, thereby increasing learners' motivation and supporting effective learning. By utilizing the entire system, the effectiveness of home study and online classes can be maximized.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1:

[0138] The user uses the device to input the homework content. Specifically, the user inputs the homework content into the text box displayed on the device screen and clicks the "Convert homework to game" button. For example, the user might input "Today's math homework is a calculation problem." This input content is sent from the device to the server. The input data is the homework content in text format, and this is output to the server.

[0139] Step 2:

[0140] The server analyzes the received homework content. Specifically, it uses natural language processing (NLP) technology to analyze the input text. Suitable NLP libraries to use are "spaCy" and "NLTK." Through this analysis, the server classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). The input data is the transmitted text data, and the output data is the classification results.

[0141] Step 3:

[0142] The server generates game-style questions based on the classified learning categories. Specifically, it uses a game engine such as Unity or Unreal Engine to create an interactive game. For example, it generates quiz-style game data containing 10 calculation questions on arithmetic operations. The input data is the learning categories, and the output data is the generated game data.

[0143] Step 4:

[0144] The server sends the generated game data to the user's device. This causes the user's device to receive game-style questions. The input data is the generated game data, and the output data is a notification to the device that transmission has been completed. Specifically, the server divides the game data into packets and sends them over the network.

[0145] Step 5:

[0146] The terminal displays the received game-style questions on the screen. The user can answer the questions through the on-screen interface. For example, in the case of a calculation question, the user looks at the calculation formula displayed on the screen and enters the answer in the answer input field. The input data is the received game data, and the output data is the user's answer input.

[0147] Step 6:

[0148] The terminal sends the user's answer data to the server. Specifically, the answer entered by the user is divided into packets and sent to the server. The input data is the user's answer data, and the output data is a transmission completion notification to the server.

[0149] Step 7:

[0150] The server evaluates the received answer data. Specifically, it compares it with pre-set correct answer data using a database. A suitable database to use is MySQL. The input data is the user's answer data, and the output data is the evaluation result.

[0151] Step 8:

[0152] The server returns the evaluation results to the user's terminal, which then displays the results to the user. The input data is the evaluation results, and the output data is a transmission completion notification to the terminal. Specifically, the success / failure information is divided into packets and sent to the terminal.

[0153] Step 9:

[0154] The server records the user's progress data and learning history. Specifically, it uses BaaS such as "Firebase" to update the learning history database. This data is used to generate and suggest questions for the next learning session. The input data is the learner's answer data and progress data, and the output data is the updated learning history.

[0155] Step 10:

[0156] The server generates rewards for learners and sends them to the device. Specifically, rewards such as points and badges are generated for questions that the learner answers correctly. The input data is the evaluation results, and the output data is the generated reward data. The device displays the reward information in real time and notifies the learner.

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] In conventional educational systems, there are problems such as learners finding it difficult to maintain their motivation while completing their homework. Also, there are issues with the lack of ways to attract customers' attention when obtaining product information in physical stores. This can lead to a decrease in learning effectiveness and a lack of understanding of products.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes means for a learner to input homework content, means for analyzing the input homework content and classifying it into a specific learning category, means for generating game-style questions according to the classified learning category and transmitting them to the terminal, means for a customer to input product information, means for analyzing the input product information and classifying it into a specific product category, and means for generating game-style information presentations according to the classified product category and transmitting them to the terminal. This allows learners to enjoy doing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[0162] "Learner" means a person who uses an educational system to carry out learning activities.

[0163] "Homework content" refers to the learning tasks assigned to students by their school or teaching materials.

[0164] "Input means" refers to the interface through which a learner or customer provides information to the system.

[0165] "Analysis means" refers to technology for processing input information and understanding its content.

[0166] "Learning categories" are criteria for classifying educational content into specific themes or subjects.

[0167] "Game-style questions" are questions that include game elements in order to enhance learning effectiveness.

[0168] "Device" means an electronic device (e.g., tablet, smartphone, smart glasses) used by a learner or customer.

[0169] A "display means" is a method for visually presenting learning or information content on a terminal.

[0170] "Means for interactive answering" means a method by which a learner or customer can input answers through a terminal and interact with the system in a two-way manner.

[0171] "Means for evaluating answers" refers to a system for evaluating the answers of learners or customers and determining whether they are correct or incorrect.

[0172] The "means of presenting results" is the method by which the results of the assessment are communicated to the learner or client.

[0173] A "means for recording progress" is a system that stores learner or customer activity data.

[0174] "Means for managing learning history" refers to a system for effectively organizing recorded progress data and utilizing it for future learning or information provision.

[0175] "Customer" means a person who uses the system to obtain product information in a physical store.

[0176] "Product information" refers to information about the features and usage of products sold in physical stores.

[0177] A "product category" is a category in which products are classified according to specific criteria.

[0178] "Information presentation" refers to a method for displaying information about a product to a customer.

[0179] System Overview

[0180] This invention provides a system that allows learners and customers in brick-and-mortar stores to learn and acquire product information in an interactive game format. This system uses devices such as tablets, smartphones, and smart glasses, and processes and provides information in cooperation with a server.

[0181] Hardware and software used

[0182] Hardware:

[0183] Tablet (e.g. iPad)

[0184] Smartphone (e.g. iPhone or Android device)

[0185] Smart glasses (e.g. Google Glass)

[0186] software:

[0187] Natural Language Processing (NLP) libraries (e.g., SpaCy, NLTK)

[0188] Game engines (e.g. Unity, Cocos2d)

[0189] Backend server (e.g. AWS Lambda, Django Rest Framework)

[0190] Explanation of program processing

[0191] Step 1: Learner or customer enters information

[0192] Users input homework details and product information using devices (tablets, smartphones, etc.). Specifically, for example, a student might input "Today's math homework is a calculation problem," while a customer at a physical store might input "I want to know about cosmetics."

[0193] Step 2: Server analysis and classification

[0194] The server uses natural language processing (NLP) technology to analyze the input text and classify it into appropriate learning or product categories, such as "math problems" or "cosmetics."

[0195] Step 3: Generate and send game-style questions and information prompts

[0196] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and the generated content is sent to the terminal.

[0197] Step 4: Terminal display and interaction

[0198] The terminal displays game-style questions and information received from the server on its screen. The learner or customer looks at this screen and inputs answers to each question. For example, if it is a calculation question, the learner answers the formula displayed on the screen, and if it is cosmetic information, the learner answers the related quiz.

[0199] Step 5: Server evaluation and feedback of answers

[0200] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0201] Step 6: Track progress and provide rewards

[0202] The server records the learner's or customer's answer data and progress data, and updates the learning history and customer history. This data is used to generate and suggest questions for the next learning session, as well as to provide product information. Correct answers to questions can provide rewards such as points or coupons.

[0203] Specific examples

[0204] 1. Math homework

[0205] A learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time a learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0206] 2. Cosmetics information

[0207] A customer types in "I want to know more about cosmetics." The server analyzes this and classifies it into a cosmetics category. A quiz-style information presentation about the product's ingredients and usage is generated and displayed on the terminal. When the customer answers the quiz, their answers are evaluated and if they are correct, they are given a coupon as a reward.

[0208] Prompt Sentence Examples

[0209] I entered "I want to know more about cosmetics." Based on this information, please generate the following trivia and quiz questions:

[0210] 1. Cosmetic Ingredients Quiz

[0211] 2. Best Practices for Usage

[0212] 3. Product reviews and recommended usage

[0213] This system allows learners to enjoy completing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] Learner or customer input of information

[0217] The user inputs homework details and product information using a device (tablet, smartphone, etc.).

[0218] Input: A learner types, "Today's math homework is a calculation problem." Or a customer types, "I want to know about cosmetics."

[0219] Data processing: Obtain the input text data and send it to the server.

[0220] Output: The text data sent to the server.

[0221] Step 2:

[0222] Server-based information analysis and classification

[0223] The server uses natural language processing (NLP) techniques to analyze the input text and classify it into appropriate learning and product categories.

[0224] Input: Text data (homework assignments or product information).

[0225] Data processing: Analyze the text using an NLP library (e.g., SpaCy, NLTK) to extract specific keywords and phrases. Then, classify the text into learning or product categories based on these keywords.

[0226] Output: Category information (e.g., "Math Problem", "Cosmetics").

[0227] Step 3:

[0228] Generate and send game-style questions and information presentations

[0229] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and transmits them to the terminal.

[0230] Input: Category information (e.g., "Math Problems," "Cosmetics").

[0231] Data processing: Using a game engine (e.g. Unity, Cocos2d), create game-style questions and information presentations according to the category.

[0232] Output: Generated game-style questions and information presentation data. Send to device.

[0233] Step 4:

[0234] Terminal display and interaction

[0235] The terminal displays game-style questions and information received from the server on its screen, and the learner or customer inputs answers to each question.

[0236] Input: Game-style questions and information presentation data sent from the server.

[0237] Data processing: The device analyzes the data received and displays it on the user interface.

[0238] How it works: The learner or customer enters an answer to a question, and the device sends the answer to the server.

[0239] Output: User's answer data.

[0240] Step 5:

[0241] Server-based answer evaluation and feedback

[0242] The server receives the user's answer, compares it with pre-set correct answer data, evaluates its accuracy, and returns the evaluation result to the terminal.

[0243] Input: User's answer data.

[0244] Data processing: The server checks the answer data against the correct answer database to determine whether it is correct or incorrect, and generates a feedback message indicating whether the answer is correct or incorrect.

[0245] Output: A feedback message (e.g., "Correct" if correct, "Incorrect" if incorrect) sent back to the device.

[0246] Step 6:

[0247] Track progress and provide rewards

[0248] The server records the answer data and progress data of the learner or customer, updates the learning history and customer history, and provides rewards (points or coupons) for correct answers.

[0249] Input: User answer data, feedback message.

[0250] Data processing: The server records the answer results in the progress management system and updates the user's history. If the answer is correct, points or coupons are generated using the reward system.

[0251] Output: Updated learning history, customer history data, reward data (points and coupons). Send to the terminal.

[0252] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0253] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and also has the function of recognizing the learner's emotions and optimizing the learning process. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[0254] Program processing

[0255] Students input and submit their homework

[0256] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0257] Analysis and classification of homework content by the server

[0258] The server analyzes the received input, uses natural language processing (NLP) technology to analyze the text, and classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[0259] Game-style question generation and submission

[0260] Next, the server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[0261] Terminal display of questions and interactive solutions

[0262] The device displays the game-style questions received from the server on the screen. The learner inputs the answers to each question while looking at this screen. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[0263] Server-based answer evaluation and feedback

[0264] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0265] Emotion recognition and regulation with emotion engine

[0266] To recognize learners' emotions, the system uses a camera and microphone on the device to capture their facial expressions and voice data. An emotion engine on the server analyzes this data and recognizes the learner's emotions (e.g., joy, anger, sadness, excitement, etc.).

[0267] Based on the recognized emotions, the server can adjust the difficulty and content of the gamified questions appropriately. For example, if the learner is tired or confused, it can lower the difficulty of the questions or provide encouraging messages.

[0268] Track progress and provide rewards

[0269] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges via the device for correctly answered questions, thereby increasing learner motivation.

[0270] Specific examples

[0271] 1. Math homework

[0272] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0273] Additionally, if the system recognizes that a learner is confused based on their facial expression, it will support them by lowering the difficulty of the questions or displaying encouraging messages.

[0274] 2. English homework

[0275] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[0276] Additionally, if the system recognizes from the learner's voice data that the learner is enjoying the lesson, it can add more challenging questions while maintaining the current level of difficulty.

[0277] This system converts homework into a game format, and by appropriately recognizing and adjusting learners' emotions, it increases their motivation to learn and promotes effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[0278] The processing flow will be explained below.

[0279] Step 1:

[0280] The user inputs the homework content.

[0281] The user uses the device to enter specific homework content into the text box and clicks the "Convert homework to game" button, which is then sent from the device to the server.

[0282] Step 2:

[0283] The server receives the homework content.

[0284] The server receives the homework content sent from the terminal and prepares for analysis.

[0285] Step 3:

[0286] The server analyzes the homework content and classifies it into learning categories.

[0287] The server uses natural language processing (NLP) technology to analyze the homework content and classify it into specific learning categories (e.g., math problems, Kanji, English, etc.).

[0288] Step 4:

[0289] The server will choose the appropriate game format.

[0290] The server selects an appropriate game format (e.g., quiz, puzzle, flashcard, etc.) from the database according to the classified category.

[0291] Step 5:

[0292] The server generates game-style questions.

[0293] The server generates a set of questions in the selected game format. For example, if the questions are math calculation questions, it generates a quiz that includes arithmetic operations.

[0294] Step 6:

[0295] The server sends the generated game to the device.

[0296] The server sends the generated game data to the terminal, which receives this data.

[0297] Step 7:

[0298] The device displays a game-style question.

[0299] The device displays the received game-style questions on the screen, allowing the user to interactively answer them.

[0300] Step 8:

[0301] The device collects the user's emotional data.

[0302] As the user works on the problem, the device uses a camera and microphone to collect the user's facial expressions and voice, and this data is sent to a server for analysis by the emotion engine.

[0303] Step 9:

[0304] The user answers the question.

[0305] The user inputs the answer to the question displayed on the screen. For example, if it is a calculation question, the answer is entered in the answer field. The input answer is sent from the terminal to the server.

[0306] Step 10:

[0307] The server evaluates the answers.

[0308] The server receives the user's answer, judges whether it is correct, compares it with the correct answer data set in advance, and returns the evaluation result to the terminal.

[0309] Step 11:

[0310] The terminal displays the evaluation results.

[0311] The device receives the evaluation results from the server and displays them to the user. If the answer is correct, the user is given the option to proceed to the next question, and if the answer is incorrect, the user is given the option to try again.

[0312] Step 12:

[0313] The server analyzes the user's emotional data.

[0314] The server's emotion engine analyzes the received facial and voice data and recognizes the user's emotions (e.g., joy, anger, sadness, confusion, etc.).

[0315] Step 13:

[0316] The server adjusts the game content.

[0317] Based on the recognized emotions, the server adjusts the difficulty and content of the game. For example, if the user is having trouble, it may lower the difficulty of the problems or send a message of encouragement.

[0318] Step 14:

[0319] The server records the progress data.

[0320] The server records the user's answers and progress data, and updates the learning history, which is used to generate questions and suggestions for the next learning session.

[0321] Step 15:

[0322] The terminal provides the reward.

[0323] The device provides users with rewards such as points and badges for correctly answering questions.

[0324] Step 16:

[0325] The server suggests what to learn next.

[0326] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and emotional data, and sends them to the device, which then displays them to the user.

[0327] The above processing steps allow users to enjoy completing their homework in a game-like manner, effectively mastering the learning content. Furthermore, the introduction of an emotion engine optimizes the learning process for each individual user, which is expected to increase their motivation to learn.

[0328] Example 2

[0329] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] Conventional learning support systems often progress homework in a monotonous manner, which can easily lower learners' motivation. Furthermore, they provide problems with a uniform level of difficulty without taking learners' emotions into consideration, which results in the problem of not maximizing learning effectiveness. Furthermore, they do not provide appropriate feedback or adjustments, which results in lower learning efficiency.

[0331] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a learner to input homework content, a means for analyzing the input homework content and classifying it into a specific learning category, a means for generating game-style questions according to the classified learning category and transmitting them to the terminal, a means for displaying the game-style questions received by the terminal so that the learner can answer them interactively, a means for evaluating the learner's answers and presenting the results, a means for recording the learner's progress and managing the learning history, and a means for recognizing the learner's emotions and adjusting the difficulty and content of the questions. This enables the learner to actively engage in learning while having fun, thereby realizing effective learning support.

[0332] "Student" refers to an entity that uses the system to carry out learning activities.

[0333] "Means for inputting homework content" refers to the input devices and software functions that allow learners to input homework content into the system.

[0334] "Means for analyzing homework content and classifying it into specific learning categories" refers to a software function that analyzes the input homework content using natural language processing technology, etc., and assigns the content to specific learning categories.

[0335] "Means for generating game-style questions and sending them to the device" refers to a software function for automatically creating appropriate game-style questions based on the analyzed homework content and sending the data to the learner's device.

[0336] "Means for interactive answering" refers to software or hardware that allows learners to input answers to questions posed on their devices in real time.

[0337] "Means for evaluating answers and presenting results" refers to software functionality for automatically evaluating a learner's answers and notifying the learner of the evaluation results.

[0338] "Means for recording progress and managing learning history" refers to software functions that record learners' answers and progress in a database and manage their learning history based on that.

[0339] "Means for recognizing emotions and adjusting the difficulty and content of questions" refers to software functions that analyze data such as the learner's facial expressions and voice to recognize their emotions, and dynamically change the difficulty and content of questions based on the results.

[0340] "Means for providing rewards such as points or badges" refers to software functionality that automatically awards rewards such as points or badges to learners for correctly answering questions.

[0341] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and further has the function of recognizing the learner's emotions and optimizing the learning process. Specific embodiments for carrying out the present invention will now be described.

[0342] System configuration

[0343] Students input and submit their homework

[0344] The learner, who is the user, inputs the homework content using a device such as a tablet or PC. The learner enters "Today's math homework is a calculation problem" in the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0345] Analysis and classification of homework content by the server

[0346] The server uses natural language processing (NLP) technology to analyze the received homework content. Specifically, the NLTK library in Python can be used. The server analyzes the input text and classifies the content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[0347] Game-style question generation and submission

[0348] The server generates game-style questions according to the selected learning category. The Django framework can be used for this generation. For example, in the case of math calculation questions, a quiz-style game containing multiple calculation questions is generated. The generated game data is sent from the server to the device.

[0349] Terminal display of questions and interactive solutions

[0350] The device displays game-style questions received from the server on the screen. The learner looks at the displayed questions and enters their answers. For calculation questions, the learner looks at the calculation formula displayed on the screen and enters the answer in the answer input field. Once the learner has finished entering their answer, the data is sent from the device to the server.

[0351] Server-based answer evaluation and feedback

[0352] The server receives the learner's answer data and compares it with pre-set correct answer data to evaluate whether it is correct or incorrect. An SQL database can be used for this evaluation. The server generates an evaluation result and sends it back to the device for presentation to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[0353] Emotion recognition and regulation with emotion engine

[0354] To recognize learners' emotions, the system uses the device's camera and microphone to capture their facial expressions and voice data. This data is sent to a server where it is analyzed by an emotion engine within the server. Specific technology used is the Microsoft Azure Emotion API. The emotion engine analyzes the learner's emotions (e.g., joy, sadness, anger, excitement, etc.) and adjusts the difficulty and content of the questions based on the results.

[0355] Track progress and provide rewards

[0356] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, the server can provide rewards such as points and badges for questions that the learner answers correctly, which helps to improve the learner's motivation.

[0357] Specific examples

[0358] 1. Math homework

[0359] When a learner types "Today's math homework is a calculation problem," the device sends this information to the server. The server uses NLP technology to recognize the category "calculation problem." It then generates a quiz-style game and sends it to the device. The device displays it, and when the learner enters an answer, the answer data is sent to the server. The server evaluates whether the answer is correct or incorrect and sends the evaluation result back to the device to show it to the learner. If the learner looks confused, the difficulty of the problem is automatically adjusted and a supportive message is displayed.

[0360] Example inputs to a generative AI model:

[0361] User: "Today's math homework includes calculation problems."

[0362] System: "Recognizes calculation problem categories and generates 10 arithmetic operation problems."

[0363] Server: "Evaluate the issue and provide feedback"

[0364] 2. English homework

[0365] When a learner types "Today's English homework is to memorize the meanings of words," the device sends this information to the server. The server uses NLP technology to recognize the "word learning" category. It then generates a puzzle-style game in which the learner must guess the meaning of the word and sends it to the device. When the learner types their answer, the data is sent to the server, which evaluates it and provides feedback. If the server recognizes that the learner is enjoying the game, it adds more challenging questions at the same difficulty level.

[0366] Example inputs to a generative AI model:

[0367] User: "Today's English homework is to memorize the meanings of words."

[0368] System: "Recognizes word learning categories and generates puzzle-style games"

[0369] Server: "Evaluate the issue and provide feedback"

[0370] Thus, the present invention is an optimal learning support system that enables learners to study in an enjoyable and effective manner.

[0371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0372] Step 1:

[0373] The user inputs the homework details using a device such as a tablet or PC. The specific action is to enter "Today's math homework is a calculation problem" into the text box and click the "Convert homework into a game" button. The entered text data is sent from the device to the server. The input of this step is the text entered by the user, and the output is the text data sent to the server.

[0374] Step 2:

[0375] The server analyzes the received text data of the homework content using natural language processing technology. Specifically, it uses Python's NLTK library to perform morphological analysis and classify the homework content into specific learning categories. The input for this step is the text data submitted by the user, and the output is a specific learning category (e.g., calculation problems, Kanji, English, etc.).

[0376] Step 3:

[0377] The server generates appropriate game-style questions based on the classified learning categories. For this generation, the Django framework is used to create a quiz-style game containing multiple calculation questions. Specifically, the server determines the type of question and its difficulty level, and builds comprehensive quiz data. The input for this step is the learning categories, and the output is the generated game-style question data.

[0378] Step 4:

[0379] The server sends the generated game-style question data to the terminal. The terminal uses the received question data to display it on the screen. Specifically, the terminal displays quiz-style questions in an interactive interface, allowing the learner to enter answers. The input for this step is the game-style question data, and the output is the question interface displayed on the terminal.

[0380] Step 5:

[0381] The learner, who is the user, enters answers to the displayed quiz-style questions. The answers are sent from the terminal to the server. Specifically, the learner enters the answer to the calculation question and clicks the "Submit" button. The input in this step is the learner's answer data, and the output is the answer data sent from the terminal to the server.

[0382] Step 6:

[0383] The server compares the received answer data with pre-set correct answer data and evaluates whether the answer is correct or incorrect. This is done by using an SQL database to compare the correct answer data with the learner's answer. The evaluation results are generated as feedback data. The input to this step is the learner's answer data, and the output is the server's evaluation result (correct or incorrect).

[0384] Step 7:

[0385] The server returns the evaluation result data to the terminal, which then presents the result to the user. Specifically, the terminal displays a feedback message on the screen and provides the option to proceed to the next question or try again. The input of this step is the evaluation result data from the server, and the output is the feedback message presented to the user.

[0386] Step 8:

[0387] The device captures the learner's facial expressions and voice data using a camera and microphone, and sends that data to the server. The server then analyzes this data using an emotion engine to recognize the learner's emotions (e.g., joy, sadness, anger, excitement, etc.). Specifically, it uses Microsoft Azure's Emotion API. The input for this step is the learner's facial expressions and voice data, and the output is the emotion analysis results by the server.

[0388] Step 9:

[0389] The server readjusts the difficulty and content of the game-style questions based on the emotion analysis results. For example, if the learner is tired, it makes specific adjustments such as lowering the difficulty of the questions or providing encouraging messages. The input of this step is the emotion analysis results, and the output is the adjusted question data or encouraging messages.

[0390] Step 10:

[0391] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges for correctly answered questions and notifies the learner of this via their device. Specifically, it includes a system function that automatically manages appropriate rewards according to the learner's learning progress. The input to this step is the learner's progress data and answer data, and the output is an updated learning history and reward data.

[0392] (Application example 2)

[0393] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0394] Conventional learning systems lacked the means to make homework fun for learners, making it feel like a game, and limited the means to increase motivation. Furthermore, there were no systems that recognized learners' emotions and provided appropriate feedback. These shortcomings led to problems such as a lack of appropriate responses when learners lost interest in learning, or felt confused or fatigued, resulting in a decline in learning efficiency.

[0395] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a learner to input homework content; means for analyzing the input homework content and classifying it into specific learning categories; means for generating game-style questions according to the classified learning categories and transmitting them to the terminal; means for displaying the game-style questions received by the terminal so that the learner can answer them interactively; means for evaluating the learner's answers and presenting the results; means for recording the learner's progress and managing the learning history; means for recognizing and analyzing the learner's emotions; and means for adjusting the difficulty and content of the questions based on the recognized emotions. This enables the learner to study enjoyably and effectively, and by providing appropriate feedback according to emotions, learning efficiency is improved and motivation is maintained.

[0396] "Student" refers to a user who uses the system to do homework or study.

[0397] "Means for inputting homework content" refers to the interface that allows learners to input the specific content of their homework via a terminal.

[0398] "Means for analyzing the input homework content and classifying it into specific learning categories" refers to a program or device that uses natural language processing technology to analyze the homework content input by the learner and classify it into learning categories such as mathematics or English.

[0399] "Means for generating game-style questions and sending them to the terminal" refers to a system for creating game-style questions such as quizzes and puzzles based on the analyzed homework content and sending that data to the learner's terminal.

[0400] "A means for displaying questions on a terminal and allowing learners to answer them interactively" refers to an interface that displays the sent game-style questions on the screen of the learner's terminal and allows the learner to answer the questions.

[0401] "Means for evaluating answers and presenting results" refers to a system for determining whether the answers entered by the learner are correct or incorrect and providing feedback on the results to the learner.

[0402] "Means for recording learners' progress and managing their learning history" refers to a system that records the content and results of questions answered by learners, and stores and manages them in a database as learning history.

[0403] "Means for recognizing and analyzing learners' emotions" refers to technology that uses cameras and microphones to capture the learners' facial expressions and voices, and analyzes them to determine the learners' emotional state.

[0404] "Means for adjusting the difficulty and content of questions based on the learner's emotions" refers to an algorithm for dynamically changing the difficulty and content of questions based on the learner's emotional state.

[0405] The present invention is a learning system that allows learners to complete their homework more enjoyably and effectively. Specific embodiments of this system will be described below.

[0406] System configuration and program processing

[0407] Hardware and Software Configuration

[0408] This learning system includes the following hardware and software:

[0409] 1. Device: A device used by a user (learner), such as a smartphone, tablet, smart glasses, or head-mounted display.

[0410] 2. Server: A cloud-based server that receives data, analyzes it, generates questions, and performs emotion recognition.

[0411] 3. Software:

[0412] Natural language processing (NLP) technology: Uses Google Cloud Natural Language API.

[0413] Data management: Use Google Firebase to manage learning data.

[0414] Emotion Recognition Model: Emotion recognition is performed using TensorFlow.

[0415] System Operation

[0416] The system operates as follows.

[0417] 1. Learners input and submit their homework

[0418] The user (student) uses the device's application to input the content of their homework via text or voice. For example, using a smartphone, they can input "Today's math homework is 2 x 3 and 4 + 5" and tap the "Convert homework into a game" button. This data is then sent to the server via the Internet.

[0419] 2. Analysis and classification of homework content

[0420] The server uses natural language processing technology to analyze the received homework content, and based on the analysis results, classifies the homework into a specific learning category (e.g., math problems, Kanji, English, etc.).

[0421] 3. Creating and submitting game-style questions

[0422] The server generates game-style questions according to the appropriate learning category based on the analyzed homework content. For example, in the case of calculation questions, quiz-style questions including arithmetic operations are generated. The server then transmits the game-style question data to the device.

[0423] 4. Terminal display and interactive answers

[0424] The terminal displays the game-style questions received from the server to the learner, who then interactively inputs the answers, which are then sent back to the server via the terminal.

[0425] 5. Evaluation and feedback of answers

[0426] The server compares the received answers with pre-defined correct answer data, evaluates the learner's answers, and sends the results to the device to provide feedback to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[0427] 6. Learner emotion recognition and regulation

[0428] The device's camera and microphone are used to capture the learner's facial expressions and voice data, which are then sent to the server. The server's emotion engine (using TensorFlow) analyzes this data and recognizes the learner's emotional state. Based on the recognized emotion, the difficulty and content of the questions are adjusted. For example, if the learner is confused, the questions are made easier or a supportive message is displayed.

[0429] 7. Progress Recording and Rewarding

[0430] The server records the learner's answer data and progress data, and manages their learning history. This history data is used in the next learning session. In addition, learners are given rewards such as points and badges for correctly answering questions.

[0431] Examples of concrete examples and prompts

[0432] Specific examples

[0433] Situations where learners enter homework details on their smartphones:

[0434] The learner types, "Today's math homework is 2 x 3 and 4 + 5."

[0435] The server classifies the homework content into calculation problem categories and generates quiz-style questions.

[0436] The learner answers the quiz, the server evaluates it, and the learner moves on to the next question.

[0437] If the emotion engine detects that the learner is tired, it will easily adjust the questions or display encouraging messages such as "Keep it up! You're almost there!"

[0438] Prompt Sentence Examples

[0439] "If learners are stumped, lower the difficulty and provide a supportive message, such as, 'Don't worry, the next question is easier, so relax!'"

[0440] In this way, this system not only allows learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[0441] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0442] Step 1:

[0443] The user (learner) uses a device to input the homework content. Input is done by text or voice. For example, a learner inputs "Today's math homework is 2 x 3 and 4 + 5" into a smartphone and taps the "Convert homework into a game" button. This data is sent to a server via the Internet. The input data is in text format, and the input is sent to the server.

[0444] Step 2:

[0445] The homework content received by the server is analyzed using natural language processing technology (Google Cloud Natural Language API). Specifically, the input text is analyzed and the homework content is classified into specific learning categories (e.g., calculation problems, kanji, English, etc.). The analyzed data is output as text information classified into learning categories.

[0446] Step 3:

[0447] The server generates appropriate game-style questions based on the classified learning category. For example, if the question is classified as a calculation problem, it generates quiz-style questions that include arithmetic operations. During this process, it selects appropriate questions from a pre-prepared question database and converts them into a game format. The generated game-style question data is then sent to the device.

[0448] Step 4:

[0449] The device displays the received game-style questions to the learner. The questions are displayed using an interactive UI (user interface), and the learner inputs their answers. The answer data is sent to the server in real time. The input answer data becomes new input data for the server.

[0450] Step 5:

[0451] The server compares the received answer data with pre-defined correct answer data and evaluates the answer. The evaluation result is classified as "correct" or "incorrect." The evaluation result is sent to the device and feedback is provided to the learner. For example, if the answer is correct, a button to proceed to the next question is displayed, and if the answer is incorrect, an option to try again is presented.

[0452] Step 6:

[0453] The learner's facial expressions and voice data are captured using the device's camera and microphone. This emotional data is sent to the server in real time. The emotional data is captured as facial images and voice recordings and analyzed by the server.

[0454] Step 7:

[0455] The emotion engine (using TensorFlow) on the server analyzes the emotion data and recognizes the learner's emotional state. For example, it classifies the learner's emotional state as confused, tired, or enjoying. Based on the analysis results, it adjusts the difficulty and content of the questions. The recognized emotional state becomes the input data for adjusting the difficulty of the learning content.

[0456] Step 8:

[0457] The server records the learner's answer data and progress data, and manages the learning history. The progress data is stored in Firebase and used for the next learning session. The learning history database is updated.

[0458] Step 9:

[0459] The server provides rewards such as points and badges for questions that learners answer correctly, thereby increasing learner motivation. The reward data is sent to the learner's device and displayed. For example, a message such as "10 points earned!" or "You've earned a new badge!" is displayed.

[0460] Through these steps, this system not only enables learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[0461] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0463] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0464] [Second embodiment]

[0465] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0466] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0467] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0468] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0469] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0470] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0471] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0472] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0473] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0474] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0475] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0476] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0477] This invention is a system that allows learners to enjoyably complete their homework in a game-like manner. This system analyzes the learner's input, generates appropriate game-style questions based on that input, and supports the learner in the process of answering them. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[0478] Program processing

[0479] Students input and submit their homework

[0480] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0481] Analysis and classification of homework content by the server

[0482] The server then analyzes the received input, using natural language processing (NLP) techniques to analyze the text and classify the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.), and selects appropriate game-style problems based on the classification results.

[0483] Game-style question generation and submission

[0484] The server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[0485] Terminal display of questions and interactive solutions

[0486] The device displays the game-style questions received from the server on the screen. The learner looks at this screen and inputs the answer to each question. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[0487] Server-based answer evaluation and feedback

[0488] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0489] Track progress and provide rewards

[0490] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, rewards such as points and badges are provided via the device for correctly answered questions, which increases the learner's motivation.

[0491] Specific examples

[0492] 1. Math homework

[0493] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0494] 2. English homework

[0495] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[0496] In this way, the present invention is a system that converts homework into a game format, thereby increasing learners' concentration and motivation to study and promoting effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[0497] The processing flow will be explained below.

[0498] Step 1:

[0499] The user inputs the homework content.

[0500] The user uses the device to enter the homework content into the text box and clicks the "Convert homework to game" button, which then sends the input data to the server.

[0501] Step 2:

[0502] The server receives the homework content.

[0503] The server receives the homework content sent from the terminal and prepares it for analysis as text data.

[0504] Step 3:

[0505] The server analyzes the homework content and classifies it into categories.

[0506] The server uses natural language processing (NLP) technology to analyze the input text and, based on the analysis results, classifies the homework content into specific learning categories (e.g., math problems, geography problems, kanji problems, English problems, etc.).

[0507] Step 4:

[0508] The server will choose the appropriate game format.

[0509] The server selects an appropriate game format (e.g., quiz format, puzzle format, flash card format, etc.) from the database according to the classified learning category.

[0510] Step 5:

[0511] The server generates game-style questions.

[0512] The server generates a set of questions based on the selected game format. For example, if the game is a math calculation game, a quiz containing 10 arithmetic operations will be generated.

[0513] Step 6:

[0514] The server sends the generated game to the device.

[0515] The server sends the generated game data to the terminal, which receives this data.

[0516] Step 7:

[0517] The device displays a game-style question.

[0518] The device displays the received game data on the screen, presenting the user with questions and answer options.

[0519] Step 8:

[0520] The user answers the question.

[0521] The user inputs or selects an answer to the question displayed on the screen. For example, if it is a calculation question, the user inputs the answer in the input field.

[0522] Step 9:

[0523] The device sends the answer to the server.

[0524] The terminal transmits the user's answer to the server.

[0525] Step 10:

[0526] The server evaluates the answers.

[0527] The server compares the received answers with pre-set correct answer data, determines whether they are correct, and returns the evaluation results to the terminal.

[0528] Step 11:

[0529] The terminal displays the evaluation results.

[0530] The terminal displays the evaluation results received from the server to the user. If the answer is correct, the user has the option to proceed to the next question, and if the answer is incorrect, the user has the option to try again.

[0531] Step 12:

[0532] The server records the progress data.

[0533] The server records the user's answer data and progress data and updates the learning history.

[0534] Step 13:

[0535] The terminal provides the reward.

[0536] The device provides users with rewards such as points and badges for correctly answering questions.

[0537] Step 14:

[0538] The server suggests what to learn next.

[0539] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and sends them to the device, which then displays these suggestions to the user.

[0540] Through the above processing steps, the user can enjoyably complete their homework in a game-like format, and can effectively master the learning content.

[0541] Example 1

[0542] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0543] Traditional homework learning methods are monotonous and burdensome for learners. This can lead to problems such as reduced motivation to learn and difficulty in effective learning. Furthermore, there is a lack of progress management of learning content and feedback tailored to individual needs, which can lead to reduced learning efficiency. Furthermore, since no rewards are offered, maintaining motivation is also an issue.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0545] In this invention, the server includes: a means for a learner to input homework content using a terminal; a means for analyzing the input homework content using natural language processing technology and classifying it into specific learning categories; a means for generating game-style questions corresponding to the classified learning categories using a game engine and sending them to the terminal; a means for interactively displaying the game-style questions received by the terminal and allowing the learner to input answers; a means for comparing the learner's answers with a database to evaluate whether they are correct or incorrect and presenting the results; a means for recording the learner's answer data and progress data and managing the learning history; and a means for generating rewards for the learner and sending them to the terminal for display. This allows the learner to enjoyably complete their homework in a game-like format, improving their motivation to learn and promoting effective learning. Furthermore, managing the learning history and providing rewards can help maintain motivation to learn.

[0546] A "terminal" is an electronic device used by a learner to enter homework content, and examples include tablets and personal computers.

[0547] "Natural language processing technology" is a computer technology that analyzes input text data and understands specific meanings and contexts, and uses NLP libraries and algorithms.

[0548] A "learning category" is a category for classifying homework content into specific fields or types, and examples include calculation problems, Kanji, English, etc.

[0549] A "game engine" is a software platform for generating game-style problems, and examples include Unity and Unreal Engine.

[0550] "Interactive" refers to a function that allows learners to directly operate and input information through their terminal.

[0551] A "database" is a system that stores learners' answer data and learning history, and collates and manages the data as needed.

[0552] "Rewards" are incentives such as points or badges that are provided to motivate learners when they answer questions correctly.

[0553] "Progress data" is data used to record the results of questions answered by a learner and the answer status, and to track the progress of learning.

[0554] MODE FOR CARRYING OUT THE INVENTION

[0555] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner. The specific processing content of the program of this system and the hardware and software used will be described in detail below.

[0556] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. Specifically, the learner enters the homework content in the text box displayed on the device screen and clicks the "Convert homework into a game" button.

[0557] This input is sent from the device to a server, which then analyzes the received homework content using natural language processing (NLP) techniques. Suitable NLP libraries to use are "spaCy" and "NLTK." The text data is analyzed, and the homework content is classified into specific learning categories (e.g., math problems, Kanji, English, etc.).

[0558] The server then generates game-style questions based on the classified learning categories. The software used for this is a game engine such as Unity or Unreal Engine. For example, a quiz-style game data set containing 10 arithmetic calculation questions is generated.

[0559] The generated game data is sent from the server to the user's device. The device receives this game data and displays it on the screen in an interactive format. The learner can answer questions and enter their answers through the on-screen interface. The learner's answer data is then sent back to the server from the device.

[0560] The server checks the received answers against pre-set correct answer data using a database. For example, a database system such as MySQL is used. The evaluation results are sent back to the device and displayed to the learner. If the learner answers correctly, they can proceed to the next question, but if they answer incorrectly, they are given the option to try again.

[0561] Furthermore, the server records the learner's answer data and progress data, and uses BaaS (Backend as a Service) such as "Firebase" to update the learning history. This data is used when generating and suggesting questions for the next learning session. In addition, rewards such as points and badges are generated for correctly answered questions and sent to the device. The device displays reward information to the learner in real time, increasing their motivation to learn.

[0562] Specific examples

[0563] 1. Math homework

[0564] The learner types in "Today's math homework is a calculation problem." The server analyzes this and classifies it into the calculation problem category. The server generates a quiz-style game containing 10 arithmetic operation problems. The learner answers the questions displayed on the device one by one. The server evaluates the answers and displays the results on the device. When the learner answers all the questions, they are awarded points that are reflected in the next learning material.

[0565] Prompt Sentence Examples

[0566] Today's math homework is a calculation problem. Turn it into a game.

[0567] 2. English homework

[0568] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. The server then generates a puzzle-style game in which the learner answers the meaning of the word. The learner answers the puzzle displayed on the terminal by pairing words with their meanings. The server evaluates the answers, and if all pairs are correct, a reward is provided.

[0569] Prompt Sentence Examples

[0570] Today's English homework is to memorize the meanings of words. Turn this into a game.

[0571] In this way, the present invention automatically analyzes homework content and provides it in a game format, thereby increasing learners' motivation and supporting effective learning. By utilizing the entire system, the effectiveness of home study and online classes can be maximized.

[0572] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0573] Step 1:

[0574] The user uses the device to input the homework content. Specifically, the user inputs the homework content into the text box displayed on the device screen and clicks the "Convert homework to game" button. For example, the user might input "Today's math homework is a calculation problem." This input content is sent from the device to the server. The input data is the homework content in text format, and this is output to the server.

[0575] Step 2:

[0576] The server analyzes the received homework content. Specifically, it uses natural language processing (NLP) technology to analyze the input text. Suitable NLP libraries to use are "spaCy" and "NLTK." Through this analysis, the server classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). The input data is the transmitted text data, and the output data is the classification results.

[0577] Step 3:

[0578] The server generates game-style questions based on the classified learning categories. Specifically, it uses a game engine such as Unity or Unreal Engine to create an interactive game. For example, it generates quiz-style game data containing 10 calculation questions on arithmetic operations. The input data is the learning categories, and the output data is the generated game data.

[0579] Step 4:

[0580] The server sends the generated game data to the user's device. This causes the user's device to receive game-style questions. The input data is the generated game data, and the output data is a notification to the device that transmission has been completed. Specifically, the server divides the game data into packets and sends them over the network.

[0581] Step 5:

[0582] The terminal displays the received game-style questions on the screen. The user can answer the questions through the on-screen interface. For example, in the case of a calculation question, the user looks at the calculation formula displayed on the screen and enters the answer in the answer input field. The input data is the received game data, and the output data is the user's answer input.

[0583] Step 6:

[0584] The terminal sends the user's answer data to the server. Specifically, the answer entered by the user is divided into packets and sent to the server. The input data is the user's answer data, and the output data is a transmission completion notification to the server.

[0585] Step 7:

[0586] The server evaluates the received answer data. Specifically, it compares it with pre-set correct answer data using a database. A suitable database to use is MySQL. The input data is the user's answer data, and the output data is the evaluation result.

[0587] Step 8:

[0588] The server returns the evaluation results to the user's terminal, which then displays the results to the user. The input data is the evaluation results, and the output data is a transmission completion notification to the terminal. Specifically, the success / failure information is divided into packets and sent to the terminal.

[0589] Step 9:

[0590] The server records the user's progress data and learning history. Specifically, it uses BaaS such as "Firebase" to update the learning history database. This data is used to generate and suggest questions for the next learning session. The input data is the learner's answer data and progress data, and the output data is the updated learning history.

[0591] Step 10:

[0592] The server generates rewards for learners and sends them to the device. Specifically, rewards such as points and badges are generated for questions that the learner answers correctly. The input data is the evaluation results, and the output data is the generated reward data. The device displays the reward information in real time and notifies the learner.

[0593] (Application example 1)

[0594] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0595] In conventional educational systems, there are problems such as learners finding it difficult to maintain their motivation while completing their homework. Also, there are issues with the lack of ways to attract customers' attention when obtaining product information in physical stores. This can lead to a decrease in learning effectiveness and a lack of understanding of products.

[0596] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0597] In this invention, the server includes means for a learner to input homework content, means for analyzing the input homework content and classifying it into a specific learning category, means for generating game-style questions according to the classified learning category and transmitting them to the terminal, means for a customer to input product information, means for analyzing the input product information and classifying it into a specific product category, and means for generating game-style information presentations according to the classified product category and transmitting them to the terminal. This allows learners to enjoy doing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[0598] "Learner" means a person who uses an educational system to carry out learning activities.

[0599] "Homework content" refers to the learning tasks assigned to students by their school or teaching materials.

[0600] "Input means" refers to the interface through which a learner or customer provides information to the system.

[0601] "Analysis means" refers to technology for processing input information and understanding its content.

[0602] "Learning categories" are criteria for classifying educational content into specific themes or subjects.

[0603] "Game-style questions" are questions that include game elements in order to enhance learning effectiveness.

[0604] "Device" means an electronic device (e.g., tablet, smartphone, smart glasses) used by a learner or customer.

[0605] A "display means" is a method for visually presenting learning or information content on a terminal.

[0606] "Means for interactive answering" means a method by which a learner or customer can input answers through a terminal and interact with the system in a two-way manner.

[0607] "Means for evaluating answers" refers to a system for evaluating the answers of learners or customers and determining whether they are correct or incorrect.

[0608] The "means of presenting results" is the method by which the results of the assessment are communicated to the learner or client.

[0609] A "means for recording progress" is a system that stores learner or customer activity data.

[0610] "Means for managing learning history" refers to a system for effectively organizing recorded progress data and utilizing it for future learning or information provision.

[0611] "Customer" means a person who uses the system to obtain product information in a physical store.

[0612] "Product information" refers to information about the features and usage of products sold in physical stores.

[0613] A "product category" is a category in which products are classified according to specific criteria.

[0614] "Information presentation" refers to a method for displaying information about a product to a customer.

[0615] System Overview

[0616] This invention provides a system that allows learners and customers in brick-and-mortar stores to learn and acquire product information in an interactive game format. This system uses devices such as tablets, smartphones, and smart glasses, and processes and provides information in cooperation with a server.

[0617] Hardware and software used

[0618] Hardware:

[0619] Tablet (e.g. iPad)

[0620] Smartphone (e.g. iPhone or Android device)

[0621] Smart glasses (e.g. Google Glass)

[0622] software:

[0623] Natural Language Processing (NLP) libraries (e.g., SpaCy, NLTK)

[0624] Game engines (e.g. Unity, Cocos2d)

[0625] Backend server (e.g. AWS Lambda, Django Rest Framework)

[0626] Explanation of program processing

[0627] Step 1: Learner or customer enters information

[0628] Users input homework details and product information using devices (tablets, smartphones, etc.). Specifically, for example, a student might input "Today's math homework is a calculation problem," while a customer at a physical store might input "I want to know about cosmetics."

[0629] Step 2: Server analysis and classification

[0630] The server uses natural language processing (NLP) technology to analyze the input text and classify it into appropriate learning or product categories, such as "math problems" or "cosmetics."

[0631] Step 3: Generate and send game-style questions and information prompts

[0632] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and the generated content is sent to the terminal.

[0633] Step 4: Terminal display and interaction

[0634] The terminal displays game-style questions and information received from the server on its screen. The learner or customer looks at this screen and inputs answers to each question. For example, if it is a calculation question, the learner answers the formula displayed on the screen, and if it is cosmetic information, the learner answers the related quiz.

[0635] Step 5: Server evaluation and feedback of answers

[0636] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0637] Step 6: Track progress and provide rewards

[0638] The server records the learner's or customer's answer data and progress data, and updates the learning history and customer history. This data is used to generate and suggest questions for the next learning session, as well as to provide product information. Correct answers to questions can provide rewards such as points or coupons.

[0639] Specific examples

[0640] 1. Math homework

[0641] A learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time a learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0642] 2. Cosmetics information

[0643] A customer types in "I want to know more about cosmetics." The server analyzes this and classifies it into a cosmetics category. A quiz-style information presentation about the product's ingredients and usage is generated and displayed on the terminal. When the customer answers the quiz, their answers are evaluated and if they are correct, they are given a coupon as a reward.

[0644] Prompt Sentence Examples

[0645] I entered "I want to know more about cosmetics." Based on this information, please generate the following trivia and quiz questions:

[0646] 1. Cosmetic Ingredients Quiz

[0647] 2. Best Practices for Usage

[0648] 3. Product reviews and recommended usage

[0649] This system allows learners to enjoy completing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[0650] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0651] Step 1:

[0652] Learner or customer input of information

[0653] The user inputs homework details and product information using a device (tablet, smartphone, etc.).

[0654] Input: A learner types, "Today's math homework is a calculation problem." Or a customer types, "I want to know about cosmetics."

[0655] Data processing: Obtain the input text data and send it to the server.

[0656] Output: The text data sent to the server.

[0657] Step 2:

[0658] Server-based information analysis and classification

[0659] The server uses natural language processing (NLP) techniques to analyze the input text and classify it into appropriate learning and product categories.

[0660] Input: Text data (homework assignments or product information).

[0661] Data processing: Analyze the text using an NLP library (e.g., SpaCy, NLTK) to extract specific keywords and phrases. Then, classify the text into learning or product categories based on these keywords.

[0662] Output: Category information (e.g., "Math Problem", "Cosmetics").

[0663] Step 3:

[0664] Generate and send game-style questions and information presentations

[0665] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and transmits them to the terminal.

[0666] Input: Category information (e.g., "Math Problems," "Cosmetics").

[0667] Data processing: Using a game engine (e.g. Unity, Cocos2d), create game-style questions and information presentations according to the category.

[0668] Output: Generated game-style questions and information presentation data. Send to device.

[0669] Step 4:

[0670] Terminal display and interaction

[0671] The terminal displays game-style questions and information received from the server on its screen, and the learner or customer inputs answers to each question.

[0672] Input: Game-style questions and information presentation data sent from the server.

[0673] Data processing: The device analyzes the data received and displays it on the user interface.

[0674] How it works: The learner or customer enters an answer to a question, and the device sends the answer to the server.

[0675] Output: User's answer data.

[0676] Step 5:

[0677] Server-based answer evaluation and feedback

[0678] The server receives the user's answer, compares it with pre-set correct answer data, evaluates its accuracy, and returns the evaluation result to the terminal.

[0679] Input: User's answer data.

[0680] Data processing: The server checks the answer data against the correct answer database to determine whether it is correct or incorrect, and generates a feedback message indicating whether the answer is correct or incorrect.

[0681] Output: A feedback message (e.g., "Correct" if correct, "Incorrect" if incorrect) sent back to the device.

[0682] Step 6:

[0683] Track progress and provide rewards

[0684] The server records the answer data and progress data of the learner or customer, updates the learning history and customer history, and provides rewards (points or coupons) for correct answers.

[0685] Input: User answer data, feedback message.

[0686] Data processing: The server records the answer results in the progress management system and updates the user's history. If the answer is correct, points or coupons are generated using the reward system.

[0687] Output: Updated learning history, customer history data, reward data (points and coupons). Send to the terminal.

[0688] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0689] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and also has the function of recognizing the learner's emotions and optimizing the learning process. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[0690] Program processing

[0691] Students input and submit their homework

[0692] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0693] Analysis and classification of homework content by the server

[0694] The server analyzes the received input, uses natural language processing (NLP) technology to analyze the text, and classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[0695] Game-style question generation and submission

[0696] Next, the server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[0697] Terminal display of questions and interactive solutions

[0698] The device displays the game-style questions received from the server on the screen. The learner inputs the answers to each question while looking at this screen. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[0699] Server-based answer evaluation and feedback

[0700] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0701] Emotion recognition and regulation with emotion engine

[0702] To recognize learners' emotions, the system uses a camera and microphone on the device to capture their facial expressions and voice data. An emotion engine on the server analyzes this data and recognizes the learner's emotions (e.g., joy, anger, sadness, excitement, etc.).

[0703] Based on the recognized emotions, the server can adjust the difficulty and content of the gamified questions appropriately. For example, if the learner is tired or confused, it can lower the difficulty of the questions or provide encouraging messages.

[0704] Track progress and provide rewards

[0705] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges via the device for correctly answered questions, thereby increasing learner motivation.

[0706] Specific examples

[0707] 1. Math homework

[0708] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0709] Additionally, if the system recognizes that a learner is confused based on their facial expression, it will support them by lowering the difficulty of the questions or displaying encouraging messages.

[0710] 2. English homework

[0711] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[0712] Additionally, if the system recognizes from the learner's voice data that the learner is enjoying the lesson, it can add more challenging questions while maintaining the current level of difficulty.

[0713] This system converts homework into a game format, and by appropriately recognizing and adjusting learners' emotions, it increases their motivation to learn and promotes effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[0714] The processing flow will be explained below.

[0715] Step 1:

[0716] The user inputs the homework content.

[0717] The user uses the device to enter specific homework content into the text box and clicks the "Convert homework to game" button, which is then sent from the device to the server.

[0718] Step 2:

[0719] The server receives the homework content.

[0720] The server receives the homework content sent from the terminal and prepares for analysis.

[0721] Step 3:

[0722] The server analyzes the homework content and classifies it into learning categories.

[0723] The server uses natural language processing (NLP) technology to analyze the homework content and classify it into specific learning categories (e.g., math problems, Kanji, English, etc.).

[0724] Step 4:

[0725] The server will choose the appropriate game format.

[0726] The server selects an appropriate game format (e.g., quiz, puzzle, flashcard, etc.) from the database according to the classified category.

[0727] Step 5:

[0728] The server generates game-style questions.

[0729] The server generates a set of questions in the selected game format. For example, if the questions are math calculation questions, it generates a quiz that includes arithmetic operations.

[0730] Step 6:

[0731] The server sends the generated game to the device.

[0732] The server sends the generated game data to the terminal, which receives this data.

[0733] Step 7:

[0734] The device displays a game-style question.

[0735] The device displays the received game-style questions on the screen, allowing the user to interactively answer them.

[0736] Step 8:

[0737] The device collects the user's emotional data.

[0738] As the user works on the problem, the device uses a camera and microphone to collect the user's facial expressions and voice, and this data is sent to a server for analysis by the emotion engine.

[0739] Step 9:

[0740] The user answers the question.

[0741] The user inputs the answer to the question displayed on the screen. For example, if it is a calculation question, the answer is entered in the answer field. The input answer is sent from the terminal to the server.

[0742] Step 10:

[0743] The server evaluates the answers.

[0744] The server receives the user's answer, judges whether it is correct, compares it with the correct answer data set in advance, and returns the evaluation result to the terminal.

[0745] Step 11:

[0746] The terminal displays the evaluation results.

[0747] The device receives the evaluation results from the server and displays them to the user. If the answer is correct, the user is given the option to proceed to the next question, and if the answer is incorrect, the user is given the option to try again.

[0748] Step 12:

[0749] The server analyzes the user's emotional data.

[0750] The server's emotion engine analyzes the received facial and voice data and recognizes the user's emotions (e.g., joy, anger, sadness, confusion, etc.).

[0751] Step 13:

[0752] The server adjusts the game content.

[0753] Based on the recognized emotions, the server adjusts the difficulty and content of the game. For example, if the user is having trouble, it may lower the difficulty of the problems or send a message of encouragement.

[0754] Step 14:

[0755] The server records the progress data.

[0756] The server records the user's answers and progress data, and updates the learning history, which is used to generate questions and suggestions for the next learning session.

[0757] Step 15:

[0758] The terminal provides the reward.

[0759] The device provides users with rewards such as points and badges for correctly answering questions.

[0760] Step 16:

[0761] The server suggests what to learn next.

[0762] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and emotional data, and sends them to the device, which then displays them to the user.

[0763] The above processing steps allow users to enjoy completing their homework in a game-like manner, effectively mastering the learning content. Furthermore, the introduction of an emotion engine optimizes the learning process for each individual user, which is expected to increase their motivation to learn.

[0764] Example 2

[0765] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0766] Conventional learning support systems often progress homework in a monotonous manner, which can easily lower learners' motivation. Furthermore, they provide problems with a uniform level of difficulty without taking learners' emotions into consideration, which results in the problem of not maximizing learning effectiveness. Furthermore, they do not provide appropriate feedback or adjustments, which results in lower learning efficiency.

[0767] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a learner to input homework content, a means for analyzing the input homework content and classifying it into a specific learning category, a means for generating game-style questions according to the classified learning category and transmitting them to the terminal, a means for displaying the game-style questions received by the terminal so that the learner can answer them interactively, a means for evaluating the learner's answers and presenting the results, a means for recording the learner's progress and managing the learning history, and a means for recognizing the learner's emotions and adjusting the difficulty and content of the questions. This enables the learner to actively engage in learning while having fun, thereby realizing effective learning support.

[0768] "Student" refers to an entity that uses the system to carry out learning activities.

[0769] "Means for inputting homework content" refers to the input devices and software functions that allow learners to input homework content into the system.

[0770] "Means for analyzing homework content and classifying it into specific learning categories" refers to a software function that analyzes the input homework content using natural language processing technology, etc., and assigns the content to specific learning categories.

[0771] "Means for generating game-style questions and sending them to the device" refers to a software function for automatically creating appropriate game-style questions based on the analyzed homework content and sending the data to the learner's device.

[0772] "Means for interactive answering" refers to software or hardware that allows learners to input answers to questions posed on their devices in real time.

[0773] "Means for evaluating answers and presenting results" refers to software functionality for automatically evaluating a learner's answers and notifying the learner of the evaluation results.

[0774] "Means for recording progress and managing learning history" refers to software functions that record learners' answers and progress in a database and manage their learning history based on that.

[0775] "Means for recognizing emotions and adjusting the difficulty and content of questions" refers to software functions that analyze data such as the learner's facial expressions and voice to recognize their emotions, and dynamically change the difficulty and content of questions based on the results.

[0776] "Means for providing rewards such as points or badges" refers to software functionality that automatically awards rewards such as points or badges to learners for correctly answering questions.

[0777] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and further has the function of recognizing the learner's emotions and optimizing the learning process. Specific embodiments for carrying out the present invention will now be described.

[0778] System configuration

[0779] Students input and submit their homework

[0780] The learner, who is the user, inputs the homework content using a device such as a tablet or PC. The learner enters "Today's math homework is a calculation problem" in the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0781] Analysis and classification of homework content by the server

[0782] The server uses natural language processing (NLP) technology to analyze the received homework content. Specifically, the NLTK library in Python can be used. The server analyzes the input text and classifies the content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[0783] Game-style question generation and submission

[0784] The server generates game-style questions according to the selected learning category. The Django framework can be used for this generation. For example, in the case of math calculation questions, a quiz-style game containing multiple calculation questions is generated. The generated game data is sent from the server to the device.

[0785] Terminal display of questions and interactive solutions

[0786] The device displays game-style questions received from the server on the screen. The learner looks at the displayed questions and enters their answers. For calculation questions, the learner looks at the calculation formula displayed on the screen and enters the answer in the answer input field. Once the learner has finished entering their answer, the data is sent from the device to the server.

[0787] Server-based answer evaluation and feedback

[0788] The server receives the learner's answer data and compares it with pre-set correct answer data to evaluate whether it is correct or incorrect. An SQL database can be used for this evaluation. The server generates an evaluation result and sends it back to the device for presentation to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[0789] Emotion recognition and regulation with emotion engine

[0790] To recognize learners' emotions, the system uses the device's camera and microphone to capture their facial expressions and voice data. This data is sent to a server where it is analyzed by an emotion engine within the server. Specific technology used is the Microsoft Azure Emotion API. The emotion engine analyzes the learner's emotions (e.g., joy, sadness, anger, excitement, etc.) and adjusts the difficulty and content of the questions based on the results.

[0791] Track progress and provide rewards

[0792] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, the server can provide rewards such as points and badges for questions that the learner answers correctly, which helps to improve the learner's motivation.

[0793] Specific examples

[0794] 1. Math homework

[0795] When a learner types "Today's math homework is a calculation problem," the device sends this information to the server. The server uses NLP technology to recognize the category "calculation problem." It then generates a quiz-style game and sends it to the device. The device displays it, and when the learner enters an answer, the answer data is sent to the server. The server evaluates whether the answer is correct or incorrect and sends the evaluation result back to the device to show it to the learner. If the learner looks confused, the difficulty of the problem is automatically adjusted and a supportive message is displayed.

[0796] Example inputs to a generative AI model:

[0797] User: "Today's math homework includes calculation problems."

[0798] System: "Recognizes calculation problem categories and generates 10 arithmetic operation problems."

[0799] Server: "Evaluate the issue and provide feedback"

[0800] 2. English homework

[0801] When a learner types "Today's English homework is to memorize the meanings of words," the device sends this information to the server. The server uses NLP technology to recognize the "word learning" category. It then generates a puzzle-style game in which the learner must guess the meaning of the word and sends it to the device. When the learner types their answer, the data is sent to the server, which evaluates it and provides feedback. If the server recognizes that the learner is enjoying the game, it adds more challenging questions at the same difficulty level.

[0802] Example inputs to a generative AI model:

[0803] User: "Today's English homework is to memorize the meanings of words."

[0804] System: "Recognizes word learning categories and generates puzzle-style games"

[0805] Server: "Evaluate the issue and provide feedback"

[0806] Thus, the present invention is an optimal learning support system that enables learners to study in an enjoyable and effective manner.

[0807] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0808] Step 1:

[0809] The user inputs the homework details using a device such as a tablet or PC. The specific action is to enter "Today's math homework is a calculation problem" into the text box and click the "Convert homework into a game" button. The entered text data is sent from the device to the server. The input of this step is the text entered by the user, and the output is the text data sent to the server.

[0810] Step 2:

[0811] The server analyzes the received text data of the homework content using natural language processing technology. Specifically, it uses Python's NLTK library to perform morphological analysis and classify the homework content into specific learning categories. The input for this step is the text data submitted by the user, and the output is a specific learning category (e.g., calculation problems, Kanji, English, etc.).

[0812] Step 3:

[0813] The server generates appropriate game-style questions based on the classified learning categories. For this generation, the Django framework is used to create a quiz-style game containing multiple calculation questions. Specifically, the server determines the type of question and its difficulty level, and builds comprehensive quiz data. The input for this step is the learning categories, and the output is the generated game-style question data.

[0814] Step 4:

[0815] The server sends the generated game-style question data to the terminal. The terminal uses the received question data to display it on the screen. Specifically, the terminal displays quiz-style questions in an interactive interface, allowing the learner to enter answers. The input for this step is the game-style question data, and the output is the question interface displayed on the terminal.

[0816] Step 5:

[0817] The learner, who is the user, enters answers to the displayed quiz-style questions. The answers are sent from the terminal to the server. Specifically, the learner enters the answer to the calculation question and clicks the "Submit" button. The input in this step is the learner's answer data, and the output is the answer data sent from the terminal to the server.

[0818] Step 6:

[0819] The server compares the received answer data with pre-set correct answer data and evaluates whether the answer is correct or incorrect. This is done by using an SQL database to compare the correct answer data with the learner's answer. The evaluation results are generated as feedback data. The input to this step is the learner's answer data, and the output is the server's evaluation result (correct or incorrect).

[0820] Step 7:

[0821] The server returns the evaluation result data to the terminal, which then presents the result to the user. Specifically, the terminal displays a feedback message on the screen and provides the option to proceed to the next question or try again. The input of this step is the evaluation result data from the server, and the output is the feedback message presented to the user.

[0822] Step 8:

[0823] The device captures the learner's facial expressions and voice data using a camera and microphone, and sends that data to the server. The server then analyzes this data using an emotion engine to recognize the learner's emotions (e.g., joy, sadness, anger, excitement, etc.). Specifically, it uses Microsoft Azure's Emotion API. The input for this step is the learner's facial expressions and voice data, and the output is the emotion analysis results by the server.

[0824] Step 9:

[0825] The server readjusts the difficulty and content of the game-style questions based on the emotion analysis results. For example, if the learner is tired, it makes specific adjustments such as lowering the difficulty of the questions or providing encouraging messages. The input of this step is the emotion analysis results, and the output is the adjusted question data or encouraging messages.

[0826] Step 10:

[0827] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges for correctly answered questions and notifies the learner of this via their device. Specifically, it includes a system function that automatically manages appropriate rewards according to the learner's learning progress. The input to this step is the learner's progress data and answer data, and the output is an updated learning history and reward data.

[0828] (Application example 2)

[0829] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0830] Conventional learning systems lacked the means to make homework fun for learners, making it feel like a game, and limited the means to increase motivation. Furthermore, there were no systems that recognized learners' emotions and provided appropriate feedback. These shortcomings led to problems such as a lack of appropriate responses when learners lost interest in learning, or felt confused or fatigued, resulting in a decline in learning efficiency.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a learner to input homework content; means for analyzing the input homework content and classifying it into specific learning categories; means for generating game-style questions according to the classified learning categories and transmitting them to the terminal; means for displaying the game-style questions received by the terminal so that the learner can answer them interactively; means for evaluating the learner's answers and presenting the results; means for recording the learner's progress and managing the learning history; means for recognizing and analyzing the learner's emotions; and means for adjusting the difficulty and content of the questions based on the recognized emotions. This enables the learner to study enjoyably and effectively, and by providing appropriate feedback according to emotions, learning efficiency is improved and motivation is maintained.

[0832] "Student" refers to a user who uses the system to do homework or study.

[0833] "Means for inputting homework content" refers to the interface that allows learners to input the specific content of their homework via a terminal.

[0834] "Means for analyzing the input homework content and classifying it into specific learning categories" refers to a program or device that uses natural language processing technology to analyze the homework content input by the learner and classify it into learning categories such as mathematics or English.

[0835] "Means for generating game-style questions and sending them to the terminal" refers to a system for creating game-style questions such as quizzes and puzzles based on the analyzed homework content and sending that data to the learner's terminal.

[0836] "A means for displaying questions on a terminal and allowing learners to answer them interactively" refers to an interface that displays the sent game-style questions on the screen of the learner's terminal and allows the learner to answer the questions.

[0837] "Means for evaluating answers and presenting results" refers to a system for determining whether the answers entered by the learner are correct or incorrect and providing feedback on the results to the learner.

[0838] "Means for recording learners' progress and managing their learning history" refers to a system that records the content and results of questions answered by learners, and stores and manages them in a database as learning history.

[0839] "Means for recognizing and analyzing learners' emotions" refers to technology that uses cameras and microphones to capture the learners' facial expressions and voices, and analyzes them to determine the learners' emotional state.

[0840] "Means for adjusting the difficulty and content of questions based on the learner's emotions" refers to an algorithm for dynamically changing the difficulty and content of questions based on the learner's emotional state.

[0841] The present invention is a learning system that allows learners to complete their homework more enjoyably and effectively. Specific embodiments of this system will be described below.

[0842] System configuration and program processing

[0843] Hardware and Software Configuration

[0844] This learning system includes the following hardware and software:

[0845] 1. Device: A device used by a user (learner), such as a smartphone, tablet, smart glasses, or head-mounted display.

[0846] 2. Server: A cloud-based server that receives data, analyzes it, generates questions, and performs emotion recognition.

[0847] 3. Software:

[0848] Natural language processing (NLP) technology: Uses Google Cloud Natural Language API.

[0849] Data management: Use Google Firebase to manage learning data.

[0850] Emotion Recognition Model: Emotion recognition is performed using TensorFlow.

[0851] System Operation

[0852] The system operates as follows.

[0853] 1. Learners input and submit their homework

[0854] The user (student) uses the device's application to input the content of their homework via text or voice. For example, using a smartphone, they can input "Today's math homework is 2 x 3 and 4 + 5" and tap the "Convert homework into a game" button. This data is then sent to the server via the Internet.

[0855] 2. Analysis and classification of homework content

[0856] The server uses natural language processing technology to analyze the received homework content, and based on the analysis results, classifies the homework into a specific learning category (e.g., math problems, Kanji, English, etc.).

[0857] 3. Creating and submitting game-style questions

[0858] The server generates game-style questions according to the appropriate learning category based on the analyzed homework content. For example, in the case of calculation questions, quiz-style questions including arithmetic operations are generated. The server then transmits the game-style question data to the device.

[0859] 4. Terminal display and interactive answers

[0860] The terminal displays the game-style questions received from the server to the learner, who then interactively inputs the answers, which are then sent back to the server via the terminal.

[0861] 5. Evaluation and feedback of answers

[0862] The server compares the received answers with pre-defined correct answer data, evaluates the learner's answers, and sends the results to the device to provide feedback to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[0863] 6. Learner emotion recognition and regulation

[0864] The device's camera and microphone are used to capture the learner's facial expressions and voice data, which are then sent to the server. The server's emotion engine (using TensorFlow) analyzes this data and recognizes the learner's emotional state. Based on the recognized emotion, the difficulty and content of the questions are adjusted. For example, if the learner is confused, the questions are made easier or a supportive message is displayed.

[0865] 7. Progress Recording and Rewarding

[0866] The server records the learner's answer data and progress data, and manages their learning history. This history data is used in the next learning session. In addition, learners are given rewards such as points and badges for correctly answering questions.

[0867] Examples of concrete examples and prompts

[0868] Specific examples

[0869] Situations where learners enter homework details on their smartphones:

[0870] The learner types, "Today's math homework is 2 x 3 and 4 + 5."

[0871] The server classifies the homework content into calculation problem categories and generates quiz-style questions.

[0872] The learner answers the quiz, the server evaluates it, and the learner moves on to the next question.

[0873] If the emotion engine detects that the learner is tired, it will easily adjust the questions or display encouraging messages such as "Keep it up! You're almost there!"

[0874] Prompt Sentence Examples

[0875] "If learners are stumped, lower the difficulty and provide a supportive message, such as, 'Don't worry, the next question is easier, so relax!'"

[0876] In this way, this system not only allows learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[0877] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0878] Step 1:

[0879] The user (learner) uses a device to input the homework content. Input is done by text or voice. For example, a learner inputs "Today's math homework is 2 x 3 and 4 + 5" into a smartphone and taps the "Convert homework into a game" button. This data is sent to a server via the Internet. The input data is in text format, and the input is sent to the server.

[0880] Step 2:

[0881] The homework content received by the server is analyzed using natural language processing technology (Google Cloud Natural Language API). Specifically, the input text is analyzed and the homework content is classified into specific learning categories (e.g., calculation problems, kanji, English, etc.). The analyzed data is output as text information classified into learning categories.

[0882] Step 3:

[0883] The server generates appropriate game-style questions based on the classified learning category. For example, if the question is classified as a calculation problem, it generates quiz-style questions that include arithmetic operations. During this process, it selects appropriate questions from a pre-prepared question database and converts them into a game format. The generated game-style question data is then sent to the device.

[0884] Step 4:

[0885] The device displays the received game-style questions to the learner. The questions are displayed using an interactive UI (user interface), and the learner inputs their answers. The answer data is sent to the server in real time. The input answer data becomes new input data for the server.

[0886] Step 5:

[0887] The server compares the received answer data with pre-defined correct answer data and evaluates the answer. The evaluation result is classified as "correct" or "incorrect." The evaluation result is sent to the device and feedback is provided to the learner. For example, if the answer is correct, a button to proceed to the next question is displayed, and if the answer is incorrect, an option to try again is presented.

[0888] Step 6:

[0889] The learner's facial expressions and voice data are captured using the device's camera and microphone. This emotional data is sent to the server in real time. The emotional data is captured as facial images and voice recordings and analyzed by the server.

[0890] Step 7:

[0891] The emotion engine (using TensorFlow) on the server analyzes the emotion data and recognizes the learner's emotional state. For example, it classifies the learner's emotional state as confused, tired, or enjoying. Based on the analysis results, it adjusts the difficulty and content of the questions. The recognized emotional state becomes the input data for adjusting the difficulty of the learning content.

[0892] Step 8:

[0893] The server records the learner's answer data and progress data, and manages the learning history. The progress data is stored in Firebase and used for the next learning session. The learning history database is updated.

[0894] Step 9:

[0895] The server provides rewards such as points and badges for questions that learners answer correctly, thereby increasing learner motivation. The reward data is sent to the learner's device and displayed. For example, a message such as "10 points earned!" or "You've earned a new badge!" is displayed.

[0896] Through these steps, this system not only enables learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[0897] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0898] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0899] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0900] [Third embodiment]

[0901] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0902] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0903] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0904] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0905] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0906] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0907] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0908] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0909] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0910] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0911] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0912] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0913] This invention is a system that allows learners to enjoyably complete their homework in a game-like manner. This system analyzes the learner's input, generates appropriate game-style questions based on that input, and supports the learner in the process of answering them. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[0914] Program processing

[0915] Students input and submit their homework

[0916] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[0917] Analysis and classification of homework content by the server

[0918] The server then analyzes the received input, using natural language processing (NLP) techniques to analyze the text and classify the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.), and selects appropriate game-style problems based on the classification results.

[0919] Game-style question generation and submission

[0920] The server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[0921] Terminal display of questions and interactive solutions

[0922] The device displays the game-style questions received from the server on the screen. The learner looks at this screen and inputs the answer to each question. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[0923] Server-based answer evaluation and feedback

[0924] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[0925] Track progress and provide rewards

[0926] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, rewards such as points and badges are provided via the device for correctly answered questions, which increases the learner's motivation.

[0927] Specific examples

[0928] 1. Math homework

[0929] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[0930] 2. English homework

[0931] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[0932] In this way, the present invention is a system that converts homework into a game format, thereby increasing learners' concentration and motivation to study and promoting effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[0933] The processing flow will be explained below.

[0934] Step 1:

[0935] The user inputs the homework content.

[0936] The user uses the device to enter the homework content into the text box and clicks the "Convert homework to game" button, which then sends the input data to the server.

[0937] Step 2:

[0938] The server receives the homework content.

[0939] The server receives the homework content sent from the terminal and prepares it for analysis as text data.

[0940] Step 3:

[0941] The server analyzes the homework content and classifies it into categories.

[0942] The server uses natural language processing (NLP) technology to analyze the input text and, based on the analysis results, classifies the homework content into specific learning categories (e.g., math problems, geography problems, kanji problems, English problems, etc.).

[0943] Step 4:

[0944] The server will choose the appropriate game format.

[0945] The server selects an appropriate game format (e.g., quiz format, puzzle format, flash card format, etc.) from the database according to the classified learning category.

[0946] Step 5:

[0947] The server generates game-style questions.

[0948] The server generates a set of questions based on the selected game format. For example, if the game is a math calculation game, a quiz containing 10 arithmetic operations will be generated.

[0949] Step 6:

[0950] The server sends the generated game to the device.

[0951] The server sends the generated game data to the terminal, which receives this data.

[0952] Step 7:

[0953] The device displays a game-style question.

[0954] The device displays the received game data on the screen, presenting the user with questions and answer options.

[0955] Step 8:

[0956] The user answers the question.

[0957] The user inputs or selects an answer to the question displayed on the screen. For example, if it is a calculation question, the user inputs the answer in the input field.

[0958] Step 9:

[0959] The device sends the answer to the server.

[0960] The terminal transmits the user's answer to the server.

[0961] Step 10:

[0962] The server evaluates the answers.

[0963] The server compares the received answers with pre-set correct answer data, determines whether they are correct, and returns the evaluation results to the terminal.

[0964] Step 11:

[0965] The terminal displays the evaluation results.

[0966] The terminal displays the evaluation results received from the server to the user. If the answer is correct, the user has the option to proceed to the next question, and if the answer is incorrect, the user has the option to try again.

[0967] Step 12:

[0968] The server records the progress data.

[0969] The server records the user's answer data and progress data and updates the learning history.

[0970] Step 13:

[0971] The terminal provides the reward.

[0972] The device provides users with rewards such as points and badges for correctly answering questions.

[0973] Step 14:

[0974] The server suggests what to learn next.

[0975] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and sends them to the device, which then displays these suggestions to the user.

[0976] Through the above processing steps, the user can enjoyably complete their homework in a game-like format, and can effectively master the learning content.

[0977] Example 1

[0978] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0979] Traditional homework learning methods are monotonous and burdensome for learners. This can lead to problems such as reduced motivation to learn and difficulty in effective learning. Furthermore, there is a lack of progress management of learning content and feedback tailored to individual needs, which can lead to reduced learning efficiency. Furthermore, since no rewards are offered, maintaining motivation is also an issue.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0981] In this invention, the server includes: a means for a learner to input homework content using a terminal; a means for analyzing the input homework content using natural language processing technology and classifying it into specific learning categories; a means for generating game-style questions corresponding to the classified learning categories using a game engine and sending them to the terminal; a means for interactively displaying the game-style questions received by the terminal and allowing the learner to input answers; a means for comparing the learner's answers with a database to evaluate whether they are correct or incorrect and presenting the results; a means for recording the learner's answer data and progress data and managing the learning history; and a means for generating rewards for the learner and sending them to the terminal for display. This allows the learner to enjoyably complete their homework in a game-like format, improving their motivation to learn and promoting effective learning. Furthermore, managing the learning history and providing rewards can help maintain motivation to learn.

[0982] A "terminal" is an electronic device used by a learner to enter homework content, and examples include tablets and personal computers.

[0983] "Natural language processing technology" is a computer technology that analyzes input text data and understands specific meanings and contexts, and uses NLP libraries and algorithms.

[0984] A "learning category" is a category for classifying homework content into specific fields or types, and examples include calculation problems, Kanji, English, etc.

[0985] A "game engine" is a software platform for generating game-style problems, and examples include Unity and Unreal Engine.

[0986] "Interactive" refers to a function that allows learners to directly operate and input information through their terminal.

[0987] A "database" is a system that stores learners' answer data and learning history, and collates and manages the data as needed.

[0988] "Rewards" are incentives such as points or badges that are provided to motivate learners when they answer questions correctly.

[0989] "Progress data" is data used to record the results of questions answered by a learner and the answer status, and to track the progress of learning.

[0990] MODE FOR CARRYING OUT THE INVENTION

[0991] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner. The specific processing content of the program of this system and the hardware and software used will be described in detail below.

[0992] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. Specifically, the learner enters the homework content in the text box displayed on the device screen and clicks the "Convert homework into a game" button.

[0993] This input is sent from the device to a server, which then analyzes the received homework content using natural language processing (NLP) techniques. Suitable NLP libraries to use are "spaCy" and "NLTK." The text data is analyzed, and the homework content is classified into specific learning categories (e.g., math problems, Kanji, English, etc.).

[0994] The server then generates game-style questions based on the classified learning categories. The software used for this is a game engine such as Unity or Unreal Engine. For example, a quiz-style game data set containing 10 arithmetic calculation questions is generated.

[0995] The generated game data is sent from the server to the user's device. The device receives this game data and displays it on the screen in an interactive format. The learner can answer questions and enter their answers through the on-screen interface. The learner's answer data is then sent back to the server from the device.

[0996] The server checks the received answers against pre-set correct answer data using a database. For example, a database system such as MySQL is used. The evaluation results are sent back to the device and displayed to the learner. If the learner answers correctly, they can proceed to the next question, but if they answer incorrectly, they are given the option to try again.

[0997] Furthermore, the server records the learner's answer data and progress data, and uses BaaS (Backend as a Service) such as "Firebase" to update the learning history. This data is used when generating and suggesting questions for the next learning session. In addition, rewards such as points and badges are generated for correctly answered questions and sent to the device. The device displays reward information to the learner in real time, increasing their motivation to learn.

[0998] Specific examples

[0999] 1. Math homework

[1000] The learner types in "Today's math homework is a calculation problem." The server analyzes this and classifies it into the calculation problem category. The server generates a quiz-style game containing 10 arithmetic operation problems. The learner answers the questions displayed on the device one by one. The server evaluates the answers and displays the results on the device. When the learner answers all the questions, they are awarded points that are reflected in the next learning material.

[1001] Prompt Sentence Examples

[1002] Today's math homework is a calculation problem. Turn it into a game.

[1003] 2. English homework

[1004] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. The server then generates a puzzle-style game in which the learner answers the meaning of the word. The learner answers the puzzle displayed on the terminal by pairing words with their meanings. The server evaluates the answers, and if all pairs are correct, a reward is provided.

[1005] Prompt Sentence Examples

[1006] Today's English homework is to memorize the meanings of words. Turn this into a game.

[1007] In this way, the present invention automatically analyzes homework content and provides it in a game format, thereby increasing learners' motivation and supporting effective learning. By utilizing the entire system, the effectiveness of home study and online classes can be maximized.

[1008] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1009] Step 1:

[1010] The user uses the device to input the homework content. Specifically, the user inputs the homework content into the text box displayed on the device screen and clicks the "Convert homework to game" button. For example, the user might input "Today's math homework is a calculation problem." This input content is sent from the device to the server. The input data is the homework content in text format, and this is output to the server.

[1011] Step 2:

[1012] The server analyzes the received homework content. Specifically, it uses natural language processing (NLP) technology to analyze the input text. Suitable NLP libraries to use are "spaCy" and "NLTK." Through this analysis, the server classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). The input data is the transmitted text data, and the output data is the classification results.

[1013] Step 3:

[1014] The server generates game-style questions based on the classified learning categories. Specifically, it uses a game engine such as Unity or Unreal Engine to create an interactive game. For example, it generates quiz-style game data containing 10 calculation questions on arithmetic operations. The input data is the learning categories, and the output data is the generated game data.

[1015] Step 4:

[1016] The server sends the generated game data to the user's device. This causes the user's device to receive game-style questions. The input data is the generated game data, and the output data is a notification to the device that transmission has been completed. Specifically, the server divides the game data into packets and sends them over the network.

[1017] Step 5:

[1018] The terminal displays the received game-style questions on the screen. The user can answer the questions through the on-screen interface. For example, in the case of a calculation question, the user looks at the calculation formula displayed on the screen and enters the answer in the answer input field. The input data is the received game data, and the output data is the user's answer input.

[1019] Step 6:

[1020] The terminal sends the user's answer data to the server. Specifically, the answer entered by the user is divided into packets and sent to the server. The input data is the user's answer data, and the output data is a transmission completion notification to the server.

[1021] Step 7:

[1022] The server evaluates the received answer data. Specifically, it compares it with pre-set correct answer data using a database. A suitable database to use is MySQL. The input data is the user's answer data, and the output data is the evaluation result.

[1023] Step 8:

[1024] The server returns the evaluation results to the user's terminal, which then displays the results to the user. The input data is the evaluation results, and the output data is a transmission completion notification to the terminal. Specifically, the success / failure information is divided into packets and sent to the terminal.

[1025] Step 9:

[1026] The server records the user's progress data and learning history. Specifically, it uses BaaS such as "Firebase" to update the learning history database. This data is used to generate and suggest questions for the next learning session. The input data is the learner's answer data and progress data, and the output data is the updated learning history.

[1027] Step 10:

[1028] The server generates rewards for learners and sends them to the device. Specifically, rewards such as points and badges are generated for questions that the learner answers correctly. The input data is the evaluation results, and the output data is the generated reward data. The device displays the reward information in real time and notifies the learner.

[1029] (Application example 1)

[1030] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1031] In conventional educational systems, there are problems such as learners finding it difficult to maintain their motivation while completing their homework. Also, there are issues with the lack of ways to attract customers' attention when obtaining product information in physical stores. This can lead to a decrease in learning effectiveness and a lack of understanding of products.

[1032] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1033] In this invention, the server includes means for a learner to input homework content, means for analyzing the input homework content and classifying it into a specific learning category, means for generating game-style questions according to the classified learning category and transmitting them to the terminal, means for a customer to input product information, means for analyzing the input product information and classifying it into a specific product category, and means for generating game-style information presentations according to the classified product category and transmitting them to the terminal. This allows learners to enjoy doing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[1034] "Learner" means a person who uses an educational system to carry out learning activities.

[1035] "Homework content" refers to the learning tasks assigned to students by their school or teaching materials.

[1036] "Input means" refers to the interface through which a learner or customer provides information to the system.

[1037] "Analysis means" refers to technology for processing input information and understanding its content.

[1038] "Learning categories" are criteria for classifying educational content into specific themes or subjects.

[1039] "Game-style questions" are questions that include game elements in order to enhance learning effectiveness.

[1040] "Device" means an electronic device (e.g., tablet, smartphone, smart glasses) used by a learner or customer.

[1041] A "display means" is a method for visually presenting learning or information content on a terminal.

[1042] "Means for interactive answering" means a method by which a learner or customer can input answers through a terminal and interact with the system in a two-way manner.

[1043] "Means for evaluating answers" refers to a system for evaluating the answers of learners or customers and determining whether they are correct or incorrect.

[1044] The "means of presenting results" is the method by which the results of the assessment are communicated to the learner or client.

[1045] A "means for recording progress" is a system that stores learner or customer activity data.

[1046] "Means for managing learning history" refers to a system for effectively organizing recorded progress data and utilizing it for future learning or information provision.

[1047] "Customer" means a person who uses the system to obtain product information in a physical store.

[1048] "Product information" refers to information about the features and usage of products sold in physical stores.

[1049] A "product category" is a category in which products are classified according to specific criteria.

[1050] "Information presentation" refers to a method for displaying information about a product to a customer.

[1051] System Overview

[1052] This invention provides a system that allows learners and customers in brick-and-mortar stores to learn and acquire product information in an interactive game format. This system uses devices such as tablets, smartphones, and smart glasses, and processes and provides information in cooperation with a server.

[1053] Hardware and software used

[1054] Hardware:

[1055] Tablet (e.g. iPad)

[1056] Smartphone (e.g. iPhone or Android device)

[1057] Smart glasses (e.g. Google Glass)

[1058] software:

[1059] Natural Language Processing (NLP) libraries (e.g., SpaCy, NLTK)

[1060] Game engines (e.g. Unity, Cocos2d)

[1061] Backend server (e.g. AWS Lambda, Django Rest Framework)

[1062] Explanation of program processing

[1063] Step 1: Learner or customer enters information

[1064] Users input homework details and product information using devices (tablets, smartphones, etc.). Specifically, for example, a student might input "Today's math homework is a calculation problem," while a customer at a physical store might input "I want to know about cosmetics."

[1065] Step 2: Server analysis and classification

[1066] The server uses natural language processing (NLP) technology to analyze the input text and classify it into appropriate learning or product categories, such as "math problems" or "cosmetics."

[1067] Step 3: Generate and send game-style questions and information prompts

[1068] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and the generated content is sent to the terminal.

[1069] Step 4: Terminal display and interaction

[1070] The terminal displays game-style questions and information received from the server on its screen. The learner or customer looks at this screen and inputs answers to each question. For example, if it is a calculation question, the learner answers the formula displayed on the screen, and if it is cosmetic information, the learner answers the related quiz.

[1071] Step 5: Server evaluation and feedback of answers

[1072] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[1073] Step 6: Track progress and provide rewards

[1074] The server records the learner's or customer's answer data and progress data, and updates the learning history and customer history. This data is used to generate and suggest questions for the next learning session, as well as to provide product information. Correct answers to questions can provide rewards such as points or coupons.

[1075] Specific examples

[1076] 1. Math homework

[1077] A learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time a learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[1078] 2. Cosmetics information

[1079] A customer types in "I want to know more about cosmetics." The server analyzes this and classifies it into a cosmetics category. A quiz-style information presentation about the product's ingredients and usage is generated and displayed on the terminal. When the customer answers the quiz, their answers are evaluated and if they are correct, they are given a coupon as a reward.

[1080] Prompt Sentence Examples

[1081] I entered "I want to know more about cosmetics." Based on this information, please generate the following trivia and quiz questions:

[1082] 1. Cosmetic Ingredients Quiz

[1083] 2. Best Practices for Usage

[1084] 3. Product reviews and recommended usage

[1085] This system allows learners to enjoy completing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[1086] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1087] Step 1:

[1088] Learner or customer input of information

[1089] The user inputs homework details and product information using a device (tablet, smartphone, etc.).

[1090] Input: A learner types, "Today's math homework is a calculation problem." Or a customer types, "I want to know about cosmetics."

[1091] Data processing: Obtain the input text data and send it to the server.

[1092] Output: The text data sent to the server.

[1093] Step 2:

[1094] Server-based information analysis and classification

[1095] The server uses natural language processing (NLP) techniques to analyze the input text and classify it into appropriate learning and product categories.

[1096] Input: Text data (homework assignments or product information).

[1097] Data processing: Analyze the text using an NLP library (e.g., SpaCy, NLTK) to extract specific keywords and phrases. Then, classify the text into learning or product categories based on these keywords.

[1098] Output: Category information (e.g., "Math Problem", "Cosmetics").

[1099] Step 3:

[1100] Generate and send game-style questions and information presentations

[1101] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and transmits them to the terminal.

[1102] Input: Category information (e.g., "Math Problems," "Cosmetics").

[1103] Data processing: Using a game engine (e.g. Unity, Cocos2d), create game-style questions and information presentations according to the category.

[1104] Output: Generated game-style questions and information presentation data. Send to device.

[1105] Step 4:

[1106] Terminal display and interaction

[1107] The terminal displays game-style questions and information received from the server on its screen, and the learner or customer inputs answers to each question.

[1108] Input: Game-style questions and information presentation data sent from the server.

[1109] Data processing: The device analyzes the data received and displays it on the user interface.

[1110] How it works: The learner or customer enters an answer to a question, and the device sends the answer to the server.

[1111] Output: User's answer data.

[1112] Step 5:

[1113] Server-based answer evaluation and feedback

[1114] The server receives the user's answer, compares it with pre-set correct answer data, evaluates its accuracy, and returns the evaluation result to the terminal.

[1115] Input: User's answer data.

[1116] Data processing: The server checks the answer data against the correct answer database to determine whether it is correct or incorrect, and generates a feedback message indicating whether the answer is correct or incorrect.

[1117] Output: A feedback message (e.g., "Correct" if correct, "Incorrect" if incorrect) sent back to the device.

[1118] Step 6:

[1119] Track progress and provide rewards

[1120] The server records the answer data and progress data of the learner or customer, updates the learning history and customer history, and provides rewards (points or coupons) for correct answers.

[1121] Input: User answer data, feedback message.

[1122] Data processing: The server records the answer results in the progress management system and updates the user's history. If the answer is correct, points or coupons are generated using the reward system.

[1123] Output: Updated learning history, customer history data, reward data (points and coupons). Send to the terminal.

[1124] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1125] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and also has the function of recognizing the learner's emotions and optimizing the learning process. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[1126] Program processing

[1127] Students input and submit their homework

[1128] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[1129] Analysis and classification of homework content by the server

[1130] The server analyzes the received input, uses natural language processing (NLP) technology to analyze the text, and classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[1131] Game-style question generation and submission

[1132] Next, the server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[1133] Terminal display of questions and interactive solutions

[1134] The device displays the game-style questions received from the server on the screen. The learner inputs the answers to each question while looking at this screen. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[1135] Server-based answer evaluation and feedback

[1136] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[1137] Emotion recognition and regulation with emotion engine

[1138] To recognize learners' emotions, the system uses a camera and microphone on the device to capture their facial expressions and voice data. An emotion engine on the server analyzes this data and recognizes the learner's emotions (e.g., joy, anger, sadness, excitement, etc.).

[1139] Based on the recognized emotions, the server can adjust the difficulty and content of the gamified questions appropriately. For example, if the learner is tired or confused, it can lower the difficulty of the questions or provide encouraging messages.

[1140] Track progress and provide rewards

[1141] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges via the device for correctly answered questions, thereby increasing learner motivation.

[1142] Specific examples

[1143] 1. Math homework

[1144] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[1145] Additionally, if the system recognizes that a learner is confused based on their facial expression, it will support them by lowering the difficulty of the questions or displaying encouraging messages.

[1146] 2. English homework

[1147] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[1148] Additionally, if the system recognizes from the learner's voice data that the learner is enjoying the lesson, it can add more challenging questions while maintaining the current level of difficulty.

[1149] This system converts homework into a game format, and by appropriately recognizing and adjusting learners' emotions, it increases their motivation to learn and promotes effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[1150] The processing flow will be explained below.

[1151] Step 1:

[1152] The user inputs the homework content.

[1153] The user uses the device to enter specific homework content into the text box and clicks the "Convert homework to game" button, which is then sent from the device to the server.

[1154] Step 2:

[1155] The server receives the homework content.

[1156] The server receives the homework content sent from the terminal and prepares for analysis.

[1157] Step 3:

[1158] The server analyzes the homework content and classifies it into learning categories.

[1159] The server uses natural language processing (NLP) technology to analyze the homework content and classify it into specific learning categories (e.g., math problems, Kanji, English, etc.).

[1160] Step 4:

[1161] The server will choose the appropriate game format.

[1162] The server selects an appropriate game format (e.g., quiz, puzzle, flashcard, etc.) from the database according to the classified category.

[1163] Step 5:

[1164] The server generates game-style questions.

[1165] The server generates a set of questions in the selected game format. For example, if the questions are math calculation questions, it generates a quiz that includes arithmetic operations.

[1166] Step 6:

[1167] The server sends the generated game to the device.

[1168] The server sends the generated game data to the terminal, which receives this data.

[1169] Step 7:

[1170] The device displays a game-style question.

[1171] The device displays the received game-style questions on the screen, allowing the user to interactively answer them.

[1172] Step 8:

[1173] The device collects the user's emotional data.

[1174] As the user works on the problem, the device uses a camera and microphone to collect the user's facial expressions and voice, and this data is sent to a server for analysis by the emotion engine.

[1175] Step 9:

[1176] The user answers the question.

[1177] The user inputs the answer to the question displayed on the screen. For example, if it is a calculation question, the answer is entered in the answer field. The input answer is sent from the terminal to the server.

[1178] Step 10:

[1179] The server evaluates the answers.

[1180] The server receives the user's answer, judges whether it is correct, compares it with the correct answer data set in advance, and returns the evaluation result to the terminal.

[1181] Step 11:

[1182] The terminal displays the evaluation results.

[1183] The device receives the evaluation results from the server and displays them to the user. If the answer is correct, the user is given the option to proceed to the next question, and if the answer is incorrect, the user is given the option to try again.

[1184] Step 12:

[1185] The server analyzes the user's emotional data.

[1186] The server's emotion engine analyzes the received facial and voice data and recognizes the user's emotions (e.g., joy, anger, sadness, confusion, etc.).

[1187] Step 13:

[1188] The server adjusts the game content.

[1189] Based on the recognized emotions, the server adjusts the difficulty and content of the game. For example, if the user is having trouble, it may lower the difficulty of the problems or send a message of encouragement.

[1190] Step 14:

[1191] The server records the progress data.

[1192] The server records the user's answers and progress data, and updates the learning history, which is used to generate questions and suggestions for the next learning session.

[1193] Step 15:

[1194] The terminal provides the reward.

[1195] The device provides users with rewards such as points and badges for correctly answering questions.

[1196] Step 16:

[1197] The server suggests what to learn next.

[1198] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and emotional data, and sends them to the device, which then displays them to the user.

[1199] The above processing steps allow users to enjoy completing their homework in a game-like manner, effectively mastering the learning content. Furthermore, the introduction of an emotion engine optimizes the learning process for each individual user, which is expected to increase their motivation to learn.

[1200] Example 2

[1201] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1202] Conventional learning support systems often progress homework in a monotonous manner, which can easily lower learners' motivation. Furthermore, they provide problems with a uniform level of difficulty without taking learners' emotions into consideration, which results in the problem of not maximizing learning effectiveness. Furthermore, they do not provide appropriate feedback or adjustments, which results in lower learning efficiency.

[1203] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a learner to input homework content, a means for analyzing the input homework content and classifying it into a specific learning category, a means for generating game-style questions according to the classified learning category and transmitting them to the terminal, a means for displaying the game-style questions received by the terminal so that the learner can answer them interactively, a means for evaluating the learner's answers and presenting the results, a means for recording the learner's progress and managing the learning history, and a means for recognizing the learner's emotions and adjusting the difficulty and content of the questions. This enables the learner to actively engage in learning while having fun, thereby realizing effective learning support.

[1204] "Student" refers to an entity that uses the system to carry out learning activities.

[1205] "Means for inputting homework content" refers to the input devices and software functions that allow learners to input homework content into the system.

[1206] "Means for analyzing homework content and classifying it into specific learning categories" refers to a software function that analyzes the input homework content using natural language processing technology, etc., and assigns the content to specific learning categories.

[1207] "Means for generating game-style questions and sending them to the device" refers to a software function for automatically creating appropriate game-style questions based on the analyzed homework content and sending the data to the learner's device.

[1208] "Means for interactive answering" refers to software or hardware that allows learners to input answers to questions posed on their devices in real time.

[1209] "Means for evaluating answers and presenting results" refers to software functionality for automatically evaluating a learner's answers and notifying the learner of the evaluation results.

[1210] "Means for recording progress and managing learning history" refers to software functions that record learners' answers and progress in a database and manage their learning history based on that.

[1211] "Means for recognizing emotions and adjusting the difficulty and content of questions" refers to software functions that analyze data such as the learner's facial expressions and voice to recognize their emotions, and dynamically change the difficulty and content of questions based on the results.

[1212] "Means for providing rewards such as points or badges" refers to software functionality that automatically awards rewards such as points or badges to learners for correctly answering questions.

[1213] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and further has the function of recognizing the learner's emotions and optimizing the learning process. Specific embodiments for carrying out the present invention will now be described.

[1214] System configuration

[1215] Students input and submit their homework

[1216] The learner, who is the user, inputs the homework content using a device such as a tablet or PC. The learner enters "Today's math homework is a calculation problem" in the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[1217] Analysis and classification of homework content by the server

[1218] The server uses natural language processing (NLP) technology to analyze the received homework content. Specifically, the NLTK library in Python can be used. The server analyzes the input text and classifies the content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[1219] Game-style question generation and submission

[1220] The server generates game-style questions according to the selected learning category. The Django framework can be used for this generation. For example, in the case of math calculation questions, a quiz-style game containing multiple calculation questions is generated. The generated game data is sent from the server to the device.

[1221] Terminal display of questions and interactive solutions

[1222] The device displays game-style questions received from the server on the screen. The learner looks at the displayed questions and enters their answers. For calculation questions, the learner looks at the calculation formula displayed on the screen and enters the answer in the answer input field. Once the learner has finished entering their answer, the data is sent from the device to the server.

[1223] Server-based answer evaluation and feedback

[1224] The server receives the learner's answer data and compares it with pre-set correct answer data to evaluate whether it is correct or incorrect. An SQL database can be used for this evaluation. The server generates an evaluation result and sends it back to the device for presentation to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[1225] Emotion recognition and regulation with emotion engine

[1226] To recognize learners' emotions, the system uses the device's camera and microphone to capture their facial expressions and voice data. This data is sent to a server where it is analyzed by an emotion engine within the server. Specific technology used is the Microsoft Azure Emotion API. The emotion engine analyzes the learner's emotions (e.g., joy, sadness, anger, excitement, etc.) and adjusts the difficulty and content of the questions based on the results.

[1227] Track progress and provide rewards

[1228] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, the server can provide rewards such as points and badges for questions that the learner answers correctly, which helps to improve the learner's motivation.

[1229] Specific examples

[1230] 1. Math homework

[1231] When a learner types "Today's math homework is a calculation problem," the device sends this information to the server. The server uses NLP technology to recognize the category "calculation problem." It then generates a quiz-style game and sends it to the device. The device displays it, and when the learner enters an answer, the answer data is sent to the server. The server evaluates whether the answer is correct or incorrect and sends the evaluation result back to the device to show it to the learner. If the learner looks confused, the difficulty of the problem is automatically adjusted and a supportive message is displayed.

[1232] Example inputs to a generative AI model:

[1233] User: "Today's math homework includes calculation problems."

[1234] System: "Recognizes calculation problem categories and generates 10 arithmetic operation problems."

[1235] Server: "Evaluate the issue and provide feedback"

[1236] 2. English homework

[1237] When a learner types "Today's English homework is to memorize the meanings of words," the device sends this information to the server. The server uses NLP technology to recognize the "word learning" category. It then generates a puzzle-style game in which the learner must guess the meaning of the word and sends it to the device. When the learner types their answer, the data is sent to the server, which evaluates it and provides feedback. If the server recognizes that the learner is enjoying the game, it adds more challenging questions at the same difficulty level.

[1238] Example inputs to a generative AI model:

[1239] User: "Today's English homework is to memorize the meanings of words."

[1240] System: "Recognizes word learning categories and generates puzzle-style games"

[1241] Server: "Evaluate the issue and provide feedback"

[1242] Thus, the present invention is an optimal learning support system that enables learners to study in an enjoyable and effective manner.

[1243] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1244] Step 1:

[1245] The user inputs the homework details using a device such as a tablet or PC. The specific action is to enter "Today's math homework is a calculation problem" into the text box and click the "Convert homework into a game" button. The entered text data is sent from the device to the server. The input of this step is the text entered by the user, and the output is the text data sent to the server.

[1246] Step 2:

[1247] The server analyzes the received text data of the homework content using natural language processing technology. Specifically, it uses Python's NLTK library to perform morphological analysis and classify the homework content into specific learning categories. The input for this step is the text data submitted by the user, and the output is a specific learning category (e.g., calculation problems, Kanji, English, etc.).

[1248] Step 3:

[1249] The server generates appropriate game-style questions based on the classified learning categories. For this generation, the Django framework is used to create a quiz-style game containing multiple calculation questions. Specifically, the server determines the type of question and its difficulty level, and builds comprehensive quiz data. The input for this step is the learning categories, and the output is the generated game-style question data.

[1250] Step 4:

[1251] The server sends the generated game-style question data to the terminal. The terminal uses the received question data to display it on the screen. Specifically, the terminal displays quiz-style questions in an interactive interface, allowing the learner to enter answers. The input for this step is the game-style question data, and the output is the question interface displayed on the terminal.

[1252] Step 5:

[1253] The learner, who is the user, enters answers to the displayed quiz-style questions. The answers are sent from the terminal to the server. Specifically, the learner enters the answer to the calculation question and clicks the "Submit" button. The input in this step is the learner's answer data, and the output is the answer data sent from the terminal to the server.

[1254] Step 6:

[1255] The server compares the received answer data with pre-set correct answer data and evaluates whether the answer is correct or incorrect. This is done by using an SQL database to compare the correct answer data with the learner's answer. The evaluation results are generated as feedback data. The input to this step is the learner's answer data, and the output is the server's evaluation result (correct or incorrect).

[1256] Step 7:

[1257] The server returns the evaluation result data to the terminal, which then presents the result to the user. Specifically, the terminal displays a feedback message on the screen and provides the option to proceed to the next question or try again. The input of this step is the evaluation result data from the server, and the output is the feedback message presented to the user.

[1258] Step 8:

[1259] The device captures the learner's facial expressions and voice data using a camera and microphone, and sends that data to the server. The server then analyzes this data using an emotion engine to recognize the learner's emotions (e.g., joy, sadness, anger, excitement, etc.). Specifically, it uses Microsoft Azure's Emotion API. The input for this step is the learner's facial expressions and voice data, and the output is the emotion analysis results by the server.

[1260] Step 9:

[1261] The server readjusts the difficulty and content of the game-style questions based on the emotion analysis results. For example, if the learner is tired, it makes specific adjustments such as lowering the difficulty of the questions or providing encouraging messages. The input of this step is the emotion analysis results, and the output is the adjusted question data or encouraging messages.

[1262] Step 10:

[1263] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges for correctly answered questions and notifies the learner of this via their device. Specifically, it includes a system function that automatically manages appropriate rewards according to the learner's learning progress. The input to this step is the learner's progress data and answer data, and the output is an updated learning history and reward data.

[1264] (Application example 2)

[1265] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1266] Conventional learning systems lacked the means to make homework fun for learners, making it feel like a game, and limited the means to increase motivation. Furthermore, there were no systems that recognized learners' emotions and provided appropriate feedback. These shortcomings led to problems such as a lack of appropriate responses when learners lost interest in learning, or felt confused or fatigued, resulting in a decline in learning efficiency.

[1267] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a learner to input homework content; means for analyzing the input homework content and classifying it into specific learning categories; means for generating game-style questions according to the classified learning categories and transmitting them to the terminal; means for displaying the game-style questions received by the terminal so that the learner can answer them interactively; means for evaluating the learner's answers and presenting the results; means for recording the learner's progress and managing the learning history; means for recognizing and analyzing the learner's emotions; and means for adjusting the difficulty and content of the questions based on the recognized emotions. This enables the learner to study enjoyably and effectively, and by providing appropriate feedback according to emotions, learning efficiency is improved and motivation is maintained.

[1268] "Student" refers to a user who uses the system to do homework or study.

[1269] "Means for inputting homework content" refers to the interface that allows learners to input the specific content of their homework via a terminal.

[1270] "Means for analyzing the input homework content and classifying it into specific learning categories" refers to a program or device that uses natural language processing technology to analyze the homework content input by the learner and classify it into learning categories such as mathematics or English.

[1271] "Means for generating game-style questions and sending them to the terminal" refers to a system for creating game-style questions such as quizzes and puzzles based on the analyzed homework content and sending that data to the learner's terminal.

[1272] "A means for displaying questions on a terminal and allowing learners to answer them interactively" refers to an interface that displays the sent game-style questions on the screen of the learner's terminal and allows the learner to answer the questions.

[1273] "Means for evaluating answers and presenting results" refers to a system for determining whether the answers entered by the learner are correct or incorrect and providing feedback on the results to the learner.

[1274] "Means for recording learners' progress and managing their learning history" refers to a system that records the content and results of questions answered by learners, and stores and manages them in a database as learning history.

[1275] "Means for recognizing and analyzing learners' emotions" refers to technology that uses cameras and microphones to capture the learners' facial expressions and voices, and analyzes them to determine the learners' emotional state.

[1276] "Means for adjusting the difficulty and content of questions based on the learner's emotions" refers to an algorithm for dynamically changing the difficulty and content of questions based on the learner's emotional state.

[1277] The present invention is a learning system that allows learners to complete their homework more enjoyably and effectively. Specific embodiments of this system will be described below.

[1278] System configuration and program processing

[1279] Hardware and Software Configuration

[1280] This learning system includes the following hardware and software:

[1281] 1. Device: A device used by a user (learner), such as a smartphone, tablet, smart glasses, or head-mounted display.

[1282] 2. Server: A cloud-based server that receives data, analyzes it, generates questions, and performs emotion recognition.

[1283] 3. Software:

[1284] Natural language processing (NLP) technology: Uses Google Cloud Natural Language API.

[1285] Data management: Use Google Firebase to manage learning data.

[1286] Emotion Recognition Model: Emotion recognition is performed using TensorFlow.

[1287] System Operation

[1288] The system operates as follows.

[1289] 1. Learners input and submit their homework

[1290] The user (student) uses the device's application to input the content of their homework via text or voice. For example, using a smartphone, they can input "Today's math homework is 2 x 3 and 4 + 5" and tap the "Convert homework into a game" button. This data is then sent to the server via the Internet.

[1291] 2. Analysis and classification of homework content

[1292] The server uses natural language processing technology to analyze the received homework content, and based on the analysis results, classifies the homework into a specific learning category (e.g., math problems, Kanji, English, etc.).

[1293] 3. Creating and submitting game-style questions

[1294] The server generates game-style questions according to the appropriate learning category based on the analyzed homework content. For example, in the case of calculation questions, quiz-style questions including arithmetic operations are generated. The server then transmits the game-style question data to the device.

[1295] 4. Terminal display and interactive answers

[1296] The terminal displays the game-style questions received from the server to the learner, who then interactively inputs the answers, which are then sent back to the server via the terminal.

[1297] 5. Evaluation and feedback of answers

[1298] The server compares the received answers with pre-defined correct answer data, evaluates the learner's answers, and sends the results to the device to provide feedback to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[1299] 6. Learner emotion recognition and regulation

[1300] The device's camera and microphone are used to capture the learner's facial expressions and voice data, which are then sent to the server. The server's emotion engine (using TensorFlow) analyzes this data and recognizes the learner's emotional state. Based on the recognized emotion, the difficulty and content of the questions are adjusted. For example, if the learner is confused, the questions are made easier or a supportive message is displayed.

[1301] 7. Progress Recording and Rewarding

[1302] The server records the learner's answer data and progress data, and manages their learning history. This history data is used in the next learning session. In addition, learners are given rewards such as points and badges for correctly answering questions.

[1303] Examples of concrete examples and prompts

[1304] Specific examples

[1305] Situations where learners enter homework details on their smartphones:

[1306] The learner types, "Today's math homework is 2 x 3 and 4 + 5."

[1307] The server classifies the homework content into calculation problem categories and generates quiz-style questions.

[1308] The learner answers the quiz, the server evaluates it, and the learner moves on to the next question.

[1309] If the emotion engine detects that the learner is tired, it will easily adjust the questions or display encouraging messages such as "Keep it up! You're almost there!"

[1310] Prompt Sentence Examples

[1311] "If learners are stumped, lower the difficulty and provide a supportive message, such as, 'Don't worry, the next question is easier, so relax!'"

[1312] In this way, this system not only allows learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[1313] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1314] Step 1:

[1315] The user (learner) uses a device to input the homework content. Input is done by text or voice. For example, a learner inputs "Today's math homework is 2 x 3 and 4 + 5" into a smartphone and taps the "Convert homework into a game" button. This data is sent to a server via the Internet. The input data is in text format, and the input is sent to the server.

[1316] Step 2:

[1317] The homework content received by the server is analyzed using natural language processing technology (Google Cloud Natural Language API). Specifically, the input text is analyzed and the homework content is classified into specific learning categories (e.g., calculation problems, kanji, English, etc.). The analyzed data is output as text information classified into learning categories.

[1318] Step 3:

[1319] The server generates appropriate game-style questions based on the classified learning category. For example, if the question is classified as a calculation problem, it generates quiz-style questions that include arithmetic operations. During this process, it selects appropriate questions from a pre-prepared question database and converts them into a game format. The generated game-style question data is then sent to the device.

[1320] Step 4:

[1321] The device displays the received game-style questions to the learner. The questions are displayed using an interactive UI (user interface), and the learner inputs their answers. The answer data is sent to the server in real time. The input answer data becomes new input data for the server.

[1322] Step 5:

[1323] The server compares the received answer data with pre-defined correct answer data and evaluates the answer. The evaluation result is classified as "correct" or "incorrect." The evaluation result is sent to the device and feedback is provided to the learner. For example, if the answer is correct, a button to proceed to the next question is displayed, and if the answer is incorrect, an option to try again is presented.

[1324] Step 6:

[1325] The learner's facial expressions and voice data are captured using the device's camera and microphone. This emotional data is sent to the server in real time. The emotional data is captured as facial images and voice recordings and analyzed by the server.

[1326] Step 7:

[1327] The emotion engine (using TensorFlow) on the server analyzes the emotion data and recognizes the learner's emotional state. For example, it classifies the learner's emotional state as confused, tired, or enjoying. Based on the analysis results, it adjusts the difficulty and content of the questions. The recognized emotional state becomes the input data for adjusting the difficulty of the learning content.

[1328] Step 8:

[1329] The server records the learner's answer data and progress data, and manages the learning history. The progress data is stored in Firebase and used for the next learning session. The learning history database is updated.

[1330] Step 9:

[1331] The server provides rewards such as points and badges for questions that learners answer correctly, thereby increasing learner motivation. The reward data is sent to the learner's device and displayed. For example, a message such as "10 points earned!" or "You've earned a new badge!" is displayed.

[1332] Through these steps, this system not only enables learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[1333] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1335] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1336] [Fourth embodiment]

[1337] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1338] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1340] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1344] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1345] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1346] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1347] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1348] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1349] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1350] This invention is a system that allows learners to enjoyably complete their homework in a game-like manner. This system analyzes the learner's input, generates appropriate game-style questions based on that input, and supports the learner in the process of answering them. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[1351] Program processing

[1352] Students input and submit their homework

[1353] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[1354] Analysis and classification of homework content by the server

[1355] The server then analyzes the received input, using natural language processing (NLP) techniques to analyze the text and classify the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.), and selects appropriate game-style problems based on the classification results.

[1356] Game-style question generation and submission

[1357] The server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[1358] Terminal display of questions and interactive solutions

[1359] The device displays the game-style questions received from the server on the screen. The learner looks at this screen and inputs the answer to each question. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[1360] Server-based answer evaluation and feedback

[1361] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[1362] Track progress and provide rewards

[1363] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, rewards such as points and badges are provided via the device for correctly answered questions, which increases the learner's motivation.

[1364] Specific examples

[1365] 1. Math homework

[1366] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[1367] 2. English homework

[1368] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[1369] In this way, the present invention is a system that converts homework into a game format, thereby increasing learners' concentration and motivation to study and promoting effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[1370] The processing flow will be explained below.

[1371] Step 1:

[1372] The user inputs the homework content.

[1373] The user uses the device to enter the homework content into the text box and clicks the "Convert homework to game" button, which then sends the input data to the server.

[1374] Step 2:

[1375] The server receives the homework content.

[1376] The server receives the homework content sent from the terminal and prepares it for analysis as text data.

[1377] Step 3:

[1378] The server analyzes the homework content and classifies it into categories.

[1379] The server uses natural language processing (NLP) technology to analyze the input text and, based on the analysis results, classifies the homework content into specific learning categories (e.g., math problems, geography problems, kanji problems, English problems, etc.).

[1380] Step 4:

[1381] The server will choose the appropriate game format.

[1382] The server selects an appropriate game format (e.g., quiz format, puzzle format, flash card format, etc.) from the database according to the classified learning category.

[1383] Step 5:

[1384] The server generates game-style questions.

[1385] The server generates a set of questions based on the selected game format. For example, if the game is a math calculation game, a quiz containing 10 arithmetic operations will be generated.

[1386] Step 6:

[1387] The server sends the generated game to the device.

[1388] The server sends the generated game data to the terminal, which receives this data.

[1389] Step 7:

[1390] The device displays a game-style question.

[1391] The device displays the received game data on the screen, presenting the user with questions and answer options.

[1392] Step 8:

[1393] The user answers the question.

[1394] The user inputs or selects an answer to the question displayed on the screen. For example, if it is a calculation question, the user inputs the answer in the input field.

[1395] Step 9:

[1396] The device sends the answer to the server.

[1397] The terminal transmits the user's answer to the server.

[1398] Step 10:

[1399] The server evaluates the answers.

[1400] The server compares the received answers with pre-set correct answer data, determines whether they are correct, and returns the evaluation results to the terminal.

[1401] Step 11:

[1402] The terminal displays the evaluation results.

[1403] The terminal displays the evaluation results received from the server to the user. If the answer is correct, the user has the option to proceed to the next question, and if the answer is incorrect, the user has the option to try again.

[1404] Step 12:

[1405] The server records the progress data.

[1406] The server records the user's answer data and progress data and updates the learning history.

[1407] Step 13:

[1408] The terminal provides the reward.

[1409] The device provides users with rewards such as points and badges for correctly answering questions.

[1410] Step 14:

[1411] The server suggests what to learn next.

[1412] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and sends them to the device, which then displays these suggestions to the user.

[1413] Through the above processing steps, the user can enjoyably complete their homework in a game-like format, and can effectively master the learning content.

[1414] Example 1

[1415] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1416] Traditional homework learning methods are monotonous and burdensome for learners. This can lead to problems such as reduced motivation to learn and difficulty in effective learning. Furthermore, there is a lack of progress management of learning content and feedback tailored to individual needs, which can lead to reduced learning efficiency. Furthermore, since no rewards are offered, maintaining motivation is also an issue.

[1417] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1418] In this invention, the server includes: a means for a learner to input homework content using a terminal; a means for analyzing the input homework content using natural language processing technology and classifying it into specific learning categories; a means for generating game-style questions corresponding to the classified learning categories using a game engine and sending them to the terminal; a means for interactively displaying the game-style questions received by the terminal and allowing the learner to input answers; a means for comparing the learner's answers with a database to evaluate whether they are correct or incorrect and presenting the results; a means for recording the learner's answer data and progress data and managing the learning history; and a means for generating rewards for the learner and sending them to the terminal for display. This allows the learner to enjoyably complete their homework in a game-like format, improving their motivation to learn and promoting effective learning. Furthermore, managing the learning history and providing rewards can help maintain motivation to learn.

[1419] A "terminal" is an electronic device used by a learner to enter homework content, and examples include tablets and personal computers.

[1420] "Natural language processing technology" is a computer technology that analyzes input text data and understands specific meanings and contexts, and uses NLP libraries and algorithms.

[1421] A "learning category" is a category for classifying homework content into specific fields or types, and examples include calculation problems, Kanji, English, etc.

[1422] A "game engine" is a software platform for generating game-style problems, and examples include Unity and Unreal Engine.

[1423] "Interactive" refers to a function that allows learners to directly operate and input information through their terminal.

[1424] A "database" is a system that stores learners' answer data and learning history, and collates and manages the data as needed.

[1425] "Rewards" are incentives such as points or badges that are provided to motivate learners when they answer questions correctly.

[1426] "Progress data" is data used to record the results of questions answered by a learner and the answer status, and to track the progress of learning.

[1427] MODE FOR CARRYING OUT THE INVENTION

[1428] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner. The specific processing content of the program of this system and the hardware and software used will be described in detail below.

[1429] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. Specifically, the learner enters the homework content in the text box displayed on the device screen and clicks the "Convert homework into a game" button.

[1430] This input is sent from the device to a server, which then analyzes the received homework content using natural language processing (NLP) techniques. Suitable NLP libraries to use are "spaCy" and "NLTK." The text data is analyzed, and the homework content is classified into specific learning categories (e.g., math problems, Kanji, English, etc.).

[1431] The server then generates game-style questions based on the classified learning categories. The software used for this is a game engine such as Unity or Unreal Engine. For example, a quiz-style game data set containing 10 arithmetic calculation questions is generated.

[1432] The generated game data is sent from the server to the user's device. The device receives this game data and displays it on the screen in an interactive format. The learner can answer questions and enter their answers through the on-screen interface. The learner's answer data is then sent back to the server from the device.

[1433] The server checks the received answers against pre-set correct answer data using a database. For example, a database system such as MySQL is used. The evaluation results are sent back to the device and displayed to the learner. If the learner answers correctly, they can proceed to the next question, but if they answer incorrectly, they are given the option to try again.

[1434] Furthermore, the server records the learner's answer data and progress data, and uses BaaS (Backend as a Service) such as "Firebase" to update the learning history. This data is used when generating and suggesting questions for the next learning session. In addition, rewards such as points and badges are generated for correctly answered questions and sent to the device. The device displays reward information to the learner in real time, increasing their motivation to learn.

[1435] Specific examples

[1436] 1. Math homework

[1437] The learner types in "Today's math homework is a calculation problem." The server analyzes this and classifies it into the calculation problem category. The server generates a quiz-style game containing 10 arithmetic operation problems. The learner answers the questions displayed on the device one by one. The server evaluates the answers and displays the results on the device. When the learner answers all the questions, they are awarded points that are reflected in the next learning material.

[1438] Prompt Sentence Examples

[1439] Today's math homework is a calculation problem. Turn it into a game.

[1440] 2. English homework

[1441] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. The server then generates a puzzle-style game in which the learner answers the meaning of the word. The learner answers the puzzle displayed on the terminal by pairing words with their meanings. The server evaluates the answers, and if all pairs are correct, a reward is provided.

[1442] Prompt Sentence Examples

[1443] Today's English homework is to memorize the meanings of words. Turn this into a game.

[1444] In this way, the present invention automatically analyzes homework content and provides it in a game format, thereby increasing learners' motivation and supporting effective learning. By utilizing the entire system, the effectiveness of home study and online classes can be maximized.

[1445] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1446] Step 1:

[1447] The user uses the device to input the homework content. Specifically, the user inputs the homework content into the text box displayed on the device screen and clicks the "Convert homework to game" button. For example, the user might input "Today's math homework is a calculation problem." This input content is sent from the device to the server. The input data is the homework content in text format, and this is output to the server.

[1448] Step 2:

[1449] The server analyzes the received homework content. Specifically, it uses natural language processing (NLP) technology to analyze the input text. Suitable NLP libraries to use are "spaCy" and "NLTK." Through this analysis, the server classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). The input data is the transmitted text data, and the output data is the classification results.

[1450] Step 3:

[1451] The server generates game-style questions based on the classified learning categories. Specifically, it uses a game engine such as Unity or Unreal Engine to create an interactive game. For example, it generates quiz-style game data containing 10 calculation questions on arithmetic operations. The input data is the learning categories, and the output data is the generated game data.

[1452] Step 4:

[1453] The server sends the generated game data to the user's device. This causes the user's device to receive game-style questions. The input data is the generated game data, and the output data is a notification to the device that transmission has been completed. Specifically, the server divides the game data into packets and sends them over the network.

[1454] Step 5:

[1455] The terminal displays the received game-style questions on the screen. The user can answer the questions through the on-screen interface. For example, in the case of a calculation question, the user looks at the calculation formula displayed on the screen and enters the answer in the answer input field. The input data is the received game data, and the output data is the user's answer input.

[1456] Step 6:

[1457] The terminal sends the user's answer data to the server. Specifically, the answer entered by the user is divided into packets and sent to the server. The input data is the user's answer data, and the output data is a transmission completion notification to the server.

[1458] Step 7:

[1459] The server evaluates the received answer data. Specifically, it compares it with pre-set correct answer data using a database. A suitable database to use is MySQL. The input data is the user's answer data, and the output data is the evaluation result.

[1460] Step 8:

[1461] The server returns the evaluation results to the user's terminal, which then displays the results to the user. The input data is the evaluation results, and the output data is a transmission completion notification to the terminal. Specifically, the success / failure information is divided into packets and sent to the terminal.

[1462] Step 9:

[1463] The server records the user's progress data and learning history. Specifically, it uses BaaS such as "Firebase" to update the learning history database. This data is used to generate and suggest questions for the next learning session. The input data is the learner's answer data and progress data, and the output data is the updated learning history.

[1464] Step 10:

[1465] The server generates rewards for learners and sends them to the device. Specifically, rewards such as points and badges are generated for questions that the learner answers correctly. The input data is the evaluation results, and the output data is the generated reward data. The device displays the reward information in real time and notifies the learner.

[1466] (Application example 1)

[1467] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1468] In conventional educational systems, there are problems such as learners finding it difficult to maintain their motivation while completing their homework. Also, there are issues with the lack of ways to attract customers' attention when obtaining product information in physical stores. This can lead to a decrease in learning effectiveness and a lack of understanding of products.

[1469] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1470] In this invention, the server includes means for a learner to input homework content, means for analyzing the input homework content and classifying it into a specific learning category, means for generating game-style questions according to the classified learning category and transmitting them to the terminal, means for a customer to input product information, means for analyzing the input product information and classifying it into a specific product category, and means for generating game-style information presentations according to the classified product category and transmitting them to the terminal. This allows learners to enjoy doing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[1471] "Learner" means a person who uses an educational system to carry out learning activities.

[1472] "Homework content" refers to the learning tasks assigned to students by their school or teaching materials.

[1473] "Input means" refers to the interface through which a learner or customer provides information to the system.

[1474] "Analysis means" refers to technology for processing input information and understanding its content.

[1475] "Learning categories" are criteria for classifying educational content into specific themes or subjects.

[1476] "Game-style questions" are questions that include game elements in order to enhance learning effectiveness.

[1477] "Device" means an electronic device (e.g., tablet, smartphone, smart glasses) used by a learner or customer.

[1478] A "display means" is a method for visually presenting learning or information content on a terminal.

[1479] "Means for interactive answering" means a method by which a learner or customer can input answers through a terminal and interact with the system in a two-way manner.

[1480] "Means for evaluating answers" refers to a system for evaluating the answers of learners or customers and determining whether they are correct or incorrect.

[1481] The "means of presenting results" is the method by which the results of the assessment are communicated to the learner or client.

[1482] A "means for recording progress" is a system that stores learner or customer activity data.

[1483] "Means for managing learning history" refers to a system for effectively organizing recorded progress data and utilizing it for future learning or information provision.

[1484] "Customer" means a person who uses the system to obtain product information in a physical store.

[1485] "Product information" refers to information about the features and usage of products sold in physical stores.

[1486] A "product category" is a category in which products are classified according to specific criteria.

[1487] "Information presentation" refers to a method for displaying information about a product to a customer.

[1488] System Overview

[1489] This invention provides a system that allows learners and customers in brick-and-mortar stores to learn and acquire product information in an interactive game format. This system uses devices such as tablets, smartphones, and smart glasses, and processes and provides information in cooperation with a server.

[1490] Hardware and software used

[1491] Hardware:

[1492] Tablet (e.g. iPad)

[1493] Smartphone (e.g. iPhone or Android device)

[1494] Smart glasses (e.g. Google Glass)

[1495] software:

[1496] Natural Language Processing (NLP) libraries (e.g., SpaCy, NLTK)

[1497] Game engines (e.g. Unity, Cocos2d)

[1498] Backend server (e.g. AWS Lambda, Django Rest Framework)

[1499] Explanation of program processing

[1500] Step 1: Learner or customer enters information

[1501] Users input homework details and product information using devices (tablets, smartphones, etc.). Specifically, for example, a student might input "Today's math homework is a calculation problem," while a customer at a physical store might input "I want to know about cosmetics."

[1502] Step 2: Server analysis and classification

[1503] The server uses natural language processing (NLP) technology to analyze the input text and classify it into appropriate learning or product categories, such as "math problems" or "cosmetics."

[1504] Step 3: Generate and send game-style questions and information prompts

[1505] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and the generated content is sent to the terminal.

[1506] Step 4: Terminal display and interaction

[1507] The terminal displays game-style questions and information received from the server on its screen. The learner or customer looks at this screen and inputs answers to each question. For example, if it is a calculation question, the learner answers the formula displayed on the screen, and if it is cosmetic information, the learner answers the related quiz.

[1508] Step 5: Server evaluation and feedback of answers

[1509] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[1510] Step 6: Track progress and provide rewards

[1511] The server records the learner's or customer's answer data and progress data, and updates the learning history and customer history. This data is used to generate and suggest questions for the next learning session, as well as to provide product information. Correct answers to questions can provide rewards such as points or coupons.

[1512] Specific examples

[1513] 1. Math homework

[1514] A learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time a learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[1515] 2. Cosmetics information

[1516] A customer types in "I want to know more about cosmetics." The server analyzes this and classifies it into a cosmetics category. A quiz-style information presentation about the product's ingredients and usage is generated and displayed on the terminal. When the customer answers the quiz, their answers are evaluated and if they are correct, they are given a coupon as a reward.

[1517] Prompt Sentence Examples

[1518] I entered "I want to know more about cosmetics." Based on this information, please generate the following trivia and quiz questions:

[1519] 1. Cosmetic Ingredients Quiz

[1520] 2. Best Practices for Usage

[1521] 3. Product reviews and recommended usage

[1522] This system allows learners to enjoy completing their homework in a game-like manner, and enables customers in physical stores to obtain product information in a game-like format.

[1523] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1524] Step 1:

[1525] Learner or customer input of information

[1526] The user inputs homework details and product information using a device (tablet, smartphone, etc.).

[1527] Input: A learner types, "Today's math homework is a calculation problem." Or a customer types, "I want to know about cosmetics."

[1528] Data processing: Obtain the input text data and send it to the server.

[1529] Output: The text data sent to the server.

[1530] Step 2:

[1531] Server-based information analysis and classification

[1532] The server uses natural language processing (NLP) techniques to analyze the input text and classify it into appropriate learning and product categories.

[1533] Input: Text data (homework assignments or product information).

[1534] Data processing: Analyze the text using an NLP library (e.g., SpaCy, NLTK) to extract specific keywords and phrases. Then, classify the text into learning or product categories based on these keywords.

[1535] Output: Category information (e.g., "Math Problem", "Cosmetics").

[1536] Step 3:

[1537] Generate and send game-style questions and information presentations

[1538] The server generates game-style questions for the learner and game-style information presentations for the customer based on the classified categories, and transmits them to the terminal.

[1539] Input: Category information (e.g., "Math Problems," "Cosmetics").

[1540] Data processing: Using a game engine (e.g. Unity, Cocos2d), create game-style questions and information presentations according to the category.

[1541] Output: Generated game-style questions and information presentation data. Send to device.

[1542] Step 4:

[1543] Terminal display and interaction

[1544] The terminal displays game-style questions and information received from the server on its screen, and the learner or customer inputs answers to each question.

[1545] Input: Game-style questions and information presentation data sent from the server.

[1546] Data processing: The device analyzes the data received and displays it on the user interface.

[1547] How it works: The learner or customer enters an answer to a question, and the device sends the answer to the server.

[1548] Output: User's answer data.

[1549] Step 5:

[1550] Server-based answer evaluation and feedback

[1551] The server receives the user's answer, compares it with pre-set correct answer data, evaluates its accuracy, and returns the evaluation result to the terminal.

[1552] Input: User's answer data.

[1553] Data processing: The server checks the answer data against the correct answer database to determine whether it is correct or incorrect, and generates a feedback message indicating whether the answer is correct or incorrect.

[1554] Output: A feedback message (e.g., "Correct" if correct, "Incorrect" if incorrect) sent back to the device.

[1555] Step 6:

[1556] Track progress and provide rewards

[1557] The server records the answer data and progress data of the learner or customer, updates the learning history and customer history, and provides rewards (points or coupons) for correct answers.

[1558] Input: User answer data, feedback message.

[1559] Data processing: The server records the answer results in the progress management system and updates the user's history. If the answer is correct, points or coupons are generated using the reward system.

[1560] Output: Updated learning history, customer history data, reward data (points and coupons). Send to the terminal.

[1561] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1562] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and also has the function of recognizing the learner's emotions and optimizing the learning process. Below, the program processing of this system is explained in natural language and in detail with concrete examples.

[1563] Program processing

[1564] Students input and submit their homework

[1565] First, the learner, who is the user, inputs the homework content using a device such as a tablet or PC. For example, the learner enters "Today's math homework is a calculation problem" into the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[1566] Analysis and classification of homework content by the server

[1567] The server analyzes the received input, uses natural language processing (NLP) technology to analyze the text, and classifies the homework content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[1568] Game-style question generation and submission

[1569] Next, the server generates game-style questions according to the selected learning category. For example, for math calculation questions, a quiz-style game including multiple calculation questions is generated. The generated game data is sent from the server to the terminal.

[1570] Terminal display of questions and interactive solutions

[1571] The device displays the game-style questions received from the server on the screen. The learner inputs the answers to each question while looking at this screen. For example, if the question is a calculation, the learner looks at the formula displayed on the screen and inputs the answer in the answer input field. The device then sends the user's answers to the server.

[1572] Server-based answer evaluation and feedback

[1573] The server receives the user's answer and evaluates it against pre-defined correct answer data. The evaluation result is sent back to the device, which displays the result to the user. If the answer is correct, the user can proceed to the next question, and if the answer is incorrect, the user is presented with the option to try again.

[1574] Emotion recognition and regulation with emotion engine

[1575] To recognize learners' emotions, the system uses a camera and microphone on the device to capture their facial expressions and voice data. An emotion engine on the server analyzes this data and recognizes the learner's emotions (e.g., joy, anger, sadness, excitement, etc.).

[1576] Based on the recognized emotions, the server can adjust the difficulty and content of the gamified questions appropriately. For example, if the learner is tired or confused, it can lower the difficulty of the questions or provide encouraging messages.

[1577] Track progress and provide rewards

[1578] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges via the device for correctly answered questions, thereby increasing learner motivation.

[1579] Specific examples

[1580] 1. Math homework

[1581] The learner types in "Today's math homework is a calculation problem." The server analyzes this, classifies it into a calculation problem category, and generates an appropriate game. For example, a quiz containing 10 arithmetic operation problems may be created. Each time the learner answers a question displayed on the device, the server evaluates the answer and controls the learner to move on to the next question. When all questions are answered, points are awarded as a reward.

[1582] Additionally, if the system recognizes that a learner is confused based on their facial expression, it will support them by lowering the difficulty of the questions or displaying encouraging messages.

[1583] 2. English homework

[1584] The learner types in "Today's English homework is to memorize the meanings of words." The server analyzes this and classifies it into a word learning category. A puzzle-style game in which the learner must guess the meaning of the word is generated and displayed on the device. As the learner pairs words with their meanings, their answers are evaluated and a reward is provided if all pairs are correct.

[1585] Additionally, if the system recognizes from the learner's voice data that the learner is enjoying the lesson, it can add more challenging questions while maintaining the current level of difficulty.

[1586] This system converts homework into a game format, and by appropriately recognizing and adjusting learners' emotions, it increases their motivation to learn and promotes effective learning. This system is suitable for a wide range of learners, from elementary school students to junior high school students, and can maximize the effectiveness of home learning and online classes.

[1587] The processing flow will be explained below.

[1588] Step 1:

[1589] The user inputs the homework content.

[1590] The user uses the device to enter specific homework content into the text box and clicks the "Convert homework to game" button, which is then sent from the device to the server.

[1591] Step 2:

[1592] The server receives the homework content.

[1593] The server receives the homework content sent from the terminal and prepares for analysis.

[1594] Step 3:

[1595] The server analyzes the homework content and classifies it into learning categories.

[1596] The server uses natural language processing (NLP) technology to analyze the homework content and classify it into specific learning categories (e.g., math problems, Kanji, English, etc.).

[1597] Step 4:

[1598] The server will choose the appropriate game format.

[1599] The server selects an appropriate game format (e.g., quiz, puzzle, flashcard, etc.) from the database according to the classified category.

[1600] Step 5:

[1601] The server generates game-style questions.

[1602] The server generates a set of questions in the selected game format. For example, if the questions are math calculation questions, it generates a quiz that includes arithmetic operations.

[1603] Step 6:

[1604] The server sends the generated game to the device.

[1605] The server sends the generated game data to the terminal, which receives this data.

[1606] Step 7:

[1607] The device displays a game-style question.

[1608] The device displays the received game-style questions on the screen, allowing the user to interactively answer them.

[1609] Step 8:

[1610] The device collects the user's emotional data.

[1611] As the user works on the problem, the device uses a camera and microphone to collect the user's facial expressions and voice, and this data is sent to a server for analysis by the emotion engine.

[1612] Step 9:

[1613] The user answers the question.

[1614] The user inputs the answer to the question displayed on the screen. For example, if it is a calculation question, the answer is entered in the answer field. The input answer is sent from the terminal to the server.

[1615] Step 10:

[1616] The server evaluates the answers.

[1617] The server receives the user's answer, judges whether it is correct, compares it with the correct answer data set in advance, and returns the evaluation result to the terminal.

[1618] Step 11:

[1619] The terminal displays the evaluation results.

[1620] The device receives the evaluation results from the server and displays them to the user. If the answer is correct, the user is given the option to proceed to the next question, and if the answer is incorrect, the user is given the option to try again.

[1621] Step 12:

[1622] The server analyzes the user's emotional data.

[1623] The server's emotion engine analyzes the received facial and voice data and recognizes the user's emotions (e.g., joy, anger, sadness, confusion, etc.).

[1624] Step 13:

[1625] The server adjusts the game content.

[1626] Based on the recognized emotions, the server adjusts the difficulty and content of the game. For example, if the user is having trouble, it may lower the difficulty of the problems or send a message of encouragement.

[1627] Step 14:

[1628] The server records the progress data.

[1629] The server records the user's answers and progress data, and updates the learning history, which is used to generate questions and suggestions for the next learning session.

[1630] Step 15:

[1631] The terminal provides the reward.

[1632] The device provides users with rewards such as points and badges for correctly answering questions.

[1633] Step 16:

[1634] The server suggests what to learn next.

[1635] The server generates appropriate learning content and questions for the next lesson based on the recorded learning history and emotional data, and sends them to the device, which then displays them to the user.

[1636] The above processing steps allow users to enjoy completing their homework in a game-like manner, effectively mastering the learning content. Furthermore, the introduction of an emotion engine optimizes the learning process for each individual user, which is expected to increase their motivation to learn.

[1637] Example 2

[1638] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1639] Conventional learning support systems often progress homework in a monotonous manner, which can easily lower learners' motivation. Furthermore, they provide problems with a uniform level of difficulty without taking learners' emotions into consideration, which results in the problem of not maximizing learning effectiveness. Furthermore, they do not provide appropriate feedback or adjustments, which results in lower learning efficiency.

[1640] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a learner to input homework content, a means for analyzing the input homework content and classifying it into a specific learning category, a means for generating game-style questions according to the classified learning category and transmitting them to the terminal, a means for displaying the game-style questions received by the terminal so that the learner can answer them interactively, a means for evaluating the learner's answers and presenting the results, a means for recording the learner's progress and managing the learning history, and a means for recognizing the learner's emotions and adjusting the difficulty and content of the questions. This enables the learner to actively engage in learning while having fun, thereby realizing effective learning support.

[1641] "Student" refers to an entity that uses the system to carry out learning activities.

[1642] "Means for inputting homework content" refers to the input devices and software functions that allow learners to input homework content into the system.

[1643] "Means for analyzing homework content and classifying it into specific learning categories" refers to a software function that analyzes the input homework content using natural language processing technology, etc., and assigns the content to specific learning categories.

[1644] "Means for generating game-style questions and sending them to the device" refers to a software function for automatically creating appropriate game-style questions based on the analyzed homework content and sending the data to the learner's device.

[1645] "Means for interactive answering" refers to software or hardware that allows learners to input answers to questions posed on their devices in real time.

[1646] "Means for evaluating answers and presenting results" refers to software functionality for automatically evaluating a learner's answers and notifying the learner of the evaluation results.

[1647] "Means for recording progress and managing learning history" refers to software functions that record learners' answers and progress in a database and manage their learning history based on that.

[1648] "Means for recognizing emotions and adjusting the difficulty and content of questions" refers to software functions that analyze data such as the learner's facial expressions and voice to recognize their emotions, and dynamically change the difficulty and content of questions based on the results.

[1649] "Means for providing rewards such as points or badges" refers to software functionality that automatically awards rewards such as points or badges to learners for correctly answering questions.

[1650] The present invention is a system that allows learners to enjoyably complete their homework in a game-like manner, and further has the function of recognizing the learner's emotions and optimizing the learning process. Specific embodiments for carrying out the present invention will now be described.

[1651] System configuration

[1652] Students input and submit their homework

[1653] The learner, who is the user, inputs the homework content using a device such as a tablet or PC. The learner enters "Today's math homework is a calculation problem" in the text box and clicks the "Convert homework into a game" button. This input is sent to the server via the device.

[1654] Analysis and classification of homework content by the server

[1655] The server uses natural language processing (NLP) technology to analyze the received homework content. Specifically, the NLTK library in Python can be used. The server analyzes the input text and classifies the content into specific learning categories (e.g., math problems, Kanji, English, etc.). Based on the classification results, an appropriate game-style problem is selected.

[1656] Game-style question generation and submission

[1657] The server generates game-style questions according to the selected learning category. The Django framework can be used for this generation. For example, in the case of math calculation questions, a quiz-style game containing multiple calculation questions is generated. The generated game data is sent from the server to the device.

[1658] Terminal display of questions and interactive solutions

[1659] The device displays game-style questions received from the server on the screen. The learner looks at the displayed questions and enters their answers. For calculation questions, the learner looks at the calculation formula displayed on the screen and enters the answer in the answer input field. Once the learner has finished entering their answer, the data is sent from the device to the server.

[1660] Server-based answer evaluation and feedback

[1661] The server receives the learner's answer data and compares it with pre-set correct answer data to evaluate whether it is correct or incorrect. An SQL database can be used for this evaluation. The server generates an evaluation result and sends it back to the device for presentation to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[1662] Emotion recognition and regulation with emotion engine

[1663] To recognize learners' emotions, the system uses the device's camera and microphone to capture their facial expressions and voice data. This data is sent to a server where it is analyzed by an emotion engine within the server. Specific technology used is the Microsoft Azure Emotion API. The emotion engine analyzes the learner's emotions (e.g., joy, sadness, anger, excitement, etc.) and adjusts the difficulty and content of the questions based on the results.

[1664] Track progress and provide rewards

[1665] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. In addition, the server can provide rewards such as points and badges for questions that the learner answers correctly, which helps to improve the learner's motivation.

[1666] Specific examples

[1667] 1. Math homework

[1668] When a learner types "Today's math homework is a calculation problem," the device sends this information to the server. The server uses NLP technology to recognize the category "calculation problem." It then generates a quiz-style game and sends it to the device. The device displays it, and when the learner enters an answer, the answer data is sent to the server. The server evaluates whether the answer is correct or incorrect and sends the evaluation result back to the device to show it to the learner. If the learner looks confused, the difficulty of the problem is automatically adjusted and a supportive message is displayed.

[1669] Example inputs to a generative AI model:

[1670] User: "Today's math homework includes calculation problems."

[1671] System: "Recognizes calculation problem categories and generates 10 arithmetic operation problems."

[1672] Server: "Evaluate the issue and provide feedback"

[1673] 2. English homework

[1674] When a learner types "Today's English homework is to memorize the meanings of words," the device sends this information to the server. The server uses NLP technology to recognize the "word learning" category. It then generates a puzzle-style game in which the learner must guess the meaning of the word and sends it to the device. When the learner types their answer, the data is sent to the server, which evaluates it and provides feedback. If the server recognizes that the learner is enjoying the game, it adds more challenging questions at the same difficulty level.

[1675] Example inputs to a generative AI model:

[1676] User: "Today's English homework is to memorize the meanings of words."

[1677] System: "Recognizes word learning categories and generates puzzle-style games"

[1678] Server: "Evaluate the issue and provide feedback"

[1679] Thus, the present invention is an optimal learning support system that enables learners to study in an enjoyable and effective manner.

[1680] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1681] Step 1:

[1682] The user inputs the homework details using a device such as a tablet or PC. The specific action is to enter "Today's math homework is a calculation problem" into the text box and click the "Convert homework into a game" button. The entered text data is sent from the device to the server. The input of this step is the text entered by the user, and the output is the text data sent to the server.

[1683] Step 2:

[1684] The server analyzes the received text data of the homework content using natural language processing technology. Specifically, it uses Python's NLTK library to perform morphological analysis and classify the homework content into specific learning categories. The input for this step is the text data submitted by the user, and the output is a specific learning category (e.g., calculation problems, Kanji, English, etc.).

[1685] Step 3:

[1686] The server generates appropriate game-style questions based on the classified learning categories. For this generation, the Django framework is used to create a quiz-style game containing multiple calculation questions. Specifically, the server determines the type of question and its difficulty level, and builds comprehensive quiz data. The input for this step is the learning categories, and the output is the generated game-style question data.

[1687] Step 4:

[1688] The server sends the generated game-style question data to the terminal. The terminal uses the received question data to display it on the screen. Specifically, the terminal displays quiz-style questions in an interactive interface, allowing the learner to enter answers. The input for this step is the game-style question data, and the output is the question interface displayed on the terminal.

[1689] Step 5:

[1690] The learner, who is the user, enters answers to the displayed quiz-style questions. The answers are sent from the terminal to the server. Specifically, the learner enters the answer to the calculation question and clicks the "Submit" button. The input in this step is the learner's answer data, and the output is the answer data sent from the terminal to the server.

[1691] Step 6:

[1692] The server compares the received answer data with pre-set correct answer data and evaluates whether the answer is correct or incorrect. This is done by using an SQL database to compare the correct answer data with the learner's answer. The evaluation results are generated as feedback data. The input to this step is the learner's answer data, and the output is the server's evaluation result (correct or incorrect).

[1693] Step 7:

[1694] The server returns the evaluation result data to the terminal, which then presents the result to the user. Specifically, the terminal displays a feedback message on the screen and provides the option to proceed to the next question or try again. The input of this step is the evaluation result data from the server, and the output is the feedback message presented to the user.

[1695] Step 8:

[1696] The device captures the learner's facial expressions and voice data using a camera and microphone, and sends that data to the server. The server then analyzes this data using an emotion engine to recognize the learner's emotions (e.g., joy, sadness, anger, excitement, etc.). Specifically, it uses Microsoft Azure's Emotion API. The input for this step is the learner's facial expressions and voice data, and the output is the emotion analysis results by the server.

[1697] Step 9:

[1698] The server readjusts the difficulty and content of the game-style questions based on the emotion analysis results. For example, if the learner is tired, it makes specific adjustments such as lowering the difficulty of the questions or providing encouraging messages. The input of this step is the emotion analysis results, and the output is the adjusted question data or encouraging messages.

[1699] Step 10:

[1700] The server records the learner's answer data and progress data and updates the learning history. This data is used to generate and suggest questions for the next learning session. It also provides rewards such as points and badges for correctly answered questions and notifies the learner of this via their device. Specifically, it includes a system function that automatically manages appropriate rewards according to the learner's learning progress. The input to this step is the learner's progress data and answer data, and the output is an updated learning history and reward data.

[1701] (Application example 2)

[1702] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1703] Conventional learning systems lacked the means to make homework fun for learners, making it feel like a game, and limited the means to increase motivation. Furthermore, there were no systems that recognized learners' emotions and provided appropriate feedback. These shortcomings led to problems such as a lack of appropriate responses when learners lost interest in learning, or felt confused or fatigued, resulting in a decline in learning efficiency.

[1704] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a learner to input homework content; means for analyzing the input homework content and classifying it into specific learning categories; means for generating game-style questions according to the classified learning categories and transmitting them to the terminal; means for displaying the game-style questions received by the terminal so that the learner can answer them interactively; means for evaluating the learner's answers and presenting the results; means for recording the learner's progress and managing the learning history; means for recognizing and analyzing the learner's emotions; and means for adjusting the difficulty and content of the questions based on the recognized emotions. This enables the learner to study enjoyably and effectively, and by providing appropriate feedback according to emotions, learning efficiency is improved and motivation is maintained.

[1705] "Student" refers to a user who uses the system to do homework or study.

[1706] "Means for inputting homework content" refers to the interface that allows learners to input the specific content of their homework via a terminal.

[1707] "Means for analyzing the input homework content and classifying it into specific learning categories" refers to a program or device that uses natural language processing technology to analyze the homework content input by the learner and classify it into learning categories such as mathematics or English.

[1708] "Means for generating game-style questions and sending them to the terminal" refers to a system for creating game-style questions such as quizzes and puzzles based on the analyzed homework content and sending that data to the learner's terminal.

[1709] "A means for displaying questions on a terminal and allowing learners to answer them interactively" refers to an interface that displays the sent game-style questions on the screen of the learner's terminal and allows the learner to answer the questions.

[1710] "Means for evaluating answers and presenting results" refers to a system for determining whether the answers entered by the learner are correct or incorrect and providing feedback on the results to the learner.

[1711] "Means for recording learners' progress and managing their learning history" refers to a system that records the content and results of questions answered by learners, and stores and manages them in a database as learning history.

[1712] "Means for recognizing and analyzing learners' emotions" refers to technology that uses cameras and microphones to capture the learners' facial expressions and voices, and analyzes them to determine the learners' emotional state.

[1713] "Means for adjusting the difficulty and content of questions based on the learner's emotions" refers to an algorithm for dynamically changing the difficulty and content of questions based on the learner's emotional state.

[1714] The present invention is a learning system that allows learners to complete their homework more enjoyably and effectively. Specific embodiments of this system will be described below.

[1715] System configuration and program processing

[1716] Hardware and Software Configuration

[1717] This learning system includes the following hardware and software:

[1718] 1. Device: A device used by a user (learner), such as a smartphone, tablet, smart glasses, or head-mounted display.

[1719] 2. Server: A cloud-based server that receives data, analyzes it, generates questions, and performs emotion recognition.

[1720] 3. Software:

[1721] Natural language processing (NLP) technology: Uses Google Cloud Natural Language API.

[1722] Data management: Use Google Firebase to manage learning data.

[1723] Emotion Recognition Model: Emotion recognition is performed using TensorFlow.

[1724] System Operation

[1725] The system operates as follows.

[1726] 1. Learners input and submit their homework

[1727] The user (student) uses the device's application to input the content of their homework via text or voice. For example, using a smartphone, they can input "Today's math homework is 2 x 3 and 4 + 5" and tap the "Convert homework into a game" button. This data is then sent to the server via the Internet.

[1728] 2. Analysis and classification of homework content

[1729] The server uses natural language processing technology to analyze the received homework content, and based on the analysis results, classifies the homework into a specific learning category (e.g., math problems, Kanji, English, etc.).

[1730] 3. Creating and submitting game-style questions

[1731] The server generates game-style questions according to the appropriate learning category based on the analyzed homework content. For example, in the case of calculation questions, quiz-style questions including arithmetic operations are generated. The server then transmits the game-style question data to the device.

[1732] 4. Terminal display and interactive answers

[1733] The terminal displays the game-style questions received from the server to the learner, who then interactively inputs the answers, which are then sent back to the server via the terminal.

[1734] 5. Evaluation and feedback of answers

[1735] The server compares the received answers with pre-defined correct answer data, evaluates the learner's answers, and sends the results to the device to provide feedback to the learner. If the answer is correct, the learner can proceed to the next question, and if the answer is incorrect, the learner is presented with the option to try again.

[1736] 6. Learner emotion recognition and regulation

[1737] The device's camera and microphone are used to capture the learner's facial expressions and voice data, which are then sent to the server. The server's emotion engine (using TensorFlow) analyzes this data and recognizes the learner's emotional state. Based on the recognized emotion, the difficulty and content of the questions are adjusted. For example, if the learner is confused, the questions are made easier or a supportive message is displayed.

[1738] 7. Progress Recording and Rewarding

[1739] The server records the learner's answer data and progress data, and manages their learning history. This history data is used in the next learning session. In addition, learners are given rewards such as points and badges for correctly answering questions.

[1740] Examples of concrete examples and prompts

[1741] Specific examples

[1742] Situations where learners enter homework details on their smartphones:

[1743] The learner types, "Today's math homework is 2 x 3 and 4 + 5."

[1744] The server classifies the homework content into calculation problem categories and generates quiz-style questions.

[1745] The learner answers the quiz, the server evaluates it, and the learner moves on to the next question.

[1746] If the emotion engine detects that the learner is tired, it will easily adjust the questions or display encouraging messages such as "Keep it up! You're almost there!"

[1747] Prompt Sentence Examples

[1748] "If learners are stumped, lower the difficulty and provide a supportive message, such as, 'Don't worry, the next question is easier, so relax!'"

[1749] In this way, this system not only allows learners to study in an enjoyable and effective manner, but also improves learning efficiency and motivation by providing appropriate feedback based on their emotions.

[1750] The flow of the s...

Claims

1. A means for learners to input homework content; A means for analyzing the input homework content and classifying it into specific learning categories; A means for generating game-style questions according to the classified learning categories and transmitting the questions to the terminal; A device displays the received game-style questions and allows learners to answer them interactively; a means of assessing learners' answers and presenting the results; a means of recording learner progress and managing learning history; A system including:

2. 2. The system according to claim 1, further comprising means for suggesting next study content and questions based on the results of the learner's answers.

3. The system of claim 1 , further comprising means for providing rewards, such as points or badges, to a learner for correctly answering questions.

Images