Method and system for managing students' learning

Abstract

A method according to one embodiment may include the steps of: arranging multiple multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a low level to a high level of the educational process stage; outputting multiple multiple-choice questions for each of the plurality of evaluation areas; obtaining multiple-choice answer data of a student for the plurality of multiple-choice questions; identifying a supplementary educational process stage where the student's learning is weak for each of the plurality of evaluation areas by identifying a section of questions where the error density among the plurality of multiple-choice questions exceeds a threshold value based on comparing the multiple-choice answer data with the correct answer for the plurality of multiple-choice questions for each of the plurality of evaluation areas; obtaining learning material data corresponding to the supplementary educational process stage from at least one of a database or memory; scheduling the learning material data according to a learning schedule; and outputting the scheduled learning material data.

Description

Technology Field

[0001] The present disclosure relates to a method and system for managing the learning of a student. Specifically, the present disclosure relates to a method and system for identifying a supplementary education course stage where a student's learning is weak based on the student's answer data for a plurality of problems, acquiring learning material data corresponding to the identified supplementary education course stage, scheduling the learning material data according to a learning schedule, and outputting the scheduled learning material data. Background Technology

[0002] In the field of student learning, various learning management technologies have been proposed to diagnose students' learning levels and provide learning content based on the diagnosis results. For example, methods that provide students with multiple multiple-choice questions and classify learning levels based on the number of correct answers or scores, or methods that adjust learning progress by calculating scores for each subject area, are widely used. In addition, some technologies are known for systems that distinguish multiple assessment areas such as listening, vocabulary, grammar, and reading, perform assessments for each area, and provide learning content or assignments based on the assessment results.

[0003] However, most of these conventional technologies often remain limited to simple classification methods based on final scores or average scores, which limits their ability to accurately identify which stage of the educational process a student is actually struggling with. In particular, even among students with the same score, there can be significant differences in the distribution of incorrect answers during the problem-solving process, response times, answer correction behaviors, or the quality of subjective responses; yet, such detailed learning behavior data is not sufficiently reflected. As a result, it is difficult to derive supplementary learning processes that directly correspond to a student's learning weaknesses, and there are limitations in systematically improving learning effectiveness. Prior art literature

[65535] Republic of Korea Published Patent Application No. 10-2013-0015411 (Publication Date: February 14, 2013) Republic of Korea Published Patent Application No. 10-2011-0079252 (Publication Date: July 7, 2011) Republic of Korea Published Patent Application No. 10-2009-0114192 (Publication Date: November 3, 2009) Republic of Korea Registered Patent No. 10-1805163 (Registration Date: November 29, 2017) Republic of Korea Registered Patent No. 10-2336900 (Registration Date: December 3, 2021) Republic of Korea Registered Patent No. 10-2269837 (Registration Date: June 22, 2021) means of solving the problem

[0004] A method according to one embodiment may include the step of arranging multiple multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a lower level to a higher level in the educational process.

[0005] A method according to one embodiment may include the step of outputting a plurality of multiple-choice questions for each of a plurality of evaluation areas.

[0006] A method according to one embodiment may include the step of obtaining multiple-choice answer data of a student for a plurality of multiple-choice questions.

[0007] A method according to one embodiment may include the step of identifying a supplementary education course stage in which a student's learning is weak for each of the multiple evaluation areas by identifying a section of questions in which the error density among the multiple multiple questions exceeds a threshold value, based on comparing multiple multiple-choice answer data with correct answers for multiple multiple-choice questions for each of the multiple evaluation areas.

[0008] A method according to one embodiment may include the step of obtaining learning material data corresponding to a supplementary education course step from at least one of a database or memory.

[0009] A method according to one embodiment may include the step of scheduling learning material data according to a learning schedule.

[0010] A method according to one embodiment may include the step of outputting scheduled learning data. Brief explanation of the drawing

[0011] FIG. 1 is a flowchart illustrating a method according to one embodiment. FIG. 2 is a flowchart illustrating a method according to one embodiment. FIG. 3 is a flowchart illustrating a method according to one embodiment. FIG. 4 is a flowchart illustrating a method according to one embodiment. FIG. 5 is a flowchart illustrating a method according to one embodiment. FIG. 6 is a flowchart illustrating a method according to one embodiment. FIG. 7 is a flowchart illustrating a method according to one embodiment. FIG. 8 is a flowchart illustrating a method according to one embodiment. FIG. 9 is a flowchart illustrating a method according to one embodiment. FIG. 10 is a flowchart illustrating a method according to one embodiment. FIG. 11 is a block diagram illustrating the components of a system according to one embodiment. Specific details for implementing the invention

[0012] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0013] The terms used in this invention have been selected based on currently widely used general terms, taking into account their functions within the invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0014] When a part of a specification is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0015] Additionally, the description 'at least one of A, B, and C' means that it may be any one of 'A', 'B', 'C', 'A and B', 'A and C', 'B and C', and 'A, B, and C'.

[0016] It should be understood that the combinations of blocks and flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. One or more computer programs may be stored entirely in a single memory or may be partitioned and stored in multiple different memories.

[0017] All functions or operations described in this document may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing and may include circuitry such as an AP (Application Processor), CP (Communication Processor), GPU (Graphical Processing Unit), NPU (Neural Processing Unit), MPU (Microprocessor Unit), SoC (System on Chip), IC (Integrated Chip), etc.

[0018] A processor may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claims, may include at least one processor and various processing circuits. In at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0019] The following describes embodiments with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0020] In the embodiments of this specification, the term "user" may mean a person controlling a system, function, or operation.

[0021] FIG. 1 is a flowchart illustrating a method according to one embodiment.

[0022] In step 110, the method according to one embodiment may include the step of arranging a plurality of multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a lower level to a higher level in the educational process.

[0023] The listening section may refer to a section designed to evaluate the ability to understand the meaning, details, or the speaker's intent of a given audio signal after listening to words, sentences, and / or paragraphs output as audio signals. Multiple-choice questions in the listening section may consist of, for example, questions designed to evaluate the ability to recognize pronunciation at the word level, the ability to understand meaning at the sentence level, or the ability to grasp the main point of long audio passages such as conversations or lectures.

[0024] The vocabulary domain may refer to an area designed to assess the ability to understand the meaning, usage, and contextual appropriateness of words. Multiple-choice questions in the vocabulary domain may consist of, for example, questions designed to assess the ability to select the basic meaning of a word, distinguish synonyms or antonyms, and select vocabulary appropriate to the context.

[0025] The grammar domain may refer to an area designed to assess the ability to understand and apply sentence structure, word order, and grammatical rules. Multiple-choice questions in the grammar domain may consist of questions designed to assess the ability to understand grammatical structures, such as tense agreement, subject-verb agreement, part of speech selection, relative clauses, or conditional sentences.

[0026] The reading comprehension section may refer to an area designed to evaluate the ability to read a passage composed of text or sentences, and to understand and interpret the meaning of said passage. Multiple-choice questions in the reading comprehension section may consist of, for example, questions designed to evaluate the ability to grasp explicit information, the ability to connect meanings between paragraphs, the ability to understand the overall argument of the passage, or the ability to infer the author's intention and implied meaning.

[0027] Curriculum stages can refer to learning process levels classified according to the student's learning level and history. Curriculum stages can be divided into elementary, secondary, higher, and advanced education courses.

[0028] Curriculum stages can be subdivided by grade level within each curriculum. For example, the elementary education curriculum can be subdivided into the 1st, 2nd, 3rd, 4th, 5th, and 6th grade curriculums. The secondary education curriculum can be subdivided into the 1st, 2nd, and 3rd grade middle school curriculums. The higher education curriculum can be subdivided into the 1st, 2nd, and 3rd grade high school curriculums. The advanced education curriculum can be subdivided into TOEIC preparation courses, TOEFL preparation courses, IELTS preparation courses, and so on.

[0029] The elementary education curriculum may refer to learning stages capable of basic character recognition, basic vocabulary comprehension, and language comprehension at the level of short sentences. The elementary education curriculum may include, for example, the basic word recognition stage of the first-grade elementary curriculum, the simple sentence comprehension stage of the second-grade elementary curriculum, and the short paragraph reading comprehension stage of the third or fourth-grade elementary curriculum. However, it is not limited thereto.

[0030] The secondary education curriculum may refer to learning stages capable of paragraph-level understanding and basic reasoning. For example, the secondary education curriculum may include the basic tense and sentence expansion stages of the first year of middle school, the comparative and contrastive expression understanding stages of the second year of middle school, and the complex sentence structure understanding and simple reasoning stages of the third year of middle school. However, it is not limited to these.

[0031] The higher education curriculum may refer to learning stages capable of reading long passages and inferring implicit meanings. The higher education curriculum may include, for example, the argumentative reading stage of the first year of high school curriculum, and the advanced vocabulary usage and author perspective analysis stages of the second or third year of high school curriculum. However, it is not limited thereto.

[0032] Advanced education courses may refer to high-level learning stages focused on professional skills or exam preparation that go beyond the standard school curriculum. For example, advanced education courses may include courses preparing for standardized English tests such as TOEFL, TOEIC, or IELTS, university-level academic English reading courses, or professional English courses. However, they are not limited to these.

[0033] A method according to one embodiment may include the step of arranging multiple multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a lower level to a higher level in the educational process.

[0034] For example, a method according to one embodiment may arrange multiple-choice questions 1 through 24 in the listening section to correspond to the elementary education curriculum. A method according to one embodiment may organize the multiple-choice questions in the listening section corresponding to the elementary education curriculum according to learning stages from the first grade of elementary school to the sixth grade of elementary school. A method according to one embodiment may sequentially arrange four questions for each stage of the listening section curriculum corresponding to each grade of elementary school. A method according to one embodiment may place the questions in the listening section corresponding to the lower grades of elementary school curriculum at an earlier number than the questions in the listening section corresponding to the upper grades of elementary school curriculum. For example, the method according to one embodiment may place listening section questions for the first grade elementary school curriculum among a total of 24 questions corresponding to elementary school curriculums at numbers 1 to 4, listening section questions for the second grade elementary school curriculum at numbers 5 to 8, listening section questions for the third grade elementary school curriculum at numbers 9 to 12, listening section questions for the fourth grade elementary school curriculum at numbers 13 to 16, listening section questions for the fifth grade elementary school curriculum at numbers 17 to 20, and listening section questions for the sixth grade elementary school curriculum at numbers 21 to 24. However, it is not limited thereto.

[0035] For example, a method according to one embodiment may arrange multiple-choice questions 25 through 37 in the listening section to correspond to the secondary education curriculum. A method according to one embodiment may organize the listening section questions corresponding to the secondary education curriculum according to the curriculum stages from the first year of middle school to the third year of middle school. A method according to one embodiment may sequentially arrange four questions for each curriculum stage of the listening section corresponding to each grade of middle school. A method according to one embodiment may arrange the listening section questions of the secondary education curriculum in the same or similar manner as the arrangement method of the listening section questions of the elementary education curriculum.

[0036] For example, a method according to one embodiment can arrange multiple-choice questions 38 through 50 of the listening section to correspond to the higher education curriculum. A method according to one embodiment can sequentially arrange four questions for each stage of the listening curriculum corresponding to each grade of high school. A method according to one embodiment can arrange the listening section questions of the higher education curriculum in the same or similar way as the arrangement method of the listening section questions of the elementary and secondary education curriculum.

[0037] For example, a method according to one embodiment can arrange multiple-choice questions 51 through 55 in the listening section to correspond to the advanced education course. A method according to one embodiment can arrange the listening section questions of the advanced education course in the same or similar way as the arrangement method of the listening section questions of the elementary, middle, and high school education courses.

[0038] A method according to one embodiment can arrange multiple multiple-choice questions for vocabulary, grammar, and reading comprehension areas in the same or similar way as the multiple-choice questions for listening areas for each evaluation area.

[0039] The examples described above are merely examples, and the total number of multiple-choice questions and / or the number of questions arranged for each stage of the educational process are not limited thereto.

[0040] A method according to one embodiment can variably adjust the number of problems placed in each stage of the educational course according to the student's learning objectives and / or the difficulty of the evaluation.

[0041] A method according to one embodiment can provide different numbers of problems to different students for the same educational course stage.

[0042] A method according to one embodiment can precisely analyze the learning performance results of students at each stage of the educational process by arranging problems such that the required educational process stage gradually increases as the problem number increases for each evaluation area.

[0043] In step 120, the method according to one embodiment may include the step of outputting a plurality of multiple-choice questions for each of a plurality of evaluation areas.

[0044] A method according to one embodiment can output multiple-choice questions corresponding to each of the listening area, vocabulary area, grammar area, and reading area. A method according to one embodiment can output multiple-choice questions corresponding to each evaluation area by placing them on different output screens or different output zones.

[0045] A method according to one embodiment can form areas partitioned by evaluation area on a single display screen and output multiple-choice questions corresponding to each partitioned area.

[0046] A method according to one embodiment may output a screen configured to select an answer after listening to the corresponding voice signal, along with a playback button for the voice signal, when outputting a multiple-choice question corresponding to a listening area.

[0047] A method according to one embodiment may output a screen containing multiple options along with words, sentences and / or example sentences when outputting a multiple-choice question corresponding to a vocabulary area.

[0048] A method according to one embodiment may output a screen containing blanks, underlined marks, and / or highlighted grammar elements within a sentence when outputting a multiple-choice question corresponding to a grammar area.

[0049] A method according to one embodiment can output a screen that includes a passage, a question based on the passage, and answer choices when outputting a multiple-choice question corresponding to a reading comprehension area.

[0050] A method according to one embodiment can output multiple-choice questions corresponding to each evaluation area in parallel at the same time. A method according to one embodiment can output multiple-choice questions corresponding to each evaluation area sequentially at different times.

[0051] A method according to one embodiment controls the output order by evaluation area so that problems corresponding to the elementary education curriculum are output first, and then problems corresponding to the secondary education curriculum, higher education curriculum, and advanced education curriculum are output sequentially.

[0052] A method according to one embodiment may delay or start early the output of multiple-choice questions in the next evaluation area depending on the student's learning concentration or response progress.

[0053] A method according to one embodiment may initiate the output of multiple-choice questions for another evaluation area after the output of multiple-choice questions for a specific evaluation area is completed.

[0054] A method according to one embodiment can control the number of multiple-choice questions output for each evaluation area according to a pre-set number and / or a dynamically determined number.

[0055] A method according to one embodiment may omit or emphasize the output of multiple-choice questions in a specific evaluation area depending on the age of the student, the study time, or the purpose of the evaluation.

[0056] A method according to one embodiment may output an answer sheet corresponding to each multiple-choice question when outputting a plurality of multiple-choice questions. The answer sheet may refer to an input interface configured to allow a student to select one of a plurality of options. For example, the answer sheet may include a check area where one of five selection items can be checked to correspond to a five-option multiple-choice question. For example, the answer sheet may be configured to include a circular button, a rectangular check box, or a touchable selection area next to each selection item. However, it is not limited thereto.

[0057] A method according to one embodiment can output multiple-choice questions and answer sheets together on the same screen.

[0058] A method according to one embodiment can output an area displaying multiple-choice questions and an area displaying an answer sheet separately on the screen.

[0059] A method according to one embodiment can display one of the selection items included in the answer sheet as a checked state based on the input of a student. The student's input may include touch input, mouse click input, keyboard input, and / or input through an external input device.

[0060] A method according to one embodiment may display a check mark corresponding to a selected item on an answer sheet when a student performs an input selecting a specific selection item. The check mark may be displayed in at least one of changing the color of the selected item, highlighting the border, displaying a check mark, or shading the selected item, but is not limited thereto.

[0061] A method according to one embodiment can deselect an existing check state when a student re-enters an already selected item.

[0062] A method according to one embodiment can deselect an existing selected item and change a newly selected item to a checked state when a student inputs a different selected item.

[0063] A method according to one embodiment can update the check status of the answer sheet in real time based on the input of a student.

[0064] A method according to one embodiment can visually maintain the check status marked on the answer sheet so that the student can recognize the currently selected answer.

[0065] In step 130, the method according to one embodiment may include the step of obtaining multiple-choice answer data of a student for a plurality of multiple-choice questions.

[0066] Multiple-choice answer data may refer to data containing information regarding the option selected by a student for each multiple-choice question. Multiple-choice answer data may include, for example, question identification information to identify each multiple-choice question and / or option identification information to identify the option selected by a student. However, it is not limited thereto.

[0067] A method according to one embodiment can detect when a student inputs one of the selection items included in an answer sheet. Based on the detected input, a method according to one embodiment can convert the selection item selected by the student into multiple-choice answer data.

[0068] A method according to one embodiment can generate multiple-choice answer data by matching selected options for each multiple-choice question on a question-by-question basis. A method according to one embodiment can store selected options chosen by a student for a specific multiple-choice question in association with a question number or a question identifier.

[0069] A method according to one embodiment can determine the finally selected option as the answer to a single multiple-choice question when a student sequentially inputs multiple selection options for a single multiple-choice question. A method according to one embodiment can, when a student's selected option changes, invalidate the option selected before the change and reflect the option selected after the change in the multiple-choice answer data. The multiple-choice answer data may include information about inputs that occurred before the answer was determined for each multiple-choice question.

[0070] A method according to one embodiment can record the input time of a selected item chosen by a student as time information.

[0071] A method according to one embodiment can confirm the multiple-choice answer data generated up to that point as final answer data when a student completes inputting answers to a plurality of multiple-choice questions or when a preset evaluation termination condition is satisfied.

[0072] A method according to one embodiment can store objective answer data confirmed as final answer data in memory or a database.

[0073] A method according to one embodiment can utilize stored multiple-choice answer data as input data for calculating incorrect answer density, analyzing response time, or identifying educational process stages.

[0074] In step 140, the method according to one embodiment may include the step of identifying a supplementary education course stage in which a student's learning is weak for each of the multiple evaluation areas by identifying a section of questions in which the error density among the multiple multiple questions exceeds a threshold value, based on comparing multiple multiple answer data with correct answers for multiple multiple questions for each of the multiple evaluation areas.

[0075] A method according to one embodiment may include a step of comparing multiple-choice answer data with correct answers for multiple-choice questions for each of a plurality of evaluation areas.

[0076] A method according to one embodiment can compare a selection item chosen by a student with a predefined correct answer selection item on a multiple-choice question basis corresponding to each evaluation area. A method according to one embodiment can generate a result of determining whether each multiple-choice question is correct as question-unit judgment data.

[0077] Error density may refer to the ratio of the number of questions entered as incorrect by students to the total number of questions included in a question group corresponding to the same curriculum stage. Error density may refer to the ratio of the number of questions judged as incorrect to the total number of questions included in a question group corresponding to the same curriculum stage. Error density can be calculated by dividing the number of questions judged as incorrect by the total number of questions included in the question group corresponding to the same curriculum stage. Multiple multiple-choice questions corresponding to the same curriculum stage may be defined as a single question group.

[0078] A question interval may refer to a set of multiple multiple-choice questions arranged to correspond to the same curriculum stage or the same grade level curriculum. A question interval may refer to a group of consecutively arranged multiple-choice questions among those arranged sequentially from a lower curriculum stage to a higher curriculum stage.

[0079] A question set may consist of questions corresponding to any one of the stages of the elementary, secondary, higher, or advanced education curriculums. Within a single curriculum stage, a question set may consist of multiple-choice questions with the same grade level curriculum. For example, a question set may refer to multiple multiple-choice questions corresponding to the first grade curriculum of the elementary school curriculum, multiple multiple-choice questions corresponding to the first grade curriculum of the middle school curriculum of the secondary education curriculum, or multiple multiple-choice questions corresponding to the second grade curriculum of the high school curriculum of the higher education curriculum, but is not limited thereto.

[0080] A problem interval can be defined based on the continuity of problem numbers. A problem interval can be defined as problems included within a range where problem numbers increase continuously.

[0081] Problem sections can be divided based on points where the curriculum stage or grade level curriculum changes. Problem sections can consist of problems arranged continuously without changes in the curriculum stage or grade level curriculum.

[0082] Question intervals can be defined according to the number of questions set in advance for each curriculum stage. Question intervals can be defined based on a structure in which the same number of multiple-choice questions are arranged for each grade level curriculum.

[0083] For example, if four questions are placed per grade in the elementary education curriculum, four multiple-choice questions corresponding to each grade of elementary school can be defined as one question range. For example, if three questions are placed per grade in the secondary education curriculum, three multiple-choice questions corresponding to each grade of middle school can be defined as one question range.

[0084] Question segments can be defined independently for each assessment area. Question segments can be set separately for each of the listening, vocabulary, grammar, or reading areas. A question segment can be defined as a question segment within only one assessment area.

[0085] The problem interval can be used as the minimum unit of analysis for calculating the error density. The problem interval can be used as the unit of analysis to determine whether the error density exceeds a threshold.

[0086] A method according to one embodiment can identify a number of consecutive problems among arranged problems as problem sections by grade level. A method according to one embodiment can calculate the number of problems judged as incorrect for multiple-choice questions included in each problem section. A method according to one embodiment can calculate the density of incorrect answers per problem section based on the total number of problems included in each problem section and the number of problems judged as incorrect.

[0087] A method according to one embodiment can compare the error density calculated for each problem section with a corresponding threshold value. The threshold value can be changed by the user and can be set to a different value depending on the evaluation area, the stage of the educational process, or the grade level of the educational process.

[0088] A method according to one embodiment can identify a problem section where the density of incorrect answers exceeds a threshold as a supplementary education course stage where the student's learning is weak.

[0089] The supplementary education stage may refer to an educational stage or grade level where normal learning progress is deemed difficult. The supplementary education stage may refer to an educational stage or grade level where additional learning, repetitive learning, or difficulty adjustment is required.

[0090] For example, if a question group corresponding to the first-grade level of elementary school education in the elementary education curriculum contains 10 multiple-choice questions, the method according to one embodiment may set the total number of questions included in the question group to 10. If a student inputs 6 questions as incorrect answers among the 10 multiple-choice questions included in the question group corresponding to the first-grade level of elementary school education, the method according to one embodiment may count the number of questions judged as incorrect as 6. The method according to one embodiment may calculate an incorrect answer density of 0.6 by dividing the number of questions judged as incorrect by 6 by the total number of questions of 10. If the threshold value set for the question group corresponding to the first-grade level of elementary school education is 0.5, the method according to one embodiment may determine that the calculated incorrect answer density of 0.6 exceeds the threshold value of 0.5. The method according to one embodiment may identify the question group corresponding to the first-grade level of elementary school education as a question section where the incorrect answer density exceeds the threshold value. A method according to one embodiment can identify the first-grade level of elementary school education during the elementary education curriculum as a supplementary education course stage where the student's learning is insufficient.

[0091] For example, if a question group corresponding to the first-year middle school level in a secondary education curriculum contains 8 multiple-choice questions and the number of questions judged as incorrect is 4, the method according to one embodiment can calculate the incorrect answer density as 0.5. If the threshold value set for the question group corresponding to the first-year middle school level is 0.4, the method according to one embodiment can determine that the calculated incorrect answer density of 0.5 exceeds the threshold value. The method according to one embodiment can identify an educational curriculum containing a question group corresponding to the first-year middle school level as a supplementary education curriculum stage.

[0092] For example, if the number of questions included in a question group corresponding to the second-year high school level in a higher education course is 12 and the number of questions judged as incorrect is 3, the method according to one embodiment can calculate the incorrect answer density as 0.25. If the threshold value set for the question group corresponding to the second-year high school level is 0.4, the calculated incorrect answer density of 0.25 is below the threshold value, so the method according to one embodiment may not identify the education course containing the question group corresponding to the second-year high school level as a supplementary education course stage.

[0093] For example, in the case where the density of incorrect answers in the problem section corresponding to the elementary education curriculum in the grammar area does not exceed a threshold, but the density of incorrect answers exceeds a threshold only in the problem section corresponding to the middle school first-year curriculum among the problems corresponding to the secondary education curriculum, the method according to one embodiment can identify the grammar education curriculum of the middle school first-year curriculum as a supplementary education curriculum stage.

[0094] For example, in the listening section, if the density of incorrect answers is calculated to be below a threshold in the problem sections of the middle school first and second grade curriculum, and the density of incorrect answers exceeds the threshold only in the problem sections of the middle school third grade curriculum, the method according to one embodiment can identify the listening education course of the middle school third grade curriculum as a supplementary education course stage.

[0095] For example, in the vocabulary domain, if the density of incorrect answers is calculated to be below a threshold in the problem section of the elementary school curriculum from grade 1 to grade 5, and the density of incorrect answers exceeds the threshold only in the problem section of the elementary school curriculum grade 6, the method according to one embodiment can identify the vocabulary education course of the elementary school grade 6 curriculum as a supplementary education course stage.

[0096] For example, in the reading comprehension area, if the density of incorrect answers is calculated to be below a threshold in the problem sections of the elementary and secondary education curriculums, and the density of incorrect answers exceeds the threshold in the problem sections of the high school first-year curriculum among the higher education curriculums, the method according to one embodiment can identify the reading comprehension curriculum of the high school first-year curriculum as a supplementary education curriculum stage.

[0097] For example, if the density of incorrect answers is calculated to be below a threshold in question sections corresponding to the elementary and secondary education curricula in the listening domain, and the density of incorrect answers exceeds the threshold only in question sections corresponding to the higher education curriculum, the method according to one embodiment can identify the listening education curriculum at the higher education level as a supplementary education curriculum stage. However, it is not limited thereto. However, it is not limited thereto.

[0098] In step 150, the method according to one embodiment may include the step of obtaining learning material data corresponding to the supplementary education course step.

[0099] Learning material data may refer to data regarding educational content provided to supplement the learning content required at the supplementary education course stage. For example, if the supplementary education course stage is the reading area of ​​the first year of middle school education, the learning material data may refer to data regarding educational content related to the reading area of ​​the first year of middle school education. Learning material data may include, but is not limited to, at least one of explanatory text materials, example questions, practice problems, commentary materials, video lecture files, audio lecture files or animation-based explanatory materials, grammar rule explanatory materials, vocabulary learning materials, reading strategy guide materials or audio materials for listening training.

[0100] Learning material data may include data regarding educational materials for an educational level corresponding to the supplementary education course stage. Educational materials for an educational level corresponding to the supplementary education course stage may refer to educational materials containing educational materials for any one of the elementary education course, secondary education course, higher education course, or advanced education course. For example, educational materials for an educational level corresponding to the supplementary education course stage may include, but are not limited to, at least one of regular textbooks, reference books, or workbooks published by a publisher; educational materials produced in-house by an academy or educational institution; printed or digital materials; electronic files of paper materials; materials in the form of e-books; or educational material data uploaded to an online learning system.

[0101] A method according to one embodiment can map educational course steps or grade-level educational courses identified as supplementary educational course steps to textbook data.

[0102] A method according to one embodiment can obtain learning material data corresponding to the supplementary education course step from at least one of a database or memory.

[0103] A method according to one embodiment may obtain training data from a database in which training data is stored. The database may be included in a server located outside the system in which the method according to one embodiment is performed. The server may be connected to the system in which the method according to one embodiment is performed via wireless communication. The wireless communication may include at least one of a mobile communication network, the wireless internet, or short-range wireless communication.

[0104] A method according to one embodiment can obtain learning material data corresponding to a supplementary education course stage from a database included in a server via a download method. A method according to one embodiment can obtain learning material data corresponding to a supplementary education course stage from a database included in a server via a streaming method. A method according to one embodiment can select a download method or a streaming method depending on the capacity of the learning material data, the learning environment, or the network status.

[0105] A method according to one embodiment can search for textbook data corresponding to a supplementary education course stage among textbook data stored in at least one of a database or memory. A method according to one embodiment can filter textbook data based on education course stage information, grade education course information, or textbook difficulty information included in the textbook data. A method according to one embodiment can select textbook data suitable for a supplementary education course stage from among a plurality of textbook data as learning material data.

[0106] In the case where there are multiple supplementary education course stages, the method according to one embodiment can acquire teaching material data corresponding to each supplementary education course stage.

[0107] For example, if the middle school first-year curriculum is identified as a supplementary curriculum stage in the grammar domain, the method according to one embodiment can obtain learning material data corresponding to a grammar textbook published by a publisher, a grammar textbook produced by an academy itself, or a part of the said textbook.

[0108] For example, if the second-year high school curriculum is identified as a supplementary curriculum stage in the reading area, the method according to one embodiment can obtain reading materials, argumentative writing-centered textbooks, or advanced reading units included in said textbooks of the second-year high school curriculum as learning material data.

[0109] For example, if the 6th grade elementary school curriculum is identified as a supplementary curriculum stage in the vocabulary domain, the method according to one embodiment can obtain vocabulary textbooks, vocabulary book-type textbooks, or context-based vocabulary practice textbooks of the 6th grade elementary school curriculum as learning material data.

[0110] In step 160, the method according to one embodiment may include the step of scheduling learning material data according to a learning schedule.

[0111] A learning schedule may refer to schedule information that defines the chronological order, timing, or frequency of learning in which a student studies learning material data. A learning schedule may include schedule information defined based on dates, days of the week, time zones, or learning sessions. For example, a learning schedule may include a daily, weekly, or monthly schedule. The learning schedule may be set to reflect the student's available study time, preferred time zones, or deadlines for achieving learning goals.

[0112] A method according to one embodiment can map learning material data corresponding to a supplementary education course stage to a learning schedule. A method according to one embodiment can assign one or more learning material data to each learning point included in the learning schedule.

[0113] A method according to one embodiment can determine the order of provision of learning material data based on the difficulty level or learning volume of a supplementary education course stage. In this case, multiple learning material data corresponding to the same supplementary education course stage may be distributed and arranged at multiple learning points.

[0114] A method according to one embodiment can arrange learning material data corresponding to different supplementary education course stages on different dates or at different times. Accordingly, the learning material data to be provided at each learning point included in the learning schedule can be arranged in chronological order.

[0115] A method according to one embodiment may designate learning material data arranged to correspond to a specific learning point included in a learning schedule as an output target when a specific learning point is reached.

[0116] A method according to one embodiment can dynamically adjust the learning schedule according to changes in the student's learning progress or learning achievement.

[0117] A method according to one embodiment can configure a learning schedule such that, when there are multiple supplementary education course stages, learning material data corresponding to each supplementary education course stage is cross-arranged.

[0118] For example, if the middle school first-year curriculum is identified as a supplementary curriculum stage in the grammar area, the method according to one embodiment may place grammar explanation materials of the middle school first-year curriculum on the first learning day and grammar practice problem materials of the middle school first-year curriculum on the second learning day.

[0119] For example, if the second-year high school curriculum is identified as a supplementary curriculum stage in the reading area, the method according to one embodiment may place reading strategy materials of the second-year high school curriculum in the weekday study schedule and long reading practice materials of the second-year high school curriculum in the weekend study schedule.

[0120] For example, if both the vocabulary area and the grammar area are identified as supplementary education course stages, the method according to one embodiment may arrange vocabulary learning materials and grammar learning materials at a time interval on the same learning day.

[0121] For example, if the study schedule is set to one week, the method according to one embodiment can schedule study material data in units of chapters of a textbook published by a publisher. For example, the method according to one embodiment can schedule study material data corresponding to Chapter 1 of Textbook A in the study schedule for Monday, Chapter 2 of Textbook A in the study schedule for Tuesday, Chapter 3 of Textbook A in the study schedule for Wednesday, Chapter 4 of Textbook A in the study schedule for Thursday, Chapter 5 of Textbook A in the study schedule for Friday, Chapter 6 of Textbook A in the study schedule for Saturday, and Chapter 7 of Textbook A in the study schedule for Sunday.

[0122] For example, if the learning schedule is set to three weeks, the method according to one embodiment may schedule learning material data on a daily and / or weekly basis based on each date included in the three-week learning period. For example, the method according to one embodiment may sequentially schedule learning material data for explaining basic concepts in the first week of the three-week learning schedule, learning material data for practice problems regarding basic concepts in the second week, learning material data for application problems regarding basic concepts in the third week, and learning material data for reviewing basic concepts in the fourth week.

[0123] For example, if the first-year middle school curriculum is identified as a supplementary curriculum stage in the grammar domain, the method according to one embodiment may schedule learning material data corresponding to basic explanation materials on tenses on the first day, learning materials for example questions related to tenses on the second day, learning materials for tense application practice problems on the third day, and review materials for tense learning content on the fourth day, by referring to grammar-related content included in a textbook covering the first-year middle school curriculum. However, it is not limited thereto.

[0124] The learning material data may include a scheduling table that organizes the contents of the learning material data for each learning time point arranged according to the learning schedule in a table format.

[0125] A method according to one embodiment can generate a scheduling table that organizes the contents of learning material data arranged according to a learning schedule. The scheduling table may refer to schedule information in a table format in which learning material data that a student must study corresponds to and is displayed according to a learning schedule. The scheduling table may include identification information of learning material data or learning content information to be provided at each learning time point, based on learning time points classified by date or day of the week.

[0126] For example, a scheduling table may include learning content categorized by day of the week, such as learning material data corresponding to the learning schedule for Monday, learning material data corresponding to the learning schedule for Tuesday, and learning material data corresponding to the learning schedule for Wednesday. In this case, learning material data that the student needs to study at the corresponding learning time can be mapped and displayed for each day.

[0127] A method according to one embodiment may store a scheduling table generated during the process of scheduling learning material data in memory or output it through a display. A method according to one embodiment may print the scheduling table using a printer and provide it to a student in paper form.

[0128] In step 170, the method according to one embodiment may include the step of outputting scheduled training data.

[0129] A method according to one embodiment can output learning material data scheduled to correspond to a specific learning day when a specific learning day included in a learning schedule is reached.

[0130] A method according to one embodiment may apply different output methods depending on the type of learning material data. For example, a method according to one embodiment may output text-based learning material data in the form of a document screen or an explanation screen. A method according to one embodiment may output learning material data including problem solving in the form of a problem solving screen. A method according to one embodiment may output learning material data corresponding to a video lecture in the form of a video playback screen. A method according to one embodiment may output learning material data corresponding to an audio lecture through an audio playback screen or an audio output device.

[0131] A method according to one embodiment can output learning material data corresponding to a publisher's textbook or an academy's own textbook in the form of a textbook screen. A method according to one embodiment can output learning content on the textbook screen in chapter units, unit units, or page units of the textbook.

[0132] A method according to one embodiment can sequentially output a plurality of learning material data scheduled for the same learning day according to a preset order. A method according to one embodiment can control the order so that explanatory learning material data is output first, followed by practice problems or review learning material data.

[0133] A method according to one embodiment can control the playback, pause, or termination of learning material data being output according to the input of a student. A method according to one embodiment can maintain the output of learning material data corresponding to the next learning day in a standby state after the output of the learning material data is completed.

[0134] For example, if the middle school first-year curriculum is identified as a supplementary curriculum stage in the grammar domain, the method according to one embodiment may output grammar rule explanation materials of the middle school first-year curriculum according to the learning schedule, and then output grammar practice problem materials.

[0135] For example, if the second-year high school curriculum is identified as a supplementary curriculum stage in the reading area, the method according to one embodiment can output reading strategy guidance materials for the second-year high school curriculum according to the learning schedule, and then output long reading practice materials.

[0136] A method according to one embodiment can output a scheduling table that organizes the contents of learning material data arranged by learning schedule. The scheduling table may refer to schedule information in a table format in which learning material data that a student must study is displayed correspondingly by day of the week or date. For example, the scheduling table may include learning content separated by day of the week, such as learning material data corresponding to the learning schedule for Monday, learning material data corresponding to the learning schedule for Tuesday, and learning material data corresponding to the learning schedule for Wednesday.

[0137] A method according to one embodiment can print a scheduling table using a printer and provide it to students in paper form.

[0138] FIG. 2 is a flowchart illustrating a method according to one embodiment.

[0139] Steps 130, 140, and 150 of Fig. 2 correspond to steps 130, 140, and 150 of Fig. 1, respectively.

[0140] In step 210, the method according to one embodiment may include the step of obtaining an error density for each of the plurality of evaluation areas at each stage of the educational process.

[0141] A method according to one embodiment can calculate the error density based only on multiple-choice questions included in the listening section. A method according to one embodiment can calculate the error density based only on multiple-choice questions included in the vocabulary section. A method according to one embodiment can calculate the error density based only on multiple-choice questions included in the grammar section. A method according to one embodiment can calculate the error density based only on multiple-choice questions included in the reading section.

[0142] A method according to one embodiment can obtain the density of incorrect answers for question groups corresponding to the elementary education curriculum for each evaluation area. A method according to one embodiment can obtain the density of incorrect answers for question groups corresponding to the secondary education curriculum for each evaluation area. A method according to one embodiment can obtain the density of incorrect answers for question groups corresponding to the higher education curriculum for each evaluation area. A method according to one embodiment can obtain the density of incorrect answers for question groups corresponding to the advanced education curriculum for each evaluation area.

[0143] A method according to one embodiment can aggregate the total number of questions included in each question group. A method according to one embodiment can aggregate the number of questions determined to be incorrect included in each question group. A method according to one embodiment can obtain an incorrect answer density by calculating the ratio of the number of questions determined to be incorrect to the total number of questions included in the question group. A method according to one embodiment can obtain an incorrect answer density corresponding to a plurality of educational process stages for each evaluation area.

[0144] For example, in the grammar domain, the density of incorrect answers can be obtained for question groups corresponding to the elementary education curriculum, and the density of incorrect answers can be obtained for question groups corresponding to the secondary education curriculum.

[0145] For example, different error densities can be obtained for the question group corresponding to the first year of middle school curriculum in the reading comprehension area and the question group corresponding to the second year of middle school curriculum in the reading comprehension area.

[0146] For example, the density of incorrect answers calculated for question groups corresponding to the first-year high school curriculum in the listening section and the density of incorrect answers calculated for question groups corresponding to the advanced curriculum in the listening section can be obtained separately. However, it is not limited to this.

[0147] A method according to one embodiment can store the error density obtained for each evaluation area in a data structure sorted according to the order of the educational process steps.

[0148] In step 220, the method according to one embodiment may include the step of identifying the educational course step in which the error density first exceeds a threshold value according to the ascending order of the educational course steps as a supplementary educational course step.

[0149] A method according to one embodiment may include a step of comparing the density of incorrect answers according to the ascending order of the educational process stages. The ascending order of the educational process stages may refer to a sequence in which the educational process progresses from a stage of low learning difficulty to a stage of high learning difficulty. The ascending order of the educational process stages may include the order of the elementary education course, the secondary education course, the higher education course, and the advanced education course. The ascending order of the educational process stages may include a sequence in which the grade level education course progresses from a lower grade level to a higher grade level within the same educational process.

[0150] A method according to one embodiment can sequentially obtain the density of incorrect answers according to the ascending order of the educational process stages for each evaluation area.

[0151] A method according to one embodiment can compare the obtained incorrect answer density with a threshold value. If the selected incorrect answer density does not exceed the threshold value, the method according to one embodiment can obtain the incorrect answer density corresponding to the next highest training course level. A method according to one embodiment can repeat this comparison process in ascending order of training course levels.

[0152] A method according to one embodiment can obtain the error density that first exceeds a threshold among the error densities compared in ascending order of the educational process stages.

[0153] A method according to one embodiment can identify an educational process step corresponding to the error density that first exceeds a threshold as a supplementary educational process step.

[0154] The supplementary education stage may refer to the initial stage of the curriculum where it is determined that the student has not sufficiently mastered the learning content. The supplementary education stage may also refer to a stage of the curriculum where supplementary learning is required prior to learning in subsequent higher-level stages.

[0155] For example, if the density of incorrect answers corresponding to the elementary education curriculum in the grammar domain is below a threshold, the density of incorrect answers corresponding to the secondary education curriculum is below a threshold, and the density of incorrect answers corresponding to the higher education curriculum exceeds a threshold, the method according to one embodiment can identify the higher education curriculum as a supplementary education curriculum stage.

[0156] For example, in the reading comprehension area, if the density of incorrect answers corresponding to the 4th grade elementary school curriculum is below a threshold, the density of incorrect answers corresponding to the 5th grade elementary school curriculum is below a threshold, and the density of incorrect answers corresponding to the 6th grade elementary school curriculum exceeds a threshold, the method according to one embodiment can identify the 6th grade elementary school curriculum as a supplementary curriculum stage.

[0157] For example, in the listening domain, even if the density of incorrect answers corresponding to both the first year middle school curriculum and the second year middle school curriculum exceeds a threshold, the method according to one embodiment can first identify the first year middle school curriculum as a supplementary curriculum stage according to the ascending order of the curriculum stages.

[0158] For example, in the vocabulary domain, if the density of incorrect answers corresponding to both the secondary education curriculum and the higher education curriculum exceeds a threshold, the method according to one embodiment can identify the secondary education curriculum as a supplementary education curriculum stage.

[0159] A method according to one embodiment can sequentially compare the density of incorrect answers according to the ascending order of the educational course stages and identify the educational course stage where the density of incorrect answers first exceeds a threshold, thereby enabling the supplementation of a student's learning weaknesses starting from a more basic stage. FIG. 3 is a flowchart illustrating a method according to one embodiment.

[0160] Steps 130, 140, and 150 of FIG. 3 correspond to steps 130, 140, and 150 of FIG. 1 and FIG. 2, respectively.

[0161] In step 310, the method according to one embodiment may include the step of measuring a first response time from the point in time when a plurality of multiple-choice questions corresponding to a first educational course step are output on a display to the point in time when multiple-choice answer data for a plurality of multiple-choice questions corresponding to a first educational course step is obtained.

[0162] A method according to one embodiment may set a reference point at which a plurality of multiple-choice questions corresponding to a first educational course stage are displayed on a display. The display may refer to an output device for visually providing multiple-choice and / or subjective questions and answer sheets to a student.

[0163] A method according to one embodiment can identify the time when a plurality of multiple-choice questions corresponding to a first educational course stage are first displayed on a display as the first output time.

[0164] A method according to one embodiment may identify the time of acquiring multiple-choice answer data for a plurality of multiple-choice questions corresponding to a first educational course stage as the first time of acquiring the answer. The time of acquiring the multiple-choice answer data may mean the time when the student's answer input is completed for all multiple-choice questions corresponding to the first educational course stage.

[0165] A method according to one embodiment may calculate the time difference from a first output point to a first answer acquisition point as a first response time. The first response time may refer to the total time taken by a student to solve a plurality of multiple-choice questions corresponding to a first educational course stage. A method according to one embodiment may measure the first response time as a time value in milliseconds, seconds, and / or minutes.

[0166] For example, if the time from the point when a multiple-choice question corresponding to the first educational course stage is displayed on the display until the point when a student inputs all answers and multiple-choice answer data is obtained is measured as 6 minutes, the method according to one embodiment can calculate the first response time as 6 minutes. However, it is not limited thereto.

[0167] A method according to one embodiment can record a first response time in the form of data along with time information.

[0168] The first curriculum stage may refer to a curriculum stage that has a higher difficulty level than the second curriculum stage based on the ascending order of curriculum stages. The first curriculum stage may refer to a curriculum stage that has a higher learning difficulty or requires a higher level of prior learning than the second curriculum stage. For example, the first curriculum stage may correspond to the higher education curriculum, and the second curriculum stage may correspond to the secondary education curriculum. For example, the first curriculum stage may correspond to the second-year middle school curriculum, and the second curriculum stage may correspond to the first-year middle school curriculum. However, it is not limited thereto.

[0169] In step 320, the method according to one embodiment may include the step of measuring a second response time from the point in time when a plurality of multiple-choice questions corresponding to the second educational course stage are displayed on a display until the point in time when multiple-choice answer data for a plurality of multiple-choice questions corresponding to the second educational course stage is obtained.

[0170] A method according to one embodiment may set the point in time when a plurality of multiple-choice questions corresponding to the second educational course stage are displayed on the display as the reference point.

[0171] A method according to one embodiment can identify the point in time when a plurality of multiple-choice questions corresponding to the second educational course stage are first displayed on the display as the second output point.

[0172] A method according to one embodiment may identify the time of acquiring multiple-choice answer data for a plurality of multiple-choice questions corresponding to the second educational course stage as the time of acquiring the second answer. The time of acquiring the multiple-choice answer data may mean the time when the student's answer input is completed for all multiple-choice questions corresponding to the second educational course stage.

[0173] A method according to one embodiment may calculate the time difference from the second output point to the second answer acquisition point as the second response time. The second response time may refer to the total time taken by a student to solve a plurality of multiple-choice questions corresponding to the second educational course stage. A method according to one embodiment may measure the second response time as a time value in milliseconds, seconds, and / or minutes.

[0174] For example, if the time from the point when a multiple-choice question corresponding to the second educational course stage is displayed on the display until the point when a student inputs all answers and multiple-choice answer data is obtained is measured as 15 minutes, the method according to one embodiment can calculate the second response time as 15 minutes. However, it is not limited thereto.

[0175] A method according to one embodiment can record the second response time in the form of data along with time information.

[0176] In step 330, the method according to one embodiment may include the step of identifying the first training course step as a supplementary training course step when the first response time is smaller than the second response time.

[0177] A method according to one embodiment can determine whether a first response time is shorter than a second response time. Determining that the first response time is shorter than the second response time may mean that the student solved a problem at a relatively high-difficulty educational course stage within a shorter time. If the first response time is shorter than the second response time, the method according to one embodiment may determine that a reversal phenomenon has occurred in the learning process. A reversal phenomenon in the learning process may refer to a phenomenon where the time taken to solve problems at a higher educational course stage is measured to be shorter than the time taken to solve problems at a lower educational course stage.

[0178] A method according to one embodiment may determine that the learning content required in the first educational process stage has not been sufficiently established when a reversal phenomenon occurs during the learning process. A method according to one embodiment may identify the first educational process stage as a supplementary educational process stage when a reversal phenomenon occurs during the learning process.

[0179] For example, in a case where the first educational curriculum stage is the second-year middle school curriculum and the second educational curriculum stage is the first-year middle school curriculum, and the first response time taken to solve the problems of the second-year middle school curriculum is measured as 5 minutes and the second response time taken to solve the problems of the first-year middle school curriculum is measured as 7 minutes, the method according to one embodiment can identify the second-year middle school curriculum as a supplementary educational curriculum stage.

[0180] For example, in a case where the first curriculum stage is the high school first-year curriculum and the second curriculum stage is the middle school third-year curriculum, and the time taken to solve problems in the high school first-year curriculum is measured to be shorter than the time taken to solve problems in the middle school third-year curriculum, the method according to one embodiment can identify the middle school second-year curriculum as a supplementary curriculum stage. However, it is not limited thereto.

[0181] FIG. 4 is a flowchart illustrating a method according to one embodiment.

[0182] Steps 130, 140, and 150 of FIG. 4 correspond to steps 130, 140, and 150 of FIG. 1 to 3, respectively.

[0183] In step 410, the method according to one embodiment may include the step of identifying a training course stage containing multiple-choice questions for which the number of answer corrections exceeds the reference number as a supplementary training course stage when the number of answer corrections exceeds the reference number by comparing the number of answer corrections with the reference number based on multiple-choice answer data.

[0184] A method according to one embodiment may include the step of obtaining the number of answer modifications included in the multiple-choice answer data. The multiple-choice answer data may include input information regarding answers selected by a student for a plurality of multiple-choice questions. The multiple-choice answer data may include data regarding the number of answer modifications made by a student before finally confirming the answers for the plurality of multiple-choice questions. The number of answer modifications may refer to the cumulative number of times a student changed an option selected for a single multiple-choice question.

[0185] A method according to one embodiment can obtain the number of answer corrections for each multiple-choice question. A method according to one embodiment can aggregate the number of answer corrections for multiple-choice questions corresponding to the same educational course stage. The threshold number may refer to a threshold value for determining whether the number of answer corrections indicates a decrease in learning stability, and may be changed by the user. The threshold number may be set to different values ​​depending on the educational course stage, evaluation area, or question type.

[0186] A method according to one embodiment can compare the number of answer corrections calculated for each multiple-choice question with a reference number.

[0187] A method according to one embodiment can identify a multiple-choice question in which the number of answer corrections exceeds a standard number as an unstable question. An unstable question may refer to a multiple-choice question in which a student showed a high level of hesitation or confusion during the problem-solving process.

[0188] A method according to one embodiment can identify a training course stage containing multiple-choice questions in which the number of answer corrections exceeds a reference number. A method according to one embodiment can identify a training course stage containing multiple-choice questions in which the number of answer corrections exceeds a reference number as a supplementary training course stage.

[0189] For example, if the number of answer corrections for 3 out of 8 multiple-choice questions corresponding to the first year of middle school curriculum in the secondary education curriculum exceeds the standard number of corrections of 3, the method according to one embodiment may determine that the curriculum of the first year of middle school curriculum has been identified as a supplementary education curriculum stage.

[0190] For example, if the number of answer corrections for 4 out of 10 multiple-choice questions in the grammar area corresponding to the 6th grade elementary school curriculum exceeds the standard number of corrections of 2, the method according to one embodiment may determine that the grammar education curriculum of the 6th grade elementary school curriculum has been identified as a stage with low learning stability.

[0191] For example, even if a multiple-choice question is not judged to be incorrect, if the number of answer corrections exceeds a standard number, the method according to one embodiment may identify the educational course stage containing the said multiple-choice question as a supplementary educational course stage. However, it is not limited thereto.

[0192] FIG. 5 is a flowchart illustrating a method according to one embodiment.

[0193] Steps 130, 140, and 150 of FIG. 5 correspond to steps 130, 140, and 150 of FIG. 1 to 4, respectively.

[0194] In step 510, the method according to one embodiment may include the step of arranging a plurality of subjective questions to evaluate a student's writing ability and translation ability according to the educational course stages from a lower stage to a higher stage.

[0195] A method according to one embodiment may include the step of arranging a plurality of subjective questions to evaluate a student's writing ability and translation ability at each stage of the educational course.

[0196] Translation ability may refer to a student's ability to convert sentences or paragraphs provided in English into Korean while preserving their meaning. Translation ability may include whether the meaning of the original text is preserved, whether the context is maintained, or the naturalness of the Korean expression.

[0197] Writing ability may refer to a student's ability to generate English sentences based on sentences or paragraphs provided in Korean. Writing ability may include the accuracy of sentence structure, the appropriateness of word order, the accuracy of tense usage, or the appropriateness of vocabulary selection.

[0198] Subjective questions refer to problems that require students to write a text-based answer directly, rather than selecting from multiple-choice options. Subjective questions may refer to problems that require writing a single sentence, multiple sentences, or paragraph-level description.

[0199] A method according to one embodiment may be configured to distinguish between subjective questions for evaluating translation ability and subjective questions for evaluating writing ability.

[0200] A method according to one embodiment can arrange subjective questions from a low level to a high level in the educational process.

[0201] A method according to one embodiment may configure a subjective question for evaluating translation ability corresponding to the elementary education curriculum as a question involving the translation of a basic English sentence into Korean. For example, the subjective question for evaluating translation ability corresponding to the elementary education curriculum may be configured in a form requiring the translation of "I don't want to gain weight." or "Tom is not at home now." into Korean. However, it is not limited thereto.

[0202] A method according to one embodiment may configure a subjective question for evaluating translation ability corresponding to a secondary education curriculum as a question involving translating English text containing relative clauses or impersonal constructions into Korean. For example, a subjective question for evaluating translation ability corresponding to a secondary education curriculum may be configured in a form requiring the translation of "Here are some stories about people who suggest different ways." or "It was easy for each cow in the village to find enough grass to eat." into Korean. However, it is not limited thereto.

[0203] A method according to one embodiment may configure a subjective question for evaluating translation ability corresponding to a higher education course as a problem involving translating English text containing abstract concepts or complex structures into Korean. For example, a subjective question for evaluating translation ability corresponding to a higher education course may be configured in a form requiring the translation of "Fake news can be very harmful in that it can make people less informed or even misled." or "Confronted with this seemingly loop, he eventually rebels against living through the same day the same way twice." into Korean. However, it is not limited thereto.

[0204] A method according to one embodiment may configure subjective questions for evaluating writing ability corresponding to the elementary education curriculum as questions that require writing basic Korean sentences in English. For example, subjective questions for evaluating writing ability corresponding to the elementary education curriculum may be configured in a form that requires writing sentences such as "I go to school" or "Tom is not at home right now" in English. However, it is not limited thereto.

[0205] A method according to one embodiment may configure a subjective question for evaluating writing ability corresponding to a secondary education curriculum as a question requiring the interpretation of sentence structure in Korean sentences to be written in English. For example, a subjective question for evaluating writing ability corresponding to a secondary education curriculum may be configured in a form requiring the writing of English sentences such as "It is dangerous for her to swim here" or "I saw that she was crying." However, it is not limited thereto.

[0206] A method according to one embodiment may configure a subjective question for evaluating writing ability corresponding to a higher education course as a question requiring the writing of a Korean sentence containing a conditional or participial phrase in English. For example, a subjective question for evaluating writing ability corresponding to a higher education course may be configured in a form requiring the writing of English sentences such as "If you were here, I would be happy" or "While walking along the road, I kept sending text messages." However, it is not limited thereto.

[0207] A method according to one embodiment can systematically evaluate a student's language conversion ability and sentence generation ability by arranging subjective questions with progressively increasing difficulty levels according to the stages of the educational course while clearly distinguishing between translation ability and writing ability.

[0208] In step 520, the method according to one embodiment may include the step of outputting a plurality of subjective questions for each of the plurality of evaluation areas.

[0209] A method according to one embodiment can output subjective questions corresponding to a writing area and subjective questions corresponding to a translation area separately from each other.

[0210] A method according to one embodiment can output a subjective question corresponding to a writing area on a single writing question screen. A method according to one embodiment can output a subjective question corresponding to a translation area on a single translation question screen.

[0211] A method according to one embodiment can sequentially output subjective questions for each evaluation area. A method according to one embodiment can first output subjective questions corresponding to a lower level of the educational process and then output subjective questions corresponding to a higher level of the educational process.

[0212] A method according to one embodiment may output a text input area for entering an answer along with a subjective question. In the case of a subjective question corresponding to a writing area, a method according to one embodiment may present a Korean sentence and output an input area for entering an English answer. In the case of a subjective question corresponding to a translation area, a method according to one embodiment may present an English sentence and output an input area for entering a Korean answer.

[0213] For example, a method according to one embodiment can first output a subjective question corresponding to a writing area to induce a student to write a Korean sentence in English.

[0214] For example, a method according to one embodiment can output a subjective question corresponding to a translation area after the input of an answer to a subjective question corresponding to a writing area is completed.

[0215] For example, a method according to one embodiment may sequentially output one or more subjective questions in a translation area after outputting all of one or more subjective questions in a writing area.

[0216] A method according to one embodiment can switch the screen to a translation evaluation section including subjective questions in a translation section after the writing evaluation section including subjective questions in a writing section has ended.

[0217] For example, a method according to one embodiment may sequentially output one or more subjective questions in a writing section after outputting all of one or more subjective questions in a translation section. A method according to one embodiment may switch the screen to a translation evaluation section containing subjective questions in a writing section after the writing evaluation section containing subjective questions in a translation section has ended. However, it is not limited thereto.

[0218] In step 530, the method according to one embodiment may include the step of obtaining subjective answer data of a student for a plurality of subjective questions.

[0219] Subjective answer data may refer to response information in the form of text directly written by students in response to subjective questions. Subjective answer data may include sentences written in English by students for subjective questions in the writing section and / or sentences written in Korean by students for subjective questions in the translation section.

[0220] A method according to one embodiment can receive input from a student through a text input area provided along with a subjective question. A method according to one embodiment can acquire subjective answer data by at least one of keyboard input, touch input, or handwriting recognition input. However, it is not limited thereto.

[0221] A method according to one embodiment can obtain individual answer data by separating the answers entered by the student for each subjective question.

[0222] A method according to one embodiment may identify the time when the input of an answer corresponding to each subjective question is completed as the time of answer acquisition. The time of answer acquisition may include the time when a student selects an input completion button and / or the time when input is performed to move to the next question.

[0223] A method according to one embodiment can store subjective answer data obtained for each subjective question together with question identification information.

[0224] A method according to one embodiment can map evaluation area information or educational course stage information together with subjective answer data.

[0225] A method according to one embodiment can obtain subjective answer data corresponding to a writing area and subjective answer data corresponding to a translation area by distinguishing them from each other.

[0226] For example, in the case where a student inputs an English sentence in response to a Korean sentence presented as a subjective question in the writing section, the method according to one embodiment can obtain the input English sentence as subjective answer data.

[0227] For example, in the case where a student inputs a Korean sentence in response to an English sentence presented as a subjective question in the translation area, the method according to one embodiment can obtain the input Korean sentence as subjective answer data.

[0228] For example, a method according to one embodiment can obtain multiple answers sequentially entered by a student for a plurality of subjective questions as independent subjective answer data. However, it is not limited thereto.

[0229] In step 540, the method according to one embodiment may include the step of identifying a supplementary education course step based on multiple-choice answer data and subjective answer data.

[0230] Multiple-choice answer data may include data regarding whether the multiple-choice question was answered incorrectly, the density of incorrect answers, response time, and / or the number of answer corrections. Subjective answer data may include text-based answers written by students for subjective questions in the writing section and / or translation section.

[0231] A method according to one embodiment can calculate a learning stability index for each stage of an educational course based on multiple-choice answer data. The learning stability index can be calculated based on at least one of an incorrect answer density, a result of comparing response times, or a result of comparing the number of answer corrections.

[0232] A method according to one embodiment can calculate a language expression ability index for each stage of the educational course based on subjective answer data. The language expression ability index may include partial scores calculated using an artificial intelligence model.

[0233] A method according to one embodiment can combine learning stability indicators and language expression ability indicators according to the stages of the educational process.

[0234] A method according to one embodiment may compare indicators combined for each stage of the educational process with a pre-set judgment criterion. The judgment criterion may refer to a standard value for determining whether a stage of the educational process requires supplementation, and may be changed by the user.

[0235] A method according to one embodiment can identify a course of education stage in which the combined indicators do not meet the judgment criteria as a supplementary course of education stage. A supplementary course of education stage may refer to a course of education stage in which the results of solving multiple-choice questions and the results of solving subjective questions do not meet the standard level, either all or partly.

[0236] A method according to one embodiment can compare a supplementary education course step identified based on objective answer data with a supplementary education course step identified based on subjective answer data.

[0237] A method according to one embodiment can finally determine the educational course stage where the need for supplementation is determined in either the objective answer data or the subjective answer data as the supplementary educational course stage.

[0238] A method according to one embodiment can identify an educational course stage where the need for supplementation is determined in both objective answer data and subjective answer data as a priority supplementary educational course stage.

[0239] For example, if the density of incorrect answers for multiple-choice questions in the grammar area of ​​the first year of middle school curriculum exceeds a threshold value, and the partial score for subjective questions in the writing area corresponding to the same curriculum stage is below a reference score, the method according to one embodiment can identify the first year of middle school curriculum as a supplementary curriculum stage.

[0240] For example, if the result of solving multiple-choice questions in the reading comprehension area of ​​the first year of high school curriculum meets the standard, but the partial score of subjective questions in the translation area corresponding to the same curriculum stage is below the standard score, the method according to one embodiment can identify the reading comprehension curriculum stage of the first year of high school curriculum as a supplementary curriculum stage.

[0241] A method according to one embodiment may set a plurality of evaluation rules applied to an artificial intelligence model to calculate partial scores for subjective answer data. The evaluation rules may include, but are not limited to, at least one of the number of omitted words, whether essential vocabulary is included, whether sentence components exist, or the state of word order arrangement.

[0242] For example, a method according to one embodiment may deduct a predetermined score when one required word is missing from subjective answer data, and gradually increase the deduction range when multiple required words are missing, but is not limited thereto.

[0243] A method according to one embodiment can adjust partial scores by assigning a weight when a specific key vocabulary is included and decreasing the weight when the said key vocabulary is not included. The evaluation rule described above can be stored as a parameter of an artificial intelligence model or as score calculation logic and applied to the subjective scoring process.

[0244] FIG. 6 is a flowchart illustrating a method according to one embodiment.

[0245] Step 150 of FIG. 6 corresponds to Step 150 of FIG. 1 to 5, and Step 530 and Step 540 of FIG. 6 correspond to Step 530 and Step 540 of FIG. 5, respectively.

[0246] In step 610, the method according to one embodiment may include the step of calculating partial scores for subjective answer data based on comparing the correct answers to a plurality of subjective questions with the sentence structure, word order, tense, and vocabulary included in the subjective answer data using an artificial intelligence model.

[0247] An artificial intelligence model may refer to a model trained to analyze input sentences, extract linguistic features, and produce comparison results. An artificial intelligence model may include machine learning models or deep learning models based on natural language processing techniques. For example, an artificial intelligence model may include sentence decomposition models, syntactic analysis models, part-of-speech tagging models, or sentence embedding models. An artificial intelligence model may include, for example, pre-trained language models or language models further trained based on training data.

[0248] Artificial intelligence models can operate through technology that performs computations using neural networks. Neural networks can be trained by receiving training data as input. A trained neural network receives input data at its input layer, and at its output layer, it can analyze the input data and perform computations to produce output data, which is the desired result. Computations through neural networks can be performed through hidden layers.

[0249] A neural network can receive input data, perform the aforementioned operations for analysis and / or classification, and output result data. In order for the neural network to accurately output result data corresponding to the input data, it is necessary to train the neural network. Here, 'training' refers to inputting various data into the neural network and training the network so that it can independently discover or learn methods for analyzing the input data, methods for classifying the input data, and / or methods for extracting features necessary for generating result data from the input data. Specifically, through the training process, the neural network can train the training data (e.g., multiple different images) to optimize and set the weight values ​​within the neural network. Then, by independently learning the input data through the neural network with optimized weight values, it outputs the desired result.

[0250] Specifically, a neural network can be classified as a deep neural network when the number of hidden layers—internal layers that perform computations—is multiple, that is, when the depth of the neural network performing computations increases. Examples of neural networks include, but are not limited to, CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), and Deep Q-Networks. Furthermore, neural networks can be subdivided. For example, a CNN neural network can be subdivided into DCNN (Deep Convolutional Neural Network) or Capsnet neural networks.

[0251] Sentence structure can refer to the structural relationships between the subject, verb, object, or complement that constitute a sentence. Sentence structure can include simple sentence structures, complex sentence structures, or structures containing subordinate clauses.

[0252] Word order can refer to the sequence in which words or phrases are arranged within a sentence. Word order may include the subject-verb-object order or the position of adverbial phrases.

[0253] Tense can refer to a grammatical element that indicates the temporal basis at which an action or state occurred in a sentence. Tense can include the present tense, past tense, future tense, or perfect tense.

[0254] Vocabulary can refer to words or expressions used in a sentence. Vocabulary may include basic vocabulary, derived vocabulary, or idiomatic expressions.

[0255] A method according to one embodiment may set a correct answer sentence as a reference sentence for each subjective question. A method according to one embodiment may use an artificial intelligence model to extract at least one of sentence structure information, word order information, tense information, or vocabulary information from the correct answer sentence.

[0256] A method according to one embodiment can extract at least one of sentence structure information, word order information, tense information, or vocabulary information from a sentence included in subjective answer data using an artificial intelligence model.

[0257] A method according to one embodiment can compare sentence structure information extracted from a correct answer sentence with sentence structure information extracted from subjective answer data. A method according to one embodiment can calculate a sentence structure score corresponding to the result of the sentence structure comparison.

[0258] A method according to one embodiment can compare word order information extracted from a correct answer sentence with word order information extracted from subjective answer data. A method according to one embodiment can calculate a word order score corresponding to the result of the word order comparison.

[0259] A method according to one embodiment can compare tense information extracted from a correct answer sentence with tense information extracted from subjective answer data. A method according to one embodiment can calculate a tense score corresponding to the result of the tense comparison.

[0260] A method according to one embodiment can compare vocabulary information extracted from a correct answer sentence with vocabulary information extracted from subjective answer data. A method according to one embodiment can calculate a vocabulary score corresponding to the result of the vocabulary comparison.

[0261] A partial score may refer to a score that includes at least one of a sentence structure score, a word order score, a tense score, and a vocabulary score. A partial score may be expressed as a grammar element-based partial score below.

[0262] A method according to one embodiment can calculate partial scores by weighted summing sentence structure scores, word order scores, tense scores, and vocabulary scores.

[0263] A method according to one embodiment may set predefined deduction criteria for each of sentence structure, word order, tense, and vocabulary in order to award partial scores for subjective answer data. Each evaluation item may be set to have the same maximum score criteria, and the range of deduction may vary depending on the type of error and the degree of impact the error has on the transmission of meaning.

[0264] For example, regarding vocabulary evaluation, 0.5 points may be deducted if a single word that does not constitute the core meaning is used incorrectly. On the other hand, 1 point or more may be deducted if a core verb or key vocabulary that determines the meaning is used incorrectly. However, this is not limited to these cases.

[0265] In relation to tense evaluation, 0.5 points may be deducted for simple tense inconsistencies, and 1 point or more may be deducted for the incorrect use of tenses that determine the semantic structure of a sentence, such as the subjunctive mood. However, this is not limited to these cases.

[0266] Regarding the evaluation of sentence structure, partial deductions may be applied if the basic framework of the sentence is maintained but the required grammatical structure is not accurately implemented, and significant deductions may be applied if the sentence structure itself differs significantly from the requirements. However, this is not limited to these cases.

[0267] Regarding the evaluation of word order, minor differences in word order that do not affect the transmission of meaning may be excluded from deductions or subject to only minor deductions, while word order errors that cause confusion in interpretation may be subject to deductions in stages. However, this is not limited to these cases.

[0268] A method according to one embodiment can calculate partial scores for subjective questions by applying deduction criteria to each evaluation item to calculate the score for each item and summing the calculated scores for each item.

[0269] For example, if the total score for one subjective question is 10 points, the partial score may consist of 2.5 points for sentence structure, 2.5 points for word order, 2.5 points for tense, and 2.5 points for vocabulary.

[0270] For example, in a case where "I go to school" is presented as subjective question 2 in the writing section and the correct answer sentence is set as "I go to school," and a student enters "I am going to school" as the subjective answer data, the method according to one embodiment can determine that both sentences have a subject-verb-prepositional phrase structure and calculate a sentence structure score out of 2.5 points. Additionally, the method according to one embodiment can determine that the word order is identical and calculate a word order score out of 2.5 points. Meanwhile, since the correct answer sentence uses the simple present tense and the subjective answer data uses the present continuous tense, a difference in tense exists but does not significantly affect the transmission of meaning; therefore, the method according to one embodiment can calculate the tense score as 1.8 points out of 2.5 points. The method according to one embodiment determines that all vocabulary used is appropriate and calculates the vocabulary score out of 2.5 points. As a result, the method according to one embodiment can calculate the partial score for subjective question 2 in the writing section as 9.3 points.

[0271] For example, in a case where "If you were here, I would be happy" is presented as subjective question number 4 in the writing section and the correct answer sentence is set as "If you were here, I would be happy," and a student enters "If you are here, I will be happy" as subjective answer data, the method according to one embodiment may determine that the indicative present structure was used in a situation where the past conditional structure is required. In this case, the sentence structure score may be calculated as 1.5 out of a maximum of 2.5 points. Additionally, the method according to one embodiment may determine that the inconsistency between tense and conditional is a grammatical error that significantly affects the meaning, and thus calculate the tense score as 0.8 out of a maximum of 2.5 points. On the other hand, the method according to one embodiment may determine that the word order and vocabulary usage are appropriate, and thus calculate the word order score as a maximum of 2.5 points and the vocabulary score as a maximum of 2.5 points, respectively. As a result, the method according to one embodiment can calculate the partial score for subjective question 4 in the writing section as 7.3 points by summing the sentence structure score of 1.5 points, the word order score of 2.5 points, the tense score of 0.8 points, and the vocabulary score of 2.5 points. However, it is not limited thereto.

[0272] A method according to one embodiment may apply different weights to each score depending on the educational process stage or evaluation area.

[0273] In step 620, the method according to one embodiment may include the step of identifying a supplementary education course step based on multiple-choice answer data and partial scores.

[0274] A method according to one embodiment can obtain a multiple-choice evaluation index based on multiple-choice answer data at each stage of the educational course. The multiple-choice evaluation index may be expressed, for example, as an incorrect answer density value and / or a ratio of incorrect answers.

[0275] A method according to one embodiment can aggregate partial scores of subjective questions corresponding to the same educational course stage. A method according to one embodiment can calculate the average partial score of subjective questions corresponding to the same educational course stage.

[0276] A method according to one embodiment can generate a comprehensive judgment value by combining objective evaluation indicators and average partial scores at each stage of the educational course. The comprehensive judgment value may represent an indicator indicating that supplementation is needed when at least one of the objective evaluation indicators and partial scores related to subjective evaluations does not meet the standard conditions.

[0277] A method according to one embodiment can identify a training course stage as a supplementary training course stage in which the overall judgment value does not satisfy a pre-set judgment criterion. The judgment criterion may be set, for example, when the error density exceeds 0.4 or the average partial score is less than 7 points, but is not limited thereto.

[0278] For example, if the error density in the multiple-choice questions corresponding to the first year of middle school curriculum is calculated to be 0.45, and the average partial score for the subjective questions in the writing and translation areas corresponding to the first year of middle school curriculum is calculated to be 6.8 points, the method according to one embodiment may determine that the error density of 0.45 exceeds the standard error density of 0.4 and the average partial score of 6.8 points falls below the standard score of 7 points. In this case, the method according to one embodiment may determine that the curriculum of the first year of middle school curriculum does not meet the standards in both the multiple-choice and subjective evaluation results. Accordingly, the method according to one embodiment may identify the first year of middle school curriculum stage as a supplementary curriculum stage.

[0279] For example, if the error density of multiple-choice questions corresponding to the first year of high school curriculum in the higher education curriculum is calculated as 0.25 and the average partial score of subjective questions corresponding to the first year of high school curriculum is calculated as 6.5 points, the method according to one embodiment may determine that the error density of 0.25 is less than or equal to the standard error density of 0.4, but the average partial score of 6.5 points is lower than the standard score of 7 points. The method according to one embodiment may determine that the result of solving multiple-choice questions is stable but the language expression ability is insufficient. In this case, the method according to one embodiment may identify the first year of high school curriculum stage as a supplementary education stage.

[0280] For example, if the error density in multiple-choice questions corresponding to the 6th grade elementary school curriculum is calculated to be 0.3 and the average partial score of subjective questions corresponding to the 6th grade elementary school curriculum is calculated to be 8.2, since the error density of 0.3 is less than the standard error density of 0.4 and the average partial score of 8.2 exceeds the standard score of 7, the method according to one embodiment may not identify the 6th grade elementary school curriculum stage as a supplementary curriculum stage. However, it is not limited thereto.

[0281] FIG. 7 is a flowchart illustrating a method according to one embodiment.

[0282] Step 530 of Fig. 7 corresponds to Step 530 of Figs. 5 and 6, and Steps 610 and 620 of Fig. 7 correspond to Steps 610 and 620 of Fig. 6, respectively.

[0283] In step 710, the method according to one embodiment may include the step of calculating semantic similarity regarding the contextual meaning between the correct answer for a plurality of subjective questions and the subjective answer data using an artificial intelligence model.

[0284] Semantic similarity can refer to a numerical value indicating how similar the meanings conveyed by the correct answer sentence and the sentences included in the subjective answer data are to each other. Semantic similarity can be calculated based on the semantic consistency of the entire sentence or the preservation of its core meaning, rather than the agreement of individual words within the sentence.

[0285] The artificial intelligence model may include a sentence embedding-based semantic comparison model. The artificial intelligence model may include, for example, a language model that converts sentences into fixed-length vectors.

[0286] A method according to one embodiment can convert the correct answer sentence and the sentence included in the subjective answer data into semantic vectors, respectively.

[0287] A method according to one embodiment can calculate the vector similarity between a semantic vector corresponding to a correct answer sentence and a semantic vector corresponding to subjective answer data. The vector similarity can be calculated as at least one of cosine similarity or inner product-based similarity.

[0288] Semantic similarity can be calculated as a real value between 0 and 1, for example. Semantic similarity can indicate that the closer the value is to 1, the more similar the meaning of the correct answer sentence and the subjective answer data is. Semantic similarity can indicate that the closer the value is to 0, the more different the meaning of the correct answer sentence and the subjective answer data is.

[0289] For example, in a case where "I don't want to gain weight" is presented as a subjective question in the translation section and the correct answer sentence is set as "I don't want to gain weight," and a student inputs "I don't want to gain weight" as subjective answer data, the method according to one embodiment can determine that the two sentences convey the same meaning. In this case, the method according to one embodiment can calculate the semantic similarity as 0.95.

[0290] For example, in a case where "I go to school" is presented as a subjective question in the writing section and the correct answer sentence is set as "I go to school," and a student enters "I am going to school" as the subjective answer data, the method according to one embodiment can determine that although there is a difference in tense, the core meaning is the same. In this case, the method according to one embodiment can calculate the semantic similarity as 0.90.

[0291] For example, in a case where "If you were here, I would be happy" is presented as a subjective question in the writing section and the correct answer sentence is set as "If you were here, I would be happy," and a student enters "If you are here, I will be happy." as the subjective answer data, the method according to one embodiment can determine that the core emotional expression is preserved despite the conditional error. In this case, the method according to one embodiment can calculate the semantic similarity as 0.75. However, it is not limited thereto.

[0292] A method according to one embodiment may apply different interpretation criteria for semantic similarity depending on the educational process stage or evaluation area.

[0293] For example, in the elementary education curriculum, if semantic similarity exceeds a certain level, it may be determined that meaning has been conveyed even if grammatical errors exist. For example, in the higher education curriculum, even if semantic similarity is high, the presence of grammatical structure errors may be reflected as a factor for point deduction. However, this is not limited to these cases.

[0294] In step 720, the method according to one embodiment may include the step of calculating partial scores for subjective answer data based on semantic similarity.

[0295] A method according to one embodiment can calculate a semantic score by multiplying the semantic similarity value by 10 points when the total score for one subjective question is set to 10 points. However, this is merely an example, and the method of calculating the semantic score is not limited thereto.

[0296] A method according to one embodiment can use the calculated semantic score as a partial score for subjective answer data.

[0297] For example, in a case where "I don't want to gain weight" is presented as a subjective question in the translation section and the correct answer sentence is set as "I don't want to gain weight," and a student inputs "I don't want to gain weight" as the subjective answer data, the method according to one embodiment can determine that the two sentences contain the same core meaning and negative intention. In this case, the semantic similarity can be calculated as 0.95, and the method according to one embodiment can calculate the semantic score as 9.5 points.

[0298] For example, in a case where "I go to school" is presented as a subjective question in the writing section and the correct answer sentence is set as "I go to school," and a student enters "I am going to school" as the subjective answer data, the method according to one embodiment can determine that although there is a difference in tense expression, the act of movement and the destination are the same. In this case, the semantic similarity can be calculated as 0.90, and the method according to one embodiment can calculate the semantic score as 9.0 points.

[0299] For example, in a case where the subjective question "If you were here, I would be happy" is presented as a subjective question in the writing section and the correct answer sentence is set as "If you were here, I would be happy," and a student enters "If you are here, I will be happy" as the subjective answer data, the method according to one embodiment can determine that although the use of the conditional mood is inaccurate, the conditional relationship with the emotion of happiness is maintained. In this case, the semantic similarity can be calculated as 0.75, and the method according to one embodiment can calculate the semantic score as 7.5 points. However, it is not limited thereto.

[0300] A method according to one embodiment may combine a semantic score with a grammar element-based partial score calculated based on sentence structure, word order, tense, and vocabulary. The grammar element-based partial score is a score calculated by summing the sentence structure score, word order score, tense score, and vocabulary score, each with a maximum score of 2.5 points, and may be set to a maximum of 10 points per question. However, the sentence structure score, word order score, tense score, vocabulary score, and / or maximum score per question are merely examples and are not limited thereto.

[0301] The semantic score can refer to a score indicating how faithfully the subjective answer data preserves the meaning of the correct answer sentence.

[0302] A method according to one embodiment can produce grammatical element-based partial scores and semantic scores as independent evaluation results. Subsequently, a method according to one embodiment can combine the two scores to calculate a final partial score for subjective answer data.

[0303] A method according to one embodiment can set the average value of the grammar element-based partial score and the semantic score as the final partial score.

[0304] For example, if the grammar element-based partial score is calculated as 9.2 points and the semantic score as 7.6 points, the method according to one embodiment may calculate the average of the two scores, 8.4 points, as the final partial score. However, it is not limited thereto.

[0305] A method according to one embodiment can calculate a final partial score by assigning different weights to a grammar element-based partial score and a semantic score. In this case, the weights assigned to the grammar element-based partial score and the semantic score can be changed by the user.

[0306] For example, in a case where a weight of 0.4 is assigned to the grammar element-based partial score and a weight of 0.6 is assigned to the semantic score, and the grammar element-based partial score is 9.2 points and the semantic score is 7.6 points, the method according to one embodiment can calculate the final partial score as 8.2 points. However, it is not limited thereto.

[0307] A method according to one embodiment may set the method of combining two scores differently depending on the evaluation area.

[0308] For example, in translation areas where the preservation of meaning is considered important, the weight of the semantic score may be set higher than the weight of the grammar element-based partial score. In this case, if the grammar element-based partial score is 8.8 points and the semantic score is 9.4 points in a subjective question in the translation area, the final partial score may be calculated as 9 points or higher, with the semantic score having a greater influence.

[0309] For example, the method according to one embodiment may set the weight of the grammar element-based partial score higher than the semantic score in a writing area where the accuracy of sentence structure and tense is deemed important. In this case, if the grammar element-based partial score is 7.2 points and the semantic score is 8.8 points in a subjective question in the writing area, the final partial score may be calculated as less than 8 points. However, it is not limited thereto.

[0310] A method according to one embodiment may set different criteria for combining two scores depending on the stage of the educational process.

[0311] For example, in the elementary curriculum, the transmission of meaning is evaluated first, so the weighting of the meaning score may be set high, while in the higher education curriculum, grammatical accuracy is emphasized, so the weighting of grammar-based partial scores may be set high. However, this is not limited to these examples.

[0312] FIG. 8 is a flowchart illustrating a method according to one embodiment.

[0313] Step 150 of FIG. 8 corresponds to Step 150 of FIG. 1 to 6, and Step 610 and Step 620 of FIG. 8 correspond to Step 610 and Step 620 of FIG. 6 and FIG. 7, respectively.

[0314] In step 810, the method according to one embodiment may include the step of obtaining partial scores for subjective answer data for at least one subjective question belonging to the same educational course stage as the educational course stage corresponding to the objective question determined to be the correct answer among the objective answer data.

[0315] Multiple-choice questions can be arranged by difficulty level according to the curriculum stages, and each multiple-choice question can be pre-mapped to correspond to a specific curriculum stage.

[0316] A method according to one embodiment can determine whether each multiple-choice question is correct based on multiple-choice answer data and identify an educational process stage corresponding to the multiple-choice question determined to be correct. Subsequently, a method according to one embodiment can select one or more subjective questions classified to belong to the same educational process stage as the identified educational process stage.

[0317] A method according to one embodiment can obtain partial scores of subjective answer data already calculated for selected subjective questions. Here, the partial score is a score calculated by combining a grammar element-based partial score and a semantic score, and may represent an indicator that quantitatively represents the learning achievement level for each subjective question.

[0318] A method according to one embodiment allows for obtaining all partial scores for each subjective question when there are multiple subjective questions belonging to the same educational course stage. In this case, the method according to one embodiment may set the average value or weighted average value of the multiple partial scores as the subjective evaluation score for the corresponding educational course stage. However, it is not limited thereto.

[0319] For example, if a large number of multiple-choice questions corresponding to the first-year middle school curriculum are determined to be correct, the method according to one embodiment may select all subjective questions in the writing area and subjective questions in the translation area that belong to the same curriculum stage as the first-year middle school curriculum. If the average of the partial scores for the subjective questions in the writing area is calculated as 8.6 points and the average of the partial scores for the subjective questions in the translation area is calculated as 7.8 points, the method according to one embodiment may obtain 8.2 points, which is the average of the two partial scores, as the subjective evaluation score for the first-year middle school curriculum.

[0320] For example, if only some of the multiple-choice questions corresponding to the first year of high school curriculum are determined to be correct, the method according to one embodiment may select at least one of the subjective questions belonging to the same curriculum stage as the first year of high school curriculum and obtain partial credit for the said subjective question.

[0321] For example, if most of the multiple-choice questions corresponding to the 6th grade elementary school curriculum are determined to be correct, the method according to one embodiment may select multiple subjective questions in the translation area that belong to the same curriculum stage as the 6th grade elementary school curriculum. If the partial scores of the selected subjective questions in the translation area are calculated as 9.1 points, 8.9 points, and 9.3 points, respectively, the method according to one embodiment may obtain approximately 9.1 points, which is the average value of the above partial scores, as the subjective evaluation score for the 6th grade elementary school curriculum. However, it is not limited thereto.

[0322] A method according to one embodiment can evaluate the learning achievement of each educational course stage more precisely and stably by selecting an educational course stage with guaranteed reliability based on the results of solving multiple-choice questions and utilizing subjective evaluation results that maintain consistency with the educational course stage in conjunction.

[0323] In step 820, the method according to one embodiment may include the step of calculating the overall achievement level of each educational course stage by applying weights to the density of incorrect answers based on partial scores.

[0324] The overall achievement level for each curriculum stage may refer to a numerical indicator that comprehensively reflects the level of conceptual understanding and linguistic expression ability demonstrated by students for a specific curriculum stage. As a value calculated based on partial scores with weighted error density, the overall achievement level may represent a numerical value indicating learning stability for each curriculum stage. The overall achievement level may be calculated by considering both the error density derived from the results of solving multiple-choice questions corresponding to the same curriculum stage and the partial scores derived from the evaluation results of subjective questions.

[0325] A method according to one embodiment can determine a weight to be applied to the error density based on partial scores calculated for each stage of the educational course. Here, partial scores are scores calculated from subjective questions belonging to the same stage of the educational course, and may represent an indicator of a student's expression ability and concept application ability.

[0326] A method according to one embodiment may set different weights to be applied to the error density depending on the magnitude of the partial score. That is, the weight may be set to mitigate the negative impact of the error density on overall achievement as the partial score increases, and the weight may be set to reflect the error density more significantly on overall achievement as the partial score decreases.

[0327] For example, in a method according to one embodiment, if the average partial score calculated from subjective questions belonging to the same educational course stage is 9 points or higher, the weight applied to the error density can be set to 1.0. Additionally, if the average partial score is 8 points or higher but less than 9 points, the weight can be set to 0.9; if the average partial score is 7 points or higher but less than 8 points, the weight can be set to 0.8; and if the average partial score is less than 7 points, the weight can be set to 0.6. However, these criteria for setting weights are not limited thereto.

[0328] A method according to one embodiment can obtain an error density calculated from multiple-choice questions corresponding to the same educational course stage. Subsequently, a method according to one embodiment can calculate an overall achievement level for each educational course stage by applying a weight set based on partial scores to the error density.

[0329] A method according to one embodiment can calculate an average partial score for subjective questions according to each stage of the educational course. The average partial score may refer to the average value of partial scores calculated based on sentence structure, word order, tense, and vocabulary for subjective questions belonging to the same stage of the educational course.

[0330] A method according to one embodiment can set a weight to be applied to the error density based on the calculated average partial score. The weight may be set so that the higher the average partial score, the more the impact of the error density on the overall achievement is mitigated.

[0331] A method according to one embodiment can calculate a correction coefficient by applying a weight set based on the average partial score to the error density. The correction coefficient may be a value indicating the degree to which the error density is reflected in the average partial score.

[0332] A method according to one embodiment can calculate the overall achievement level for each stage of the educational course by reflecting a correction factor in the average partial score.

[0333] A method according to one embodiment can calculate an overall achievement level from an average partial score and a correction factor based on at least one of a combination method, multiplication, subtraction, or a function-based calculation method. However, it is not limited thereto.

[0334] The multiplication method may refer to a method of calculating overall achievement by multiplying the average partial score by a correction factor calculated based on the density of incorrect answers. For example, if the average partial score is 8.0 points and the correction factor based on the density of incorrect answers is set to 0.8, the method according to one embodiment may calculate the overall achievement as 6.4 points by multiplying the average partial score of 8.0 points by the correction factor of 0.8. However, it is not limited thereto.

[0335] The subtraction method may refer to a method of calculating overall achievement by converting the error density into a deduction factor and subtracting it from the average partial score. For example, if the average partial score is 8.0 points and the error density is 0.3, the method according to one embodiment may calculate the overall achievement as 6.8 points by subtracting a deduction value of 1.2 points, corresponding to the error density of 0.3, from the average partial score. In this case, the deduction value may be calculated by multiplying the error density by a certain coefficient. However, it is not limited thereto.

[0336] A function-based calculation method may refer to a method of calculating overall achievement according to a predefined function that takes average partial scores and error density as input values. For example, a method according to one embodiment may calculate overall achievement according to a non-linear function or a segment-based function that takes average partial scores and error density as inputs. In this case, the function may be set so that the reflection ratio of the average partial score decreases sharply in segments where the error density exceeds a specific threshold. However, it is not limited thereto.

[0337] A method according to one embodiment may set different combination methods depending on the educational process stage or evaluation area. For example, a subtraction method or a gentle function-based calculation method may be applied in the elementary education process, while a multiplication method or a function-based calculation method that significantly reflects the influence of error density may be applied in the higher education process.

[0338] For example, if the average partial score of subjective questions corresponding to the first year of middle school curriculum in the secondary education curriculum is calculated to be 8.2 points, the method according to one embodiment can calculate a correction coefficient by setting the weight to 0.9 and applying the corresponding weight to the error density of 0.45 calculated from objective questions corresponding to the same curriculum stage. In this case, the method according to one embodiment can calculate the overall achievement level for the first year of middle school curriculum as approximately 4.88 points by reflecting the calculated correction coefficient to the average partial score of 8.2 points.

[0339] For example, if the average partial score of subjective questions corresponding to the first year of high school curriculum in the higher education curriculum is calculated to be 7.5 points, the method according to one embodiment can calculate the overall achievement level as 5.7 points by setting the weight to 0.8 and applying the corresponding weight to the error density of 0.30.

[0340] For example, if the average partial score of subjective questions corresponding to the 6th grade elementary school curriculum is calculated to be 9.0 points, the method according to one embodiment can calculate the overall achievement level for the 6th grade elementary school curriculum as 7.65 points by setting the weight to 1.0 based on the high average partial score and minimizing the influence of the error density of 0.15. However, it is not limited thereto.

[0341] A method according to one embodiment can identify which stage of the educational process a student is relatively weak in by comparing the overall achievement levels calculated for each stage of the educational process.

[0342] In step 830, the method according to one embodiment may include the step of identifying a supplementary education course stage based on the overall achievement level of each education course stage.

[0343] A method according to one embodiment can compare the overall achievement calculated for each stage of the educational process with a pre-set standard achievement. The standard achievement may refer to an indicator of the minimum level of achievement for determining that the corresponding stage of the educational process has been sufficiently understood, and may be changed by the user. A method according to one embodiment can identify the corresponding stage of the educational process as a supplementary educational process stage if the overall achievement for each stage of the educational process is less than the standard achievement.

[0344] A method according to one embodiment can identify the educational course stage with the lowest overall achievement among a plurality of educational course stages as a priority supplementary educational course stage. A method according to one embodiment can identify all of a plurality of educational course stages below a standard achievement level as supplementary educational course stages.

[0345] For example, in a case where the overall achievement is calculated for the middle school 1st, 2nd, and 3rd grade curriculums included in the secondary education curriculum, and the overall achievement of the middle school 1st grade curriculum is calculated as 6.5 points, the overall achievement of the middle school 2nd grade curriculum is calculated as 7.4 points, and the overall achievement of the middle school 3rd grade curriculum is calculated as 8.1 points, and the standard achievement is set as 7.0 points, the method according to one embodiment can determine that only the overall achievement of the middle school 1st grade curriculum is below the standard achievement, and thus identify the middle school 1st grade curriculum as a supplementary education curriculum stage.

[0346] For example, in the case where the overall achievement scores for the 4th, 5th, and 6th grade elementary school curriculums are calculated as 6.8, 6.6, and 8.9 points, respectively, and the standard achievement score is 6.9 points, the method according to one embodiment can identify both the 4th grade elementary school curriculum and the 5th grade elementary school curriculum as supplementary education curriculum stages.

[0347] A method according to one embodiment may prioritize providing learning material data corresponding to a training course step identified as a supplementary training course step, or repeatedly place it within a learning schedule.

[0348] Accordingly, the method according to one embodiment can clearly identify the learning weaknesses of a student based on the overall achievement level at each stage of the educational course, and rationally derive the supplementary educational course stage based on overall learning outcomes rather than the number of simple incorrect answers.

[0349] FIG. 9 is a flowchart illustrating a method according to one embodiment.

[0350] Step 150 of FIG. 9 corresponds to Step 150 of FIG. 1 to 6 and FIG. 8.

[0351] In step 910, the method according to one embodiment may include the step of generating a plurality of sets of evaluation questions that are included in the scope of education of a supplementary education course step for evaluating whether a student has correctly learned the learning material data.

[0352] An assessment item set may refer to an assessment tool to verify whether learning concepts covered in the supplementary education course stage have actually been understood and applied. An assessment item set may consist of multiple multiple-choice questions and multiple open-ended questions.

[0353] A method according to one embodiment may construct a set of evaluation items based on an educational scope corresponding to an educational course stage identified as a supplementary educational course stage. The educational scope may be limited to the learning content covered in the educational course stage identified as a supplementary educational course stage. That is, if the supplementary educational course stage is the first-year middle school curriculum among secondary education courses, the educational scope may be limited to the grammar concepts, vocabulary range, sentence structure types, and reading comprehension level required in the first-year middle school curriculum. However, it is not limited thereto.

[0354] A method according to one embodiment can generate all items included in a plurality of sets of evaluation items only within the educational scope of the supplementary education course stage. That is, if the supplementary education course stage is the middle school first-year curriculum, the items included in the sets of evaluation items may be restricted so as not to include concepts at the middle school second-year or higher education level.

[0355] A set of multiple assessment items may consist of multiple multiple-choice items and multiple open-ended items, and each item may be designed to assess the same learning concept corresponding to a stage of the supplementary education curriculum. Here, maintaining the same learning concept may mean that the grammar item or language rule being assessed is identical to the existing problem.

[0356] A method according to one embodiment may configure a plurality of multiple-choice questions and a plurality of subjective questions included in a plurality of evaluation question sets as questions related to learning material data.

[0357] A method according to one embodiment can generate multiple-choice or short-answer questions by referring to the main text, example sentences, practice problems, or unit-specific core concepts of a textbook included in the learning material data. In this case, the generated multiple-choice and short-answer questions can be configured to directly correspond to the learning concepts covered in the learning material data.

[0358] For example, if the concept of the infinitive is explained in a specific unit of a textbook included in the learning material data, the method according to one embodiment can generate multiple-choice questions and / or short-answer questions based on example sentences or similar sentence structures included in the textbook.

[0359] For example, if a reading passage included in learning material data includes a specific vocabulary level or type of sentence structure, a method according to one embodiment can generate items for evaluating vocabulary comprehension and / or sentence interpretation ability within the same educational scope as said reading passage.

[0360] For example, if the supplementary education stage is the first year of middle school curriculum and the core grammar concept of that curriculum is the infinitive, then the multiple-choice and short-answer questions included in multiple sets of assessment questions may all be designed to assess whether the infinitive is used. In this case, grammar concepts from higher education stages, such as conditional sentences, participial phrases, or relative pronouns, may not be included in the sets of assessment questions.

[0361] A method according to one embodiment can generate evaluation questions by referring to a plurality of previously used multiple-choice questions, a plurality of subjective questions, or example questions included in learning material data. In this case, all questions referenced may be limited to questions that fall within the scope of education of the supplementary education course stage.

[0362] A set of multiple evaluation items may include multiple items in which unique data within multiple multiple-choice questions and multiple subjective questions has been modified. Unique data may refer to elements for changing the appearance of an item without affecting the evaluation concept of the item. Unique data may include at least one of proper nouns, numbers, personal pronouns, or tenses included in the multiple multiple-choice questions and multiple subjective questions.

[0363] Evaluation items maintain the same learning concepts as multiple multiple-choice questions and multiple subjective questions, but can be generated in a modified form with only the unique data included in the items.

[0364] A method according to one embodiment can identify unique data included in an existing problem and change the unique data to a different value according to a predefined substitution rule.

[0365] For example, if an existing item evaluating the concept of the infinitive at the first year of middle school curriculum includes the sentence "Tom wants to buy a book," the method according to one embodiment can change the proper noun "Tom" to "Jane" or change the object "a book" to "a pen" to generate an evaluation item "Jane wants to buy a pen." In this case, the concept to be evaluated can be maintained as the infinitive of the first year of middle school curriculum.

[0366] For example, if a subjective question evaluating the concept of tense at the first year of middle school education includes the sentence "I played soccer yesterday.", the method according to one embodiment can generate an evaluation question such as "She studied English yesterday." by changing the unique data. Even in this case, the scope of evaluation may be limited to the concept of the past tense at the first year of middle school education.

[0367] For example, if a subjective question evaluating the concept of the conditional includes the sentence "If I were rich, I would travel around the world," the method according to one embodiment may generate an evaluation question such as "If she were taller, she would play basketball" by modifying the unique data. In this case, the evaluation concept of the past conditional structure may be maintained as is. However, it is not limited thereto.

[0368] A method according to one embodiment can generate multiple sets of evaluation items having different unique data combinations for a single supplementary education course stage.

[0369] A method according to one embodiment can generate evaluation questions by referring to example questions included in learning material data. A method according to one embodiment can reconstruct evaluation questions by modifying unique data while maintaining the learning concept of the example questions.

[0370] A method according to one embodiment can generate multiple sets of evaluation items having different unique data combinations for a single supplementary education course stage.

[0371] Accordingly, the set of assessment questions all fall within the same educational scope, while effectively preventing the selection of correct answers based on rote memorization during repeated assessments.

[0372] A method according to one embodiment can reliably verify whether a student has achieved learning in a corresponding educational stage by strictly limiting the evaluation target to an educational scope that exactly corresponds to the supplementary education course stage, while maintaining the concepts of existing problem and learning material data and changing only unique data to generate a set of evaluation questions.

[0373] In step 920, the method according to one embodiment may include the step of obtaining test answer data of a student for a plurality of questions included in a plurality of sets of evaluation questions.

[0374] Test answer data may refer to response information entered by students for each question included in the set of evaluation questions.

[0375] A method according to one embodiment can output a set of evaluation questions to a student and then sequentially obtain test answer data based on the student's input.

[0376] The items included in the set of evaluation items may include multiple-choice and short-answer items, and test answer data may be obtained in different forms depending on the type of item.

[0377] Test answer data for multiple-choice questions can be obtained as information on the option selected by the student among multiple options. For example, if a student selects the third option in a specific multiple-choice question, the test answer data can be recorded as data identifying that option.

[0378] Test answer data for open-ended questions can be obtained from character-based response data entered by students. If the open-ended question belongs to the writing domain, the test answer data may include English sentence data entered by students. If the open-ended question belongs to the translation domain, the test answer data may include Korean sentence data entered by students.

[0379] A method according to one embodiment can independently obtain test answer data for each set of evaluation questions when a plurality of sets of evaluation questions are generated. In this case, the test answer data can be stored separately by set of evaluation questions and by question.

[0380] A method according to one embodiment can store item identification information, evaluation item set identification information, or educational course stage information together with test answer data.

[0381] For example, if the supplementary education course stage is identified as the first-year middle school curriculum, the method according to one embodiment may provide a set of evaluation questions limited to the first-year middle school curriculum to the student. Subsequently, the method according to one embodiment may obtain answer information selected by the student for the multiple-choice questions included in the set of evaluation questions and sentence data entered by the student for the subjective questions as test answer data. However, it is not limited thereto.

[0382] In step 930, the method according to one embodiment may include the step of determining the learning achievement of a student in a plurality of evaluation areas based on test answer data.

[0383] A method according to one embodiment can map each item included in the test answer data to the evaluation area to which the item belongs.

[0384] A method according to one embodiment can classify test answer data by evaluation area and then calculate the achievement level for each evaluation area using the classified test answer data.

[0385] A method according to one embodiment can determine whether a multiple-choice question is correct by comparing test answer data with the correct answer for each question. A method according to one embodiment can calculate the number of correct answers or the ratio of correct answers for multiple-choice questions by evaluation area.

[0386] A method according to one embodiment can analyze responses included in test answer data for subjective questions using an artificial intelligence model. A method according to one embodiment can calculate scores for each subjective question based on sentence structure, word order, tense, vocabulary, or semantic similarity. The method of calculating scores for each subjective question according to one embodiment may be identical or similar to the method of calculating partial scores for subjective questions according to one embodiment.

[0387] A method according to one embodiment can calculate a comprehensive score for each evaluation area by combining scores calculated from multiple-choice questions and scores calculated from subjective questions for each evaluation area. The combination method may include, but is not limited to, calculating an average value, calculating a weighted average, or a pre-set rule-based combination method.

[0388] A method according to one embodiment can determine the level of learning achievement for a given evaluation area by comparing the total score for each evaluation area with a pre-set achievement standard. The achievement standard may be a reference value representing the minimum learning level required for the given evaluation area.

[0389] For example, if the total score calculated for the grammar area is above the achievement standard, the method according to one embodiment may determine that the learning achievement in the grammar area is sufficient. Conversely, if the total score calculated for the grammar area is below the achievement standard, the method according to one embodiment may determine that the learning achievement in the grammar area is insufficient.

[0390] For example, if the total score calculated for the grammar area is 7.8 points and the achievement standard for the grammar area is set to 7.0 points, the method according to one embodiment can determine that the learning achievement level for the grammar area is sufficient because the total score for the grammar area is higher than or equal to the achievement standard.

[0391] For example, it can be assumed that in the vocabulary domain, the correct answer rate for multiple-choice questions is high, but the score calculated from subjective questions is low. In this case, the method according to one embodiment may determine that while the recognition of vocabulary meaning is sufficient, the ability to actually use it is insufficient, and thus determine the learning achievement level in the vocabulary domain as partially insufficient. However, it is not limited thereto.

[0392] A method according to one embodiment can derive different learning achievements for each evaluation area, thereby enabling a detailed identification of which evaluation area a student shows strengths in and which evaluation area requires additional improvement.

[0393] A method according to one embodiment can precisely determine the learning achievement of students by area by systematically analyzing test answer data by evaluation area and synthesizing the results of multiple-choice and subjective evaluations.

[0394] FIG. 10 is a flowchart illustrating a method according to one embodiment.

[0395] Steps 920 and 930 of Fig. 10 correspond to steps 920 and 930 of Fig. 9, respectively.

[0396] In step 1010, the method according to one embodiment may include the step of obtaining a plurality of diagnostic scores for each evaluation area based on objective answer data and subjective answer data.

[0397] Diagnostic scores can refer to an indicator that quantitatively represents the basic learning level demonstrated by a student in each assessment area prior to the supplementary education course.

[0398] A method according to one embodiment can classify objective answer data and subjective answer data by evaluation area, and then calculate a diagnostic score based on the classified data.

[0399] Multiple-choice answer data may include response results to multiple-choice questions entered by students for each evaluation area. A method according to one embodiment may calculate the number of correct answers or the ratio of correct answers by comparing the multiple-choice answer data with the correct answers for each question for each evaluation area.

[0400] Subjective answer data may include subjective sentence data entered by students for each evaluation area. A method according to one embodiment may analyze the subjective answer data using an artificial intelligence model and calculate a score for each subjective question based on sentence structure, word order, tense, vocabulary, and semantic similarity.

[0401] A method according to one embodiment can obtain a diagnostic score for each evaluation area by combining the score calculated from multiple-choice questions and the score calculated from subjective questions for each evaluation area. The combination method may include, but is not limited to, the calculation of an average value, the calculation of a weighted average, or a pre-set rule-based combination method.

[0402] For example, if the correct answer rate for multiple-choice questions in the grammar area is calculated as 70 percent and the average score of subjective questions is converted to a maximum of 80 points, the method according to one embodiment can combine the two values ​​to calculate a diagnostic score of 75 points for the grammar area.

[0403] For example, in cases where the correct answer rate for multiple-choice questions in the vocabulary domain is calculated to be high but the score calculated for subjective questions is relatively low, the method according to one embodiment can determine that the level of vocabulary recognition is sufficient but the ability to actually use it is limited, and thus calculate the diagnostic score for the vocabulary domain at an intermediate level.

[0404] In step 1020, the method according to one embodiment may include the step of obtaining a supplementary achievement score for each of the multiple evaluation areas by comparing test answer data for a plurality of items included in an evaluation item set with correct answers for a plurality of items for each of the multiple evaluation areas.

[0405] The supplementary achievement score may refer to an indicator representing the extent to which a student has achieved learning outcomes in each assessment area after learning for the supplementary education course stages has been performed.

[0406] A method according to one embodiment can map each item included in an evaluation item set to an evaluation area to which the item belongs. Subsequently, a method according to one embodiment can classify test answer data by evaluation area and compare the classified test answer data with the correct answer of the corresponding item.

[0407] A method according to one embodiment can determine whether a multiple-choice question is correct by comparing test answer data with the correct answer for each question. A method according to one embodiment can calculate the number of correct answers or the percentage of correct answers for multiple-choice questions by evaluation area.

[0408] A method according to one embodiment can analyze student responses included in test answer data for subjective questions using an artificial intelligence model. A method according to one embodiment can calculate scores for each subjective question based on sentence structure, word order, tense, vocabulary, and semantic similarity.

[0409] A method according to one embodiment can obtain a supplementary achievement score by combining the score calculated from multiple-choice questions and the score calculated from subjective questions for each evaluation area. The combination method may include calculating an average value, calculating a weighted average, or a pre-set rule-based combination method. However, it is not limited thereto.

[0410] For example, if the correct answer rate of the multiple-choice questions included in the set of evaluation questions for the grammar area is calculated to be 85 percent and the average score calculated from the subjective questions is 8.5 points out of 10, the method according to one embodiment can combine the two values ​​to calculate a supplementary achievement score for the grammar area of ​​8.5 points or an equivalent converted score.

[0411] For example, if the correct answer rate for multiple-choice questions in the vocabulary domain is calculated to be 90 percent but the average score calculated for subjective questions is relatively low, the method according to one embodiment may determine that vocabulary recognition ability has improved but actual application ability requires additional supplementation, and thus calculate the supplementary achievement score for the vocabulary domain at an intermediate level. However, it is not limited thereto.

[0412] In step 1030, the method according to one embodiment may include the step of calculating a skill improvement value for each of the multiple evaluation areas by comparing a diagnostic score and a supplementary achievement score for each of the multiple evaluation areas.

[0413] The proficiency improvement score may refer to an indicator that quantitatively represents the extent to which a student's learning outcomes have improved, based on the period before and after the remedial education course. The diagnostic score may be a score representing the basic learning level for each assessment area obtained prior to the remedial education course, and the remedial achievement score may be a score calculated based on a set of assessment items administered after the remedial education course. The proficiency improvement score can be calculated by directly comparing the diagnostic score and the remedial achievement score for the same assessment area.

[0414] A method according to one embodiment can calculate a skill improvement score by subtracting the diagnostic score from the supplementary achievement score for each evaluation area. The skill improvement score may have a positive, negative, or zero (0) value.

[0415] For example, if the diagnostic score for the grammar area is calculated as 6.8 points and the supplementary achievement score after the supplementary education course is calculated as 8.1 points, the method according to one embodiment can calculate the difference between the two scores, 1.3 points, as the improvement score for the grammar area.

[0416] For example, if the diagnostic score for the vocabulary area is 7.5 points and the supplementary achievement score is calculated as 7.6 points, the skill improvement score may be calculated as 0.1 points, which may indicate that learning improvement in the vocabulary area was limited.

[0417] For example, if the diagnostic score for the reading comprehension area is 8.0 and the supplementary achievement score is calculated as 7.8, the skill improvement score may be calculated as -0.2, which may mean that the achievement level in that area has not improved sufficiently even after supplementary education. However, it is not limited to this.

[0418] A method according to one embodiment can identify which evaluation area a student has achieved substantial learning improvement by comparing the calculated skill improvement figures by evaluation area.

[0419] A method according to one embodiment can determine an evaluation area where the skill improvement value is below a preset minimum improvement standard as an area requiring additional improvement.

[0420] For example, if the minimum improvement standard is set to 0.5 points, the method according to one embodiment may determine that sufficient improvement has been made in the grammar area of ​​the student's learning if the skill improvement score in the grammar area is 1.3 points, and the method according to one embodiment may determine that additional supplementation is needed in the vocabulary area if the skill improvement score in the vocabulary area is 0.1 points. However, it is not limited thereto.

[0421] In step 1040, the method according to one embodiment may include the step of determining learning achievement for a plurality of evaluation areas based on skill improvement figures.

[0422] Learning achievement may refer to the result of determining whether a student has reached the target learning level in each evaluation area after the remedial education course. The proficiency improvement score is a value calculated by comparing the diagnostic scores and remedial achievement scores before and after the remedial education course, and can be used as an indicator of whether the remedial education course actually contributed to the improvement of the student's learning ability.

[0423] A method according to one embodiment may compare the skill improvement figures calculated for each evaluation area with pre-set achievement judgment criteria. The achievement judgment criteria may refer to a minimum improvement standard for determining that sufficient supplementary learning has been achieved in the corresponding evaluation area. The achievement judgment criteria may be set differently for each evaluation area.

[0424] A method according to one embodiment may determine that the learning achievement for a given evaluation area is satisfied if the skill improvement score for each evaluation area is greater than or equal to the achievement judgment standard. Conversely, if the skill improvement score for each evaluation area is less than the achievement judgment standard, the learning achievement for that evaluation area may be determined to be insufficient.

[0425] For example, in a case where the achievement judgment criterion for the grammar area is set to 0.5 points and the skill improvement score for the grammar area is calculated to be 1.3 points, the method according to one embodiment can determine that the learning achievement level for the grammar area has been sufficiently achieved. Conversely, in a case where the skill improvement score for the grammar area is calculated to be 0.2 points, the method according to one embodiment can determine that the learning achievement level for the grammar area is not yet sufficient.

[0426] For example, in a case where the achievement judgment criterion for the vocabulary area is set to 0.3 points and the skill improvement score for the vocabulary area is calculated to be 0.6 points, the method according to one embodiment may determine that the learning achievement level for the vocabulary area is satisfied. However, it is not limited thereto.

[0427] A method according to one embodiment can individually determine learning achievement for each evaluation area. That is, a student may be determined to have satisfied learning achievement in a specific evaluation area, and may be determined to require additional supplementation in another evaluation area.

[0428] A method according to one embodiment can determine the overall learning status of a student by synthesizing the results of the learning achievement judgment for each evaluation area.

[0429] A method according to one embodiment can control the provision of additional supplementary training courses or repetitive learning for evaluation areas where learning achievement is judged to be insufficient.

[0430] A method according to one embodiment can schedule learning material data for the next stage of the educational process when the learning achievement level meets a standard value.

[0431] Subjective questions and / or subjective items are not limited to the writing or translation domains and may further include questions in the form of dictation. Subjective questions and / or subjective items in the form of dictation may refer to questions that require a student to accurately input the content in text after listening to a sentence or paragraph output as audio. Since the subjective questions and / or subjective items in the form of dictation are linked to the listening domain, the method according to one embodiment can evaluate the student's listening comprehension ability, spelling accuracy, and sentence reproduction ability together.

[0432] A method according to one embodiment can obtain character data entered by a student for a dictation-type subjective question and / or subjective item as subjective answer data. A method according to one embodiment can determine spelling errors, omissions of words, additions of words, or sentence structure errors by comparing the obtained subjective answer data with the correct answer sentence.

[0433] A method according to one embodiment can identify the supplementary education course steps by subdividing them into a plurality of sub-steps. The supplementary education course steps may be divided, for example, into a first supplementary step and a second supplementary step.

[0434] The first supplementary stage may be defined as a case where the error density or overall achievement falls within the first threshold range. The first supplementary stage may correspond to a stage where the student recognizes the concept but repetitive mistakes occur. A method according to one embodiment can automatically schedule individual learning material data for self-directed learning when identified as the first supplementary stage.

[0435] The second supplementary stage may be defined as a case where the density of incorrect answers or overall achievement exceeds a second threshold range. The second supplementary stage may correspond to a stage where separate instructional intervention is required due to a lack of concept understanding itself. A method according to one embodiment may generate notification information for class assignment to the user when identified as the second supplementary stage.

[0436] A method according to one embodiment may further include a preliminary evaluation step for determining the basic learning level of a student prior to the main evaluation. The preliminary evaluation may consist of questions ranging from the elementary education curriculum to the beginning of the secondary education curriculum.

[0437] A method according to one embodiment may omit some evaluation areas or prioritize the placement of basic learning material data based on the results of a preliminary evaluation.

[0438] The user may perform an admission test for a student using the method according to one embodiment. The admission test may refer to an initial evaluation procedure to determine the student's current learning level.

[0439] Users may conduct test result counseling based on the results of the entrance test. Test result counseling may refer to a procedure for informing students or guardians of the evaluation results derived from the entrance test.

[0440] Users can decide whether to admit a student based on test results and counseling. The decision on admission may refer to a procedure for determining whether the student is suitable for the relevant educational course. Users can select a class of a level appropriate for the student based on the total score of the admission test.

[0441] Users can determine personalized assignments for students whose enrollment has been confirmed. Personalized assignments may refer to learning tasks corresponding to the student's learning weaknesses or supplementary education course stages.

[0442] Users can set a student's learning schedule based on personalized assignments. The learning schedule is a timeline that reflects personalized assignments and may include study plans that must be completed before the class.

[0443] The user can conduct the class with the students after administering a test on the day of the class. The test on the day of the class may refer to a procedure to verify whether learning has been performed or the level of understanding regarding personalized assignments.

[0444] After individual lessons, the user may administer a verification test to the student to check the lesson content. The verification test may refer to an evaluation procedure to check the level of understanding of the learning material covered in the lesson.

[0445] The user may conduct a retest for students based on the results of the verification test. A retest may refer to an additional evaluation procedure performed when the standard score is not met in the verification test.

[0446] The user may send the student home if the re-examination results meet the criteria. Even if the re-examination is not performed, the user may send the student home based on the results of the verification test.

[0447] FIG. 11 is a block diagram illustrating the components of a system according to one embodiment.

[0448] A system (1100) according to one embodiment may include a processor (1110) and a memory (1120).

[0449] The processor (1110) is configured to control a series of processes to enable the system (1100) to operate according to the embodiments described above, and may be composed of one or more processors.

[0450] The processor (1110) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. One or more processors included in the processor (1110) may be circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). The processor (1110) may be composed of at least one of, for example, a Central Processing Unit, a microprocessor, a Graphic Processing Unit, Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and Field Programmable Gate Arrays (FPGAs), but is not limited thereto.

[0451] The processor (1110) can write data to memory (1120) or read data stored in memory (1120), and in particular, can process data according to predefined operation rules by executing a program or at least one instruction stored in memory (1120).

[0452] The memory (1120) may store various data, programs, or applications for driving and controlling the system (1100) according to one embodiment. The memory (1120) may include non-volatile memory such as RAM (Random Access Memory) or SRAM (Static Random Access Memory), and volatile memory such as non-volatile memory including at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and PROM (Programmable Read-Only Memory).

[0453] The memory (1120) may store instructions, data structures, and program code that can be read by the processor (1110). In the following embodiments, the processor (1110) may be implemented by executing the instructions or code of the program stored in the memory (1120).

[0454] The memory (1120) may store learning data used in the process of performing the method according to one embodiment.

[0455] The memory (1120) may store learning material data classified by the stages of the supplementary education course. For example, the memory (1120) may store learning material data corresponding to the elementary education course, the secondary education course, the higher education course, or the advanced education course, each classified separately.

[0456] In the memory (1120), along with the learning material data, information on the educational process stage, evaluation area, or learning schedule corresponding to each learning material data can be stored in conjunction.

[0457] A method according to one embodiment may retrieve and output learning material data corresponding to a supplementary education course stage from among the learning material data stored in memory (1120), or schedule it according to a learning schedule. Additionally, the learning material data stored in memory (1120) may also be referenced during the process of generating a set of evaluation questions or providing learning materials for review.

[0458] A system (1110) according to one embodiment may include a communication unit (not shown).

[0459] The communication unit can communicate with an external device or server through at least one wired or wireless communication network by means of a processor (1110).

[0460] The communication unit may include at least one short-range communication module that performs communication according to communication standards such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC / RFID, Wi-Fi Direct, UWB, or ZIGBEE, and a long-range communication module that performs communication with a server to support long-range communication according to long-range communication standards. The long-range communication module may perform communication through a communication network according to 3G, 4G, and / or 5G communication standards, or a network for internet communication.

[0461] A system (1110) according to one embodiment can obtain learning data stored in an external database through a communication unit. Here, the database is a storage means for storing learning data and may be included in a server device separated from the system (1110). The database may be connected to the system (1110) via a wired communication network or a wireless communication network. For example, the database may be included in a server built on an internet network, a mobile communication network, or a private network.

[0462] A system (1110) according to one embodiment can transmit a request signal for learning material data stored in a database to a server through a communication unit. The request signal may include information specifying the necessary learning material data based on supplementary education course stages, evaluation areas, or learning schedule information.

[0463] The server may transmit requested learning material data to the system (1110) in response to a request signal received from the system (1110). The transmitted learning material data may include grammar explanation data, vocabulary learning data, audio data for listening, text data for reading, or practice problem data.

[0464] The method according to the above-described embodiment can be performed by a system (1100) including a memory (1120) and a processor (1110).

[0465] That is, the processor (1110) can control the operation of the system (1100) according to one embodiment so that the system (1100) according to one embodiment performs the operations described above by executing at least one instruction and / or instruction stored in the memory (1120).

[0466] A method according to one embodiment may include the step of arranging multiple multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a lower level to a higher level in the educational process.

[0467] A method according to one embodiment may include the step of outputting a plurality of multiple-choice questions for each of a plurality of evaluation areas.

[0468] A method according to one embodiment may include the step of obtaining multiple-choice answer data of a student for a plurality of multiple-choice questions.

[0469] A method according to one embodiment may include the step of identifying a supplementary education course stage in which a student's learning is weak for each of the multiple evaluation areas by identifying a section of questions in which the error density among the multiple multiple questions exceeds a threshold value, based on comparing multiple multiple-choice answer data with correct answers for multiple multiple-choice questions for each of the multiple evaluation areas.

[0470] A method according to one embodiment may include the step of obtaining learning material data corresponding to a supplementary education course step from at least one of a database or memory.

[0471] A method according to one embodiment may include the step of scheduling learning material data according to a learning schedule.

[0472] A method according to one embodiment may include the step of outputting scheduled learning data.

[0473] According to one embodiment, the incorrect answer density may refer to the ratio of the number of questions entered as incorrect answers by a student to the total number of questions included in a question group corresponding to the same educational course stage.

[0474] A method according to one embodiment may include the step of obtaining an error density for each of a plurality of evaluation areas at each stage of the educational process.

[0475] A method according to one embodiment may include the step of identifying a training course stage in which the density of incorrect answers first exceeds a threshold in ascending order of training course stages as a supplementary training course stage.

[0476] A method according to one embodiment may include a step of measuring a first response time from the point in time when a plurality of multiple-choice questions corresponding to a first educational course stage are output on a display to the point in time when multiple-choice answer data for a plurality of multiple-choice questions corresponding to a first educational course stage is obtained.

[0477] A method according to one embodiment may include a step of measuring a second response time from the point in time when a plurality of multiple-choice questions corresponding to a second educational course stage are output on a display to the point in time when multiple-choice answer data for a plurality of multiple-choice questions corresponding to a second educational course stage is obtained.

[0478] A method according to one embodiment may include the step of identifying a first educational course step as a supplementary educational course step when the first response time is smaller than the second response time.

[0479] According to one embodiment, the first educational course stage may be a higher educational course stage than the second educational course stage.

[0480] Multiple-choice answer data according to one embodiment may include data on the number of times an answer was modified before a student finally confirmed the answers to a plurality of multiple-choice questions.

[0481] A method according to one embodiment may include the step of identifying a training course stage containing multiple-choice questions for which the number of answer corrections exceeds the standard number of corrections as a supplementary training course stage when the number of answer corrections exceeds the standard number of corrections by comparing the number of answer corrections with the standard number of corrections based on multiple-choice answer data.

[0482] A method according to one embodiment may include the step of arranging a plurality of subjective questions to evaluate a student's writing ability and translation ability at each stage of the educational course from a lower stage to a higher stage.

[0483] A method according to one embodiment may include the step of outputting a plurality of subjective questions for each of a plurality of evaluation areas.

[0484] A method according to one embodiment may include the step of obtaining subjective answer data of a student for a plurality of subjective questions.

[0485] A method according to one embodiment may include a step of identifying a supplementary education course step based on multiple-choice answer data and subjective answer data.

[0486] A method according to one embodiment may include the step of calculating partial scores for subjective answer data based on comparing the correct answers to a plurality of subjective questions with the sentence structure, word order, tense, and vocabulary included in the subjective answer data using an artificial intelligence model.

[0487] A method according to one embodiment may include the step of identifying supplementary education course stages based on multiple-choice answer data and partial scores.

[0488] A method according to one embodiment may include the step of calculating semantic similarity regarding the contextual meaning between correct answers to a plurality of subjective questions and subjective answer data using an artificial intelligence model.

[0489] A method according to one embodiment may include the step of calculating partial scores for subjective answer data based on semantic similarity.

[0490] A method according to one embodiment may include the step of obtaining partial scores for subjective answer data for at least one subjective question belonging to the same educational course stage as the educational course stage corresponding to the objective question judged as the correct answer among the objective answer data.

[0491] A method according to one embodiment may include a step of calculating an overall achievement level for each stage of the educational process by applying weights to the density of incorrect answers based on partial scores.

[0492] A method according to one embodiment may include a step of identifying a supplementary education course stage based on the overall achievement level of each education course stage.

[0493] A method according to one embodiment may include the step of generating a plurality of sets of evaluation questions included in the scope of education of a supplementary education course stage for evaluating whether a student has correctly learned the learning material data.

[0494] A method according to one embodiment may include the step of obtaining test answer data of a student for a plurality of questions included in a plurality of sets of evaluation questions.

[0495] A method according to one embodiment may include a step of determining the learning achievement of a student in a plurality of evaluation areas based on test answer data.

[0496] A plurality of evaluation question sets according to one embodiment may consist of a plurality of multiple-choice questions and a plurality of subjective questions.

[0497] A plurality of evaluation question sets according to one embodiment may include a plurality of multiple-choice questions and a plurality of subjective questions in which unique data is changed.

[0498] The unique data according to one embodiment may include at least one of a proper noun, a numerical value, a personal pronoun, or a tense included in a plurality of multiple-choice questions and a plurality of subjective questions.

[0499] A method according to one embodiment may include the step of obtaining diagnostic scores for a plurality of evaluation areas based on objective answer data and subjective answer data.

[0500] A method according to one embodiment may include the step of obtaining a supplementary achievement score for each of the multiple evaluation areas by comparing test answer data for a plurality of items included in an evaluation item set with the correct answers for a plurality of items for each of the multiple evaluation areas.

[0501] A method according to one embodiment may include the step of calculating a skill improvement value for each of the multiple evaluation areas by comparing a diagnostic score and a supplementary achievement score for each of the multiple evaluation areas.

[0502] A method according to one embodiment may include a step of determining learning achievement levels for each of a plurality of evaluation areas based on skill improvement figures.

[0503] A method according to one embodiment of the present disclosure may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0504] Additionally, the method according to the disclosed embodiments may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0505] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) that is electronically distributed through a manufacturer of an electronic device or an electronic market (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0506] A computer program product may include a storage medium of a server or a storage medium of a client device in a system composed of a server and a client device. Alternatively, if a third device (e.g., a smartphone) is communicationally connected to the server or client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the S / W program itself, which is transmitted from the server to the client device or the third device, or from the third device to the client device.

[0507] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0508] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) can execute a computer program product stored on the server to control a client device connected to the server in communication to perform the method according to the disclosed embodiments.

[0509] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.

Claims

Claim 1 A system for managing the learning of a student comprises: a memory for storing a program or at least one instruction; and at least one processor for executing at least one instruction stored in the memory.The system comprises, wherein the at least one processor executes at least one instruction stored in the memory, thereby arranging a plurality of multiple-choice questions for each of a plurality of evaluation areas, including a listening area, a vocabulary area, a grammar area, and a reading area, from a lower course level to a higher level; outputting the plurality of multiple-choice questions for each of the plurality of evaluation areas; obtaining the objective answer data of the student for the plurality of multiple-choice questions; obtaining the error density for each of the plurality of evaluation areas for each of the course level based on comparing the objective answer data with the correct answer for the plurality of multiple-choice questions for each of the plurality of evaluation areas; identifying a question section among the plurality of multiple-choice questions where the error density exceeds a threshold value, thereby identifying at least one course level for each of the plurality of evaluation areas where the error density exceeds the threshold value; obtaining partial scores for subjective answer data for at least one subjective question belonging to the same course level as the course level corresponding to the objective question judged as the correct answer among the objective answer data; and A system that calculates an overall achievement level for each educational course stage by applying weights to the error density for each educational course stage based on partial scores, determines one of the at least one educational course stage as a supplementary educational course stage based on the overall achievement level, obtains learning material data corresponding to the supplementary educational course stage from at least one of a database or memory, schedules the learning material data according to a learning schedule, outputs the scheduled learning material data, and wherein the error density is the ratio of the number of questions entered as incorrect answers by the student to the total number of questions included in the question group corresponding to the same educational course stage. Claim 2 delete Claim 3 In claim 1, the system, by executing at least one instruction stored in the memory by the at least one processor, measures a first response time from the point in time when a plurality of multiple-choice questions corresponding to a first educational course stage are output on a display to the point in time when multiple-choice answer data for the plurality of multiple-choice questions corresponding to the first educational course stage is obtained, measures a second response time from the point in time when a plurality of multiple-choice questions corresponding to a second educational course stage are output on the display to the point in time when multiple-choice answer data for the plurality of multiple-choice questions corresponding to the second educational course stage is obtained, and if the first response time is smaller than the second response time, identifies the first educational course stage as the supplementary educational course stage, and the first educational course stage is an educational course stage of a higher level than the second educational course stage. Claim 4 In paragraph 3, the multiple-choice answer data further includes data on the number of times the student has modified the answer before finally confirming the answer to the plurality of multiple-choice questions, and by the at least one processor executing at least one instruction stored in the memory, the system identifies the educational course stage containing the multiple-choice question for which the number of answer modifications exceeds the reference number as the supplementary educational course stage when the number of answer modifications exceeds the reference number by comparing the number of answer modifications with a reference number based on the multiple-choice answer data. Claim 5 In claim 4, the system, by executing at least one instruction stored in the memory, arranges a plurality of subjective questions for evaluating the writing ability and translation ability of the student according to the educational course stages from a lower stage to a higher stage, outputs the plurality of subjective questions according to the plurality of evaluation areas, obtains the student's subjective answer data for the plurality of subjective questions, and identifies the supplementary educational course stage based on the objective answer data and the subjective answer data. Claim 6 In claim 5, the system, by having at least one processor execute at least one instruction stored in the memory, calculates a partial score for the subjective answer data based on comparing the correct answer to the plurality of subjective questions with the sentence structure, word order, tense, and vocabulary included in the subjective answer data using an artificial intelligence model, and identifies the supplementary education course stage based on the objective answer data and the partial score. Claim 7 A system according to claim 6, wherein the system, by executing at least one instruction stored in the memory by the at least one processor, calculates a semantic similarity regarding the contextual meaning between the correct answer to the plurality of subjective questions and the subjective answer data using the artificial intelligence model, and calculates a partial score for the subjective answer data based on the semantic similarity. Claim 8 delete Claim 9 In claim 7, the system, by executing at least one instruction stored in the memory by the at least one processor, generates a plurality of sets of evaluation questions included in the scope of education of the supplementary education course stage for evaluating whether the student has correctly learned the learning material data, obtains test answer data of the student for a plurality of questions included in the plurality of sets of evaluation questions, determines the learning achievement of the student for each of the plurality of evaluation areas based on the test answer data, wherein the plurality of sets of evaluation questions are composed of a plurality of multiple-choice questions and a plurality of subjective questions, and the plurality of sets of evaluation questions include the plurality of questions in which unique data within the plurality of multiple-choice questions and the plurality of subjective questions has been changed, and the unique data includes at least one of a proper noun, a numerical value, a personal pronoun, or a tense included in the plurality of multiple-choice questions and the plurality of subjective questions. Claim 10 In claim 9, the system, by executing at least one instruction stored in the memory by the at least one processor, obtains a diagnostic score for each of the plurality of evaluation areas based on the objective answer data and the subjective answer data, obtains a supplementary achievement score for each of the plurality of evaluation areas by comparing the test answer data for a plurality of items included in the set of evaluation questions for each of the plurality of evaluation areas with the correct answer for the plurality of items, calculates a skill improvement value for each of the plurality of evaluation areas by comparing the diagnostic score and the supplementary achievement score for each of the plurality of evaluation areas, and determines a learning achievement level for each of the plurality of evaluation areas based on the skill improvement value.

Images