Method and apparatus for assisting in preparing evaluation results using generative ai model

Abstract

Provided, according to one feature of the present disclosure, is a method for assisting in preparing an evaluation plan using a generative AI model executed by an evaluation assistance server. The method assists in preparing an evaluation plan using a generative AI model, the method comprising the steps of: receiving an evaluation plan generation command and generating an evaluation plan; receiving an input of scoring elements; and generating detailed scoring elements using the generative AI model on the basis of the scoring elements.

Description

Method and device for assisting in the generation of evaluation results using a generative AI model

[0001] The present disclosure relates to a method for assisting in the evaluation of descriptive problems, and more specifically, to a method and apparatus for assisting in the generation of evaluation results using a generative AI model.

[0002] Essay questions are highly effective for evaluating a student's or examinee's thinking skills, creativity, and logical organization abilities. Unlike multiple-choice questions, essay questions provide the examinee with the opportunity to freely express their thoughts and require comprehensive thinking skills rather than simple rote memorization. Due to these characteristics, essay questions play a crucial role in deeply assessing academic achievement or understanding of specific topics. Furthermore, they are useful for providing educational feedback as they allow for the identification of the examinee's thought process and problem-solving methods.

[0003] Despite the advantages of descriptive questions, the grading process is time-consuming and labor-intensive. Since evaluators must review each response individually, there is a possibility of variance due to subjective judgment, and it can be difficult to grade a large number of responses consistently. Furthermore, defining the range of correct answers is complex because different responses may exist for the same question. While standardized criteria are necessary to reduce variance among graders, strictly applying such standards in practice is difficult. Consequently, grading descriptive questions can be inefficient and has limitations in guaranteeing fairness.

[0004] The present disclosure aims to reduce evaluation time and subjective variation and ensure consistent evaluation when evaluating descriptive questions.

[0005] The present disclosure aims to help maintain consistency and fairness in evaluation when multiple answers exist within the range of correct answers in the evaluation of descriptive questions.

[0006] According to one feature of the present disclosure, a method is provided for assisting in the generation of evaluation results using a generative AI model executed on an evaluation assistance server. The method comprises the steps of: receiving a predetermined evaluation design; receiving at least one assignment result, wherein the predetermined evaluation design includes subject name and grade information and grading criteria; and evaluating the assignment result using a generative AI model based on the grading criteria to generate a grading result.

[0007] According to one embodiment of the present disclosure, the step of generating an evaluation result by evaluating a task result using a generative AI model based on the scoring criteria comprises: the scoring criteria including at least one scoring element, at least one point corresponding to the at least one scoring element, and at least one detailed scoring criterion associated with the at least one point; and further comprises the step of generating a scoring result by evaluating a task result using an AI model based on the detailed scoring criteria.

[0008] According to one embodiment of the present disclosure, the scoring result includes at least one of a scoring basis, a scoring score, feedback, or a relevance score for at least one scoring element.

[0009] According to one embodiment of the present disclosure, the scoring basis is generated by citing the content described in the assignment result, the scoring score is generated based on the points associated with the detailed scoring elements, the feedback is an evaluation opinion provided to the person who submitted the assignment result, generated by considering the level of the person who submitted the assignment result based on the grade information, and the relevance score may be the similarity between the assignment result and a predetermined scoring criterion.

[0010] According to one embodiment of the present disclosure, the method further comprises the steps of: receiving achievement criteria of a predetermined evaluation plan—wherein the achievement criteria are defined as predetermined competency data, and the predetermined competency data are defined based on associated class information; and generating an achievement result by evaluating a task result using a generative AI model based on the achievement criteria.

[0011] According to one embodiment of the present disclosure, the step of generating an achievement result by evaluating a task result using a generative AI model based on the achievement standard further includes the step of generating an achievement result by evaluating a task result using an AI model based on predetermined competency data defined in the subject of the class information.

[0012] According to one embodiment of the present disclosure, the achievement result includes at least one of a part in which capability was discovered, a reason for determining that capability exists, a discovered capability, and a capability record. The part in which capability was discovered is generated based on the content described in the task result by evaluating whether there is a part in the task result that matches the capability data. The discovered capability is one of the predetermined capability data. The capability record may be generated based on the content of the part in which capability was discovered, the reason for determining that capability exists, and the discovered capability.

[0013] According to one embodiment of the present disclosure, the achievement standard includes a grade and a detailed achievement standard associated with the grade, and further includes the step of generating an achievement result by evaluating a task result using a generative AI model based on the detailed achievement standard.

[0014] According to another feature of the present disclosure, a recording medium is provided for executing on a computing device a method for assisting in the creation of evaluation results using a generative AI model executed by a computing device, wherein any one of the aforementioned methods is executed on the computing device.

[0015] According to another feature of the present disclosure, an apparatus is provided to assist in generating evaluation results configured to execute any one of the methods described above using a generative AI model.

[0016] A method for assisting in the creation of a comprehensive record using a generative AI model is provided. The method comprises the steps of: receiving a predetermined evaluation plan—the predetermined evaluation plan includes at least one of a subject name and grade information, an achievement standard, or a scoring standard—; receiving an evaluation result of a task submitted by a predetermined student regarding the predetermined evaluation plan; and generating an evaluation record corresponding to the evaluation result.

[0017] According to one embodiment of the present disclosure, the step of receiving an evaluation result of an assignment result submitted by a predetermined student for a predetermined evaluation plan further includes the step of receiving a grading result of an assignment result submitted by the predetermined student, wherein the grading result includes at least one of a grading score, a grading basis, feedback, or a relevance score obtained by evaluating the assignment result according to at least one grading element; and the step of generating an evaluation record corresponding to the evaluation result further includes the step of generating an evaluation record using a generative AI model based on a detailed grading element, a grading basis, and feedback related to the grading score in the grading element.

[0018] According to one embodiment of the present disclosure, the step of generating an evaluation record using a generative AI model based on detailed scoring elements, scoring grounds, and feedback related to scoring scores in the scoring elements is performed for all scoring elements included in the evaluation plan; and further comprises the step of generating the grounds for generating the evaluation record.

[0019] According to one embodiment of the present disclosure, the step of receiving an evaluation result of a task result submitted by a predetermined student for the predetermined evaluation plan further includes the step of receiving an achievement result of a task result submitted by the predetermined student—the achievement result includes a competency record—; and the step of generating an evaluation record corresponding to the evaluation result further includes the step of generating an evaluation record using a generative AI model based on the competency record.

[0020] According to one embodiment of the present disclosure, the step of generating an evaluation record corresponding to the evaluation result further includes the step of generating an improvement plan when generating the evaluation record if the scoring score is below a predetermined standard.

[0021] According to one embodiment of the present disclosure, the method further includes the step of integrating and summarizing evaluation records generated for each of the predetermined evaluation plans to generate a comprehensive record.

[0022] According to one embodiment of the present disclosure, the method further includes the step of highlighting a portion of at least one evaluation record by evaluation plan cited when generating the comprehensive record.

[0023] According to one embodiment of the present disclosure, the step of generating the comprehensive record further includes a student observation record.

[0024] According to another feature of the present disclosure, a computer-readable recording medium is provided for executing on a computing device a method of assisting in the creation of a comprehensive record using a generative AI model executed by a computing device, wherein any one of the aforementioned methods is executed on a computer.

[0025] According to another feature of the present disclosure, an apparatus for assisting in the creation of a comprehensive record using a generative AI model is provided, wherein the apparatus for assisting in the creation of a comprehensive record using a generative AI model configured to execute any one of the methods described above is provided.

[0026] According to one feature of the present disclosure, a method is provided for assisting in the creation of an evaluation plan using a generative AI model executed on an evaluation assistance server. The method comprises the steps of: receiving an evaluation plan creation command and creating an evaluation plan; receiving scoring elements as input; and creating detailed scoring elements using a generative AI model based on the scoring elements.

[0027] According to one embodiment of the present disclosure, the method further comprises the steps of: receiving an achievement standard; and generating a detailed scoring element using a generative AI model based on the scoring element and the achievement standard.

[0028] According to one embodiment of the present disclosure, the method further comprises the step of receiving a subject name; and the step of storing the evaluation plan in association with the subject name, wherein the evaluation plan includes at least one of the subject name, the achievement standard, or the scoring element.

[0029] According to one embodiment of the present disclosure, the method further comprises the step of receiving a predetermined grade, and the grade is stored in association with the detailed scoring element.

[0030] According to one embodiment of the present disclosure, the method further comprises the step of receiving an evaluation mode; and, if the evaluation mode is an analytical evaluation mode, the step of generating a detailed scoring element that determines the task results by area and assigns a score according to the criteria for each scoring element.

[0031] According to one embodiment of the present disclosure, the method comprises the step of receiving an evaluation mode; and

[0032] If the above evaluation mode is a comprehensive evaluation mode, it further includes the step of generating detailed scoring elements that assign scores based on whether each scoring element is satisfied.

[0033] According to one embodiment of the present disclosure, the method further includes the step of generating a deep detailed scoring element based on a detailed scoring element associated with the scoring element.

[0034] According to another feature of the present disclosure, a computer-readable recording medium is provided for executing a relation extraction method executed by a computing device on a computing device, wherein any one of the above-described methods is executed on a computer.

[0035] According to another feature of the present disclosure, an apparatus for assisting in the creation of an evaluation plan using a generative AI model is provided, wherein the apparatus for assisting in the creation of an evaluation plan using a generative AI model configured to execute any one of the methods described above is provided.

[0036] According to an embodiment of the present disclosure, time and effort can be saved by utilizing generative AI to assist in the evaluation of descriptive problems. The speed and efficiency of evaluation can be improved through the automatic evaluation and analysis functions of generative AI.

[0037] According to an embodiment of the present disclosure, an unbiased, fair, and consistent evaluation can be provided. By applying consistent scoring criteria, an unbiased, fair, and objective evaluation can be provided. Through this, learners can increase the reliability of the evaluation results and continuously develop their thinking skills and logical construction abilities through feedback.

[0038] FIG. 1 is a schematic diagram illustrating an evaluation assistance system (100) according to one embodiment of the present disclosure.

[0039] FIG. 2 is a functional block diagram schematically illustrating the functional configuration of a user terminal (110) illustrated in FIG. 1 according to one embodiment of the present disclosure.

[0040] FIG. 3 is a functional block diagram schematically illustrating the functional configuration of the evaluation aid device (130) shown in FIG. 1 according to one embodiment of the present disclosure.

[0041] FIG. 4 is a functional block diagram schematically illustrating the functional configuration of the evaluation plan module illustrated in FIG. 3 according to one embodiment of the present disclosure.

[0042] FIG. 5 is a drawing illustrating, in accordance with one embodiment of the present disclosure, evaluation plan data stored in an evaluation plan database in an exemplary manner, and FIG. 6 is a drawing illustrating, in an exemplary manner, a screen on which evaluation plan data stored in an evaluation plan database is displayed to a user.

[0043] FIG. 7 is an exemplary screen in which, according to one embodiment of the present disclosure, a scoring criteria generation AI module generates detailed scoring elements of a scoring element based on a scoring criteria (e.g., a scoring element) and / or an achievement standard.

[0044] FIG. 8 is an exemplary screen in which, according to one embodiment of the present disclosure, a scoring criteria generation AI module generates detailed scoring elements based on scoring criteria (scoring elements) or achievement criteria according to a predetermined evaluation mode.

[0045] FIG. 9 is an operation flowchart conceptually illustrating the process of an evaluation planning module generating detailed scoring elements of a scoring element according to one embodiment of the present disclosure.

[0046] FIG. 10 is a functional block diagram schematically illustrating the functional configuration of an evaluation result generating AI module illustrated in FIG. 3 according to one embodiment of the present disclosure.

[0047] FIG. 11 is a drawing illustrating an exemplary performance evaluation plan according to one embodiment of the present disclosure, and FIG. 12 is a drawing illustrating an exemplary scoring result generated by evaluating a task result according to the evaluation plan in an AI module that generates evaluation results according to scoring criteria.

[0048] FIG. 13 is an operation flowchart conceptually showing the process of an evaluation result generating AI module generating an evaluation result according to one embodiment of the present disclosure.

[0049] FIG. 14 is a drawing illustrating, in accordance with one embodiment of the present disclosure, an exemplary comprehensive record generated by a comprehensive record generation AI module for a predetermined student.

[0050] FIG. 15 is an operation flowchart conceptually showing the process of an evaluation result generating AI module generating an evaluation result according to one embodiment of the present disclosure.

[0051] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following, specific descriptions of already known functions and configurations are omitted where it is deemed that doing so would unnecessarily obscure the essence of the present disclosure. Furthermore, it should be understood that the content described below relates only to one embodiment of the present disclosure and that the present disclosure is not limited thereto.

[0052] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure. For example, a component expressed in the singular should be understood as a concept including a plural component unless the context clearly implies only the singular. It should be understood that the term "and / or" used in this disclosure encompasses any possible combination of one or more of the items listed. Terms such as "comprising" or "having" used in this disclosure are intended only to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and the use of such terms is not intended to exclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0053] In the embodiments of the present disclosure, a 'module' or 'part' refers to a functional part that performs at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or 'parts' may be integrated into at least one software module and implemented by at least one processor, except for 'modules' or 'parts' that need to be implemented in specific hardware.

[0054] Additionally, unless otherwise defined, all terms used in this disclosure, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted as being overly limited or expanded unless explicitly defined otherwise in this disclosure.

[0055] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0056] FIG. 1 is a schematic diagram illustrating an evaluation assistance system (100) according to one embodiment of the present disclosure.

[0057] According to one embodiment of the present disclosure, an evaluation assistance system (100) may include a plurality of user terminals (110), a communication network (120), and an evaluation assistance device (130).

[0058] According to one embodiment of the present disclosure, each of the plurality of user terminals (110) may be any user electronic device equipped with wired or wireless communication functions. It should be noted that each of the user terminals (110) may be various wired or wireless communication terminals, such as smartphones, tablet PCs, desktops, laptops, PDAs, digital TVs, etc., and is not limited to a specific form. In this drawing, only three user terminals are shown, but the present disclosure is not limited thereto. According to another embodiment of the present disclosure, it should be noted that the descriptive problem evaluation assistance system (100) may include a larger number of various types of user terminals. According to one embodiment of the present disclosure, a user may log in to the evaluation assistance device (130) using one or more user terminals (110) and obtain a desired service from the evaluation assistance device (130).

[0059] According to one embodiment of the present disclosure, each user terminal (110) can communicate with an evaluation aid device (130) through a communication network (120), that is, transmit and receive necessary information. According to one embodiment of the present disclosure, the user terminal (110) can receive various forms of user input from the outside, such as voice and / or text input, image and / or video input, touch input, and various other control inputs. According to one embodiment of the present disclosure, various forms of user input received on the user terminal (110) can be transmitted to the evaluation aid device (130) through the communication network (120). According to one embodiment of the present disclosure, the user terminal (110) can receive various signals transmitted from the outside (e.g., evaluation aid device (130)) through the communication network (120).

[0060] According to one embodiment of the present disclosure, the communication network (120) may include any wired or wireless communication network, such as a TCP / IP communication network. According to one embodiment of the present disclosure, the communication network (120) may include, for example, a Wi-Fi network, a LAN network, a WAN network, an Internet network, etc., but the present disclosure is not limited thereto. According to one embodiment of the present disclosure, the communication network (120) may be implemented using, for example, Ethernet, GSM, EDGE (Enhanced Data GSM Environment), CDMA, TDMA, OFDM, OFDMA, 3G, 4G, 5G, 6G, nG, Bluetooth, VoIP, Wi-MAX, Wibro, or any other various wired or wireless communication protocols.

[0061] According to one embodiment of the present disclosure, the evaluation assistance device (130) can communicate with a user terminal (110) through a communication network (120). According to one embodiment of the present disclosure, the evaluation assistance device (130) can evaluate a task result and generate an evaluation result and an evaluation basis in response to a request received from a user terminal (110) through, for example, the communication network (120), and generate a comprehensive record based thereon.

[0062] In one embodiment, the evaluation aid device (130) may be a database that stores and manages data for each user account, a database that stores and manages data for each of one or more classes, or a database that stores and manages data for each task associated with a predetermined evaluation plan. In one embodiment, the evaluation aid device (130) may store information regarding the evaluation plan and generate the evaluation plan, and may generate detailed scoring elements based on scoring elements or achievement criteria according to a predetermined evaluation mode. In one embodiment, the evaluation aid device (130) may evaluate a predetermined task result and / or generate evaluation results and / or evaluation grounds. In one embodiment, the evaluation aid device (130) may generate an evaluation record regarding at least one evaluation conducted in a predetermined subject and generate a comprehensive record based on the evaluation record regarding individual evaluations.

[0063]

[0064] FIG. 2 is a functional block diagram schematically illustrating the functional configuration of a user terminal (110) illustrated in FIG. 1 according to one embodiment of the present disclosure.

[0065] As described above, the user terminal (110) may include a user input receiving module (202), a program memory module (204), a processing module (206), a communication module (208), and an output module (210).

[0066] According to one embodiment of the present disclosure, the user input receiving module (202) may receive various forms of input from a user, such as voice and / or text input, touch input, and various other control inputs. According to one embodiment of the present disclosure, the user input receiving module (202) may include various types of input devices, such as various pointing devices like a mouse, joystick, and trackball, a camera, a keyboard, a microphone and audio circuit, a touch panel, a touchscreen, a stylus, and various input interface buttons, and may acquire input signals input by the user through these input devices. According to one embodiment of the present disclosure, the user input received by the user input receiving module (202) may be associated with a control command requesting the evaluation assistance device (130) to perform a predetermined operation.

[0067] According to one embodiment of the present disclosure, the program memory module (204) may be any storage medium in which various programs that can be executed on a user terminal (110), such as various application programs and related data, are stored. According to one embodiment of the present disclosure, the program memory module (204) may store various application programs, such as a messenger service provision application, a camera application, a clock application, etc., through communication with an evaluation aid device (130), and data related to the execution of these programs. According to one embodiment of the present disclosure, the program memory module (204) may be configured to include various forms of volatile or non-volatile memory, such as DRAM, SRAM, DDR RAM, ROM, magnetic disk, optical disk, flash memory, etc.

[0068] According to one embodiment of the present disclosure, the processing module (206) can communicate with each component module of the user terminal (110) and perform various operations. According to one embodiment of the present disclosure, the processing module (206) can drive and execute various application programs on the program memory module (204). According to one embodiment of the present disclosure, the processing module (206) can, if necessary, receive signals obtained from the user input receiving module (202) and the sensor module (204) and perform appropriate processing regarding these signals. According to one embodiment of the present disclosure, the processing module (206) can, if necessary, perform appropriate processing on signals received from the outside through the communication module (208).

[0069] According to one embodiment of the present disclosure, the communication module (208) enables the user terminal (110) to communicate with the evaluation aid device (130) through the communication network (120) of FIG. 1. According to one embodiment of the present disclosure, the communication module (208) may enable signals obtained on, for example, a user input receiving module (202) and a sensor module (204) to be transmitted to the evaluation aid device (130) through the communication network (120) according to a predetermined protocol. According to one embodiment of the present disclosure, the communication module (208) may receive various signals received from the evaluation aid device (130) through the communication network (120), for example, user messages in the form of voice and / or text, or various control signals, and perform appropriate processing according to a predetermined protocol.

[0070] According to one embodiment of the present disclosure, the output module (210) may output various forms of signals, such as visual, auditory, and / or tactile signals, corresponding to user input received on a user terminal (110) and signals received from the outside through a communication network (120) and a communication module (208). According to one embodiment of the present disclosure, the output module (210) may include various display devices, such as a touch screen based on technologies such as LCD, LED, OLED, QLED, etc., and may present various visual output signals, such as text, symbols, videos, images, hyperlinks, animations, various notices, etc., to the user through these display devices. According to one embodiment of the present disclosure, the output module (210) may include, for example, a speaker or a headset, and may provide various auditory output signals, such as voice and / or sound signals, to the user through the same.

[0071] According to one embodiment of the present disclosure, the output module (210) can visualize and display various information and data on the user terminal (110). According to one embodiment of the present disclosure, the output module (210) can visualize and display various statistical data based thereon and result records performed through the evaluation aid device (130) described below on the user terminal (110).

[0072] FIG. 3 is a functional block diagram schematically illustrating the functional configuration of the evaluation aid device (130) shown in FIG. 1 according to one embodiment of the present disclosure.

[0073] As described above, the evaluation aid device (130) may include, but is not limited to, a communication module (302), a user information management module (304), a motion control module (306), a class management module (308), an evaluation planning module (310), a task management module (312), an evaluation result generation AI module (314), or a comprehensive record generation AI module (316).

[0074] According to one embodiment of the present disclosure, a communication module (302) enables an evaluation aid device (130) to communicate with a user terminal (110) and other external devices (not shown) through a communication network (120) according to a predetermined wired or wireless communication protocol. According to one embodiment of the present disclosure, the communication module (302) can receive user input transmitted from the user terminal (110) through the communication network (120). According to one embodiment of the present disclosure, the communication module (302) can receive status information of the user terminal (110) transmitted from the user terminal (110), etc., through the communication network (120), together with or separately from the aforementioned user input. According to one embodiment of the present disclosure, the state information may be various state information related to, for example, a user terminal (110) (e.g., the physical state of the user terminal (110), the software and / or hardware state of the user terminal (110), the environmental state information surrounding the user terminal (110), etc.). According to one embodiment of the present disclosure, the communication module (302) may also perform appropriate measures necessary to transmit a predetermined message signal and / or control signal generated on the evaluation aid device (130) or received from another user terminal (110) to the user terminal (110) through the communication network (120).

[0075] According to one embodiment of the present disclosure, the user information management module (304) may be a database that stores and manages each user account data. According to one embodiment of the present disclosure, each user account may be associated with one or more user terminals (110). According to one embodiment of the present disclosure, each user account data included in the user information management module (304) may include various information such as user registration information for each user account (e.g., name, phone number, email address, ID, password, staff authority, user authority, subject, school information, academic year, unique key, various user profile information, user account associated device information, and other various information, but is not limited thereto), registration date, user login information, etc. According to one embodiment of the present disclosure, the user information management module (304) may continuously collect and update various history information, including activities for each user account through the evaluation assistance device (130), such as various service requests provided by the evaluation assistance device (130) and the reception of responses thereto.

[0076] According to one embodiment of the present disclosure, the operation control module (306) can perform various necessary operations and operation processing in conjunction with each other module of the evaluation aid device (130). According to one embodiment of the present disclosure, the operation control module (306) can receive data transmitted from each user terminal (110), for example, through a communication module (302). According to one embodiment of the present disclosure, the operation control module (306) can also perform necessary predetermined operation processing, recording and updating of predetermined information, and generating and transmitting predetermined messages in conjunction with the class management module (308), evaluation planning module (310), assignment management module (312), evaluation result generation AI module (314), and comprehensive record generation AI module (316) described later.

[0077] According to one embodiment of the present disclosure, the operation control module (306) may receive an explicit or implicit request from, for example, a user terminal (110). According to one embodiment of the present disclosure, the operation control module (306) may interpret the received request by referring to the class management module (308), the evaluation planning module (310), the assignment management module (312), the evaluation result generation AI module (314), and the comprehensive record generation AI module (316), and may perform an operation corresponding thereto, such as monitoring a predetermined user activity (e.g., obtaining and recording predetermined information regarding a predetermined user activity), and may generate and transmit a response corresponding thereto. According to one embodiment of the present disclosure, the operation control module (306) may record and / or update the information and / or response obtained, recorded, and generated above in relation to the corresponding evaluation on the evaluation planning module (310), the assignment management module (312), the evaluation result generation AI module (314), and the comprehensive record generation AI module (316). According to one embodiment of the present disclosure, the operation control module (306) can generate certain statistical information based on various data, such as accumulated monitoring records regarding each user activity, evaluation results, and evaluation grounds, in response to a request from a user terminal (110), on records collected and recorded on the class management module (308), evaluation planning module (310), task management module (312), evaluation result generation AI module (314), and comprehensive record generation AI module (316).

[0078] According to one embodiment of the present disclosure, the class management module (308) may be a database that stores and manages data regarding one or more classes. According to one embodiment of the present disclosure, each class data included in the class management module (308) may include, for example, information such as attribute (feature) information determined for each class, semester, grade, subject, class name, etc., but is not limited thereto. According to one embodiment of the present disclosure, each class data included in the class management module (308) may include, for example, student information by class unit or student information by individual student unit to participate in the class.

[0079] According to one embodiment of the present disclosure, student information included in the class management module (308) may include observation records of students who participated in the class. In one embodiment, the student observation records may be recorded as keywords. For example, for a specific student, words (keywords) such as 'questioning activity', 'liveliness', 'excellent concentration', 'active', 'participation', 'communication', 'consideration', 'respect', 'cooperation', 'creative ideas', 'collaboration', 'acceptance of feedback', 'time management' may be entered. According to one embodiment of the present disclosure, the aforementioned words (keywords) may be stored in the class management module (308). In another embodiment, the aforementioned words (keywords) may be edited and stored in the class management module (308) in response to a request from the user terminal (110).

[0080] According to another embodiment of the present disclosure, student observation records may be recorded in the form of phrases or sentences. For example, they may be entered in the form of sentences or phrases, such as 'logically presenting opinions during discussion time,' 'listening attentively to explanations,' or 'taking on a leader role in a team project to gather and coordinate the opinions of team members.'

[0081] According to one embodiment of the present disclosure, the evaluation plan module (310) is a module that stores information regarding an evaluation plan and generates an evaluation plan.

[0082] FIG. 4 is a functional block diagram schematically illustrating the functional configuration of the evaluation plan module illustrated in FIG. 3 according to one embodiment of the present disclosure.

[0083] According to one embodiment of the present disclosure, the evaluation plan module (310) may include an evaluation plan database (402) that stores information regarding the evaluation plan and a scoring criteria generation AI module (404) that generates scoring criteria for the evaluation.

[0084] In one embodiment, the evaluation plan database (402) may be a database that stores and manages information such as a subject name, grade, semester, evaluation name, at least one achievement standard, a judgment grade of a predetermined achievement standard (e.g., high / medium / low), a scoring standard, at least one scoring element corresponding to a predetermined scoring standard, at least one point associated with a predetermined scoring element, and a detailed scoring element associated with at least one point, for each evaluation plan. Here, the scoring standard or scoring element may be defined in various ways according to user input.

[0085] FIG. 5 is a drawing illustrating, in accordance with one embodiment of the present disclosure, evaluation plan data stored in an evaluation plan database in an exemplary manner, and FIG. 6 is a drawing illustrating, in an exemplary manner, a screen on which evaluation plan data stored in an evaluation plan database is displayed to a user.

[0086] In one embodiment, items of evaluation plan data stored in the evaluation plan database (402) may be stored in association with each other. At least one evaluation plan may be stored in the evaluation plan database (402) in association with a predetermined subject name. For example, if the subject name is 'Social Studies', at least one evaluation plan may be stored in the evaluation plan database (402) in association with the subject name ('Social Studies'). Here, the evaluation plan may be stored including items related to evaluation, such as an evaluation (plan) name, at least one achievement standard, a grading standard, and at least one grading element associated with the grading standard. For example, if the subject name is 'Social Studies', an evaluation plan ('Evaluation Name: Creating Future Society Scenarios') may be stored in the evaluation plan database (402) in association with the subject name ('Social Studies').

[0087] In one embodiment, the evaluation plan database (402) may store a grade for judging at least one achievement standard associated with a predetermined achievement standard for a predetermined evaluation plan. For example, if the achievement standard is 'understand the meaning of social change and analyze the patterns of change and problems of modern society,' the grade for judging the achievement standard in the evaluation plan database (402) may be 'High' as ​​'understand the meaning of social change and can analyze the patterns of change and problems of modern society through examples,' 'Medium' as 'understand the meaning of social change and can explain the patterns of change and problems of modern society,' and 'Low' as 'can present the patterns of change of modern society.' Thus, the achievement standard and the grade for judging the achievement standard may be stored in association in the evaluation plan database (402).

[0088] In one embodiment, the evaluation plan database (402) may store at least one scoring element associated with a scoring criterion for a predetermined evaluation plan, at least one point associated with a predetermined scoring element, and at least one detailed scoring element associated with a point. For example, the evaluation plan database (402) may store a scoring element ('exploration of past and present appearances'), a point (20, 16) associated with a scoring criterion for a predetermined evaluation plan (evaluation name: social change), and detailed scoring elements (point (20), detailed scoring element ('concretely expressing past and present appearances based on various materials such as articles, statistics, etc.'), (point (16), detailed scoring element ('expression of past and present appearances based on related materials is somewhat lacking in specificity') associated with each point.

[0089] In one embodiment of the present disclosure, the scoring criteria generation AI module (404) can generate detailed scoring elements based on scoring elements or achievement criteria according to the points.

[0090] FIG. 7 is an exemplary screen in which, according to one embodiment of the present disclosure, a scoring criteria generation AI module generates detailed scoring elements of a scoring element based on a scoring criteria (e.g., a scoring element) and / or an achievement standard.

[0091] In one embodiment, the scoring criteria generation AI module (404) receives a scoring element ('whether the source of the data was indicated') and an achievement standard ('investigate major social issues of modern times and explore solutions thereto') and can generate a detailed scoring element according to each point of the scoring element. For example, the scoring criteria generation AI module (404) receives the aforementioned scoring element and achievement standard and can generate a point (10), a detailed scoring element ('investigated major social issues of modern times and indicated the source of the data'), (point (5), a detailed scoring element ('mentioned social issues of modern times but did not indicate the source of the data'), (point (0), and a detailed scoring element ('did not participate'), respectively.

[0092] In one embodiment of the present disclosure, the scoring criteria generation AI module (404) can generate advanced detailed scoring elements based on scoring criteria (e.g., scoring elements), detailed scoring elements and / or achievement criteria. In this way, the scoring criteria generation AI module (404) can recursively generate more detailed evaluation criteria based on the predetermined criteria generated by the scoring criteria generation AI module (404).

[0093] FIG. 8 is an exemplary screen in which a scoring criterion generation AI module generates detailed scoring elements based on scoring criteria (scoring elements) or achievement criteria according to a predetermined evaluation mode, in accordance with one embodiment of the present disclosure. In one embodiment, the predetermined mode may be an analytical evaluation mode or a holistic evaluation mode.

[0094] In one embodiment, the scoring criteria generation AI module (404) can generate detailed scoring elements that can assign scores according to criteria for each scoring element by judging the task results by area when in the analytical evaluation mode (802). Here, the analytical evaluation mode is a method of judging the task results by area and assigning scores based on whether the detailed scoring elements for each scoring element are satisfied.

[0095] For example, a specified scoring element may be 'understanding graphs' or 'understanding quadratic equations'. The points associated with the scoring element 'understanding graphs' may be '8', '6', or '4', and the detailed scoring element associated with the point '8' may be 'being able to draw various types of graphs such as bar graphs, line graphs, and pie charts, and explaining the characteristics of each graph in one's own words', the detailed scoring element associated with the point '6' may be 'representing a table as a graph and analyzing the relationship between two graphs', and the detailed scoring element associated with the point '4' may be 'being able to represent data as a graph and explain the pattern of change only in relatively simple cases'. The points associated with the grading element 'Understanding Quadratic Equations' are '5' or '3', and the detailed grading element associated with points '5' is 'ability to solve quadratic equations using various methods and explain the process', while the detailed grading element associated with points '3' may be 'ability to set up a simple quadratic equation that fits the meaning of the problem'.

[0096] In another embodiment, the scoring criteria generation AI module (404) can generate detailed scoring elements that can assign scores based on whether each scoring element is satisfied when in the holistic evaluation mode (804). Here, the holistic evaluation mode is a method of holistic evaluation in which a task result is judged comprehensively and scores are assigned based on whether each scoring element is satisfied. For example, when the scoring criteria generation AI module (404) is in the holistic evaluation mode, for a scoring element (exploration of various economic cases), it can generate detailed scoring elements: points (2) ('concretely discovering and presenting real-world economic phenomena by utilizing various information resources'), points (0) ('needs to supplement conceptual understanding of economic phenomena'). The points associated with the scoring element ('Presentation of content related to the topic') are '2' or '0'; the detailed scoring element associated with points '2' is 'Describing content related to the topic in detail using specific examples', and the detailed scoring element associated with points '0' may be 'Content related to the topic is somewhat ambiguous'. The points associated with the scoring element 'Base Score' are '2' or '0'; the detailed scoring element associated with points '2' is 'Participation in the activity', and the detailed scoring element associated with points '0' may be 'Not participating in the activity'.

[0097]

[0098] FIG. 9 is an operation flowchart conceptually illustrating the process of an evaluation planning module generating detailed scoring elements of a scoring element according to one embodiment of the present disclosure.

[0099] First, in step (S901), the evaluation plan module (310) receives an evaluation plan creation command and creates an evaluation plan based on it. This step is an initial process that defines the basic direction and structure of the evaluation and serves as the basis for the work to be performed in subsequent steps.

[0100] In step (S903), the evaluation plan module (310) may receive information regarding the evaluation plan. In one embodiment, the evaluation plan may include the subject name, class code, and information of the professor in charge, etc., to be evaluated. This information may be used as basic data necessary to establish a detailed plan for each evaluation item.

[0101] In step (S905), the evaluation planning module (310) may receive scoring elements. Scoring elements may serve as criteria for evaluating students' academic achievement. In one embodiment, the evaluation planning module (310) may further receive achievement criteria. Achievement criteria serve as criteria for evaluating whether a student possesses a certain competency. The evaluation planning module (310) may further receive a certain grade for the scoring elements or achievement criteria.

[0102] In one embodiment, the achievement standard may be a competency defined by a designated institution (e.g., the Ministry of Education). For example, the Korean Ministry of Education defines six major competencies as 'self-management competency,' 'knowledge and information processing competency,' 'creative thinking competency,' 'aesthetic sensibility competency,' 'cooperative communication competency,' and 'community competency,' and for some subjects, subject-specific competencies are defined.

[0103] In step (S907), the evaluation planning module (310) may receive an evaluation mode. The evaluation mode is a method for setting evaluation criteria. For example, it may be an analytical evaluation mode or a holistic evaluation mode. In one embodiment, if the evaluation mode is an analytical evaluation mode, the evaluation planning module (310) may generate a detailed scoring element that determines the task results by area and assigns a score according to the criteria for each scoring element, and if the evaluation mode is a holistic evaluation mode, it may generate a detailed scoring element that assigns a score based on whether each scoring element is satisfied.

[0104] In step (S909), the evaluation planning module (310) can generate detailed scoring elements using a generative AI model based on the scoring elements. In one embodiment, the generative AI model may be a multimodal language model that receives an image or text and outputs text. As an example, the generative AI model may be Llama-3 or Qwen2.

[0105] In another embodiment, the evaluation planning module (310) can generate detailed scoring elements using a generative AI model based on scoring elements and achievement standards. In yet another embodiment, the evaluation planning module (310) can generate detailed scoring elements according to grade using a generative AI model based on scoring elements, achievement standards, and grades.

[0106] In step (S911), the evaluation plan module (310) can store an evaluation plan associated with at least one of a subject name, achievement standard, grading element, grade, or detailed grading element.

[0107] The generated detailed scoring elements serve as important criteria to enhance the fairness and consistency of evaluations, and the automated process of generating detailed scoring elements through AI models can contribute to reducing evaluation preparation time and alleviating the workload of educators.

[0108] Refer again to Fig. 3.

[0109] According to one embodiment of the present disclosure, the task management module (312) may be a database that stores and manages data regarding each task associated with a predetermined evaluation plan.

[0110] In one embodiment, the assignment management module (312) may be a database that stores and manages assignment information for each assignment, such as evaluation name, assignment name, assignment description, assignment submission method, class, student, assignment results associated with the student, status (e.g., in progress / completed), assignment period, submission rate (e.g., percentage), and number of unsubmitted items (e.g., submitted items / total items).

[0111] In one embodiment, the task management data stored in the task management module (312) may be stored in association with each other. At least one task information may be stored in the task management module (312) in association with a predetermined evaluation name. For example, when the evaluation name is 'Reading and Cultural Experience Activity', the task management module (312) may store the task name ('Task for Reading and Cultural Experience Activity'), class ('1-1', '1-2'), status ('In Progress'), task period ('5 / 06 15:00 - 05 / 13 14:55'), submission rate ('61'), non-submission ('14 / 23'), achievement standard (e.g., investigate major social issues of modern times and explore solutions thereto), etc. in association with the evaluation name ('Reading and Cultural Experience Activity').

[0112] In one embodiment, the assignment management module (312) may store a submission method according to the assignment to be assigned. In one embodiment, the submission method ('PDF batch upload') is a method in which a first user (e.g., an assignment manager such as a teacher) scans and uploads assignment results in bulk. In another embodiment, the submission method ('direct submission') is a method in which at least one user (e.g., an assignment performer such as a student) scans and uploads assignment results, and for each of the multiple users, multiple assignment results may be stored associated with a specific assignment name.

[0113] In one embodiment of the present disclosure, the evaluation result generating AI module (314) may evaluate a predetermined task result and / or evaluation basis. In one embodiment, the evaluation result generating AI module (314) may evaluate the task result according to a predetermined evaluation plan and generate an evaluation score, evaluation basis, and feedback. Here, the evaluation plan may include items related to evaluation, such as an evaluation (plan) name, at least one achievement standard, a scoring standard, and at least one scoring element associated with the scoring standard.

[0114] FIG. 10 is a functional block diagram schematically illustrating the functional configuration of an evaluation result generating AI module illustrated in FIG. 3 according to one embodiment of the present disclosure.

[0115] In one embodiment, the evaluation result generating AI module (314) may include an evaluation result generating AI module (1002) based on achievement standards, an evaluation result generating AI module (1004) based on scoring standards, and an evaluation result generating AI module (1006) based on other standards.

[0116] Table 1 is a table that lists, for example, each item of the evaluation result generated by the evaluation result generating AI module (314). For example, the evaluation result generating AI module (1002) based on the achievement standard evaluates the task result according to the predetermined achievement standard and generates the achievement result. The achievement result may include, but is not limited to, 'parts where competence was found,' 'reason (basis) for judging that competence exists,' 'discovered competence,' and / or 'competency record.' For example, the evaluation result generating AI module (1004) based on the scoring standard may include, but is not limited to, 'scoring basis,' 'scoring score,' 'feedback,' and / or 'relevance score.' For example, the evaluation result generating AI module (1006) based on other standards may include, but is not limited to, 'evaluation basis,' 'evaluation score,' and / or 'feedback.'

[0117] Evaluation Results Generated AI Module Evaluation Results Based on Achievement Standards Generated AI Module Evaluation Results Based on Scoring Standards Generated AI Module Evaluation Results Based on Other Standards Generated AI Module Evaluation Results Achievement Results Scoring Results Evaluation Results Areas where competence was found, reasons (basis) for judging competence, discovered competence, competence records, etc. Scoring Basis, Scored Score, Feedback, Relevance Score, etc. Evaluation Basis, Evaluation Score, Feedback

[0118] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards can evaluate task results according to predetermined achievement standards. In one embodiment, the achievement standards may be defined as predetermined competency data. In one embodiment, the predetermined competency data may be defined based on related class information. For example, class information is information such as 'subject' or 'grade', and if the subject is 'Social Studies', the competency data may be defined as: 'Creative thinking ability is the ability to generate new and valuable ideas', 'Critical thinking ability is the ability to analytically evaluate situations', 'Problem-solving and decision-making ability is the ability to make rational decisions to solve various social problems', 'Communication and collaboration ability is the ability to clearly express one's views and interact effectively with others', and 'Information utilization ability is the ability to collect, interpret, utilize, and create information by utilizing various materials and technologies'.

[0119] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards may evaluate a task result according to a predetermined achievement standard and generate an achievement result. In one embodiment, the achievement result may include a part where competence was discovered, a reason (basis) for judging that competence exists, a discovered competence, and a competence record, but is not limited thereto.

[0120] In one embodiment, an AI module (1002) for generating evaluation results based on achievement standards can determine whether there is a part of the task result that matches competency data (i.e., whether there is 'competency' in the task result) and generate an achievement result. For example, if there is a part that matches the competency data, the achievement result may be the part where the competency was found, the reason (basis) for determining that the competency exists, the discovered competency, and the competency record.

[0121] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards determines whether there is a part that matches competency data for a predetermined task result (i.e., whether there is 'competency' in the task result) and can generate a part where competency is found. The part where competency is found may be content written in the task result. For example, the AI ​​module (314) for generating evaluation results may generate "Part where competency is found: The United States is the Republic of Korea's greatest ally, maintaining friendly relations with the Republic of Korea. The relationship between the United States and Korea began after World War II, when the United States and the Soviet Union began to split into two systems while pursuing different ideologies. At that time, the Korean War broke out, and the United States quickly deployed a large amount of military force. The reason the United States helped South Korea was to prevent a third war on the Korean Peninsula. However, since the Korean War is still under armistice and neither side has won, the United States is still maintaining relations with South Korea." and the part may be content written in the task result.

[0122] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards can generate the reason for which competence was found for a predetermined task result. For example, the AI ​​module (314) for generating evaluation results can generate "reason for which competence was found: deeply analyzing political relations, particularly historical events, and logically explaining their causes and long-term impact" for a predetermined task result, and this may be the reason (ground) for judging that competence exists.

[0123] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards can generate a competency found for a predetermined task result. For example, the AI ​​module (314) for generating evaluation results can generate "discovered competency: critical thinking skills" for a predetermined task result.

[0124] In one embodiment, the AI ​​module (1002) for generating evaluation results according to achievement standards can generate a competency record based on the content of the part where competency was found, the reason (grounds) for judging that competency exists, and the discovered competency. For example, the AI ​​module (314) for generating evaluation results can generate "Competency Record: Excellent critical thinking ability by logically explaining the reasons for the start and continuation of the historical relationship between Korea and the United States, thereby demonstrating deep analysis of political, particularly historical, events and understanding the long-term impact thereof."

[0125] In one embodiment, the AI ​​module (1004) for generating evaluation results based on scoring criteria may evaluate a task result according to a predetermined scoring criterion and generate a scoring result. In one embodiment, the scoring result may include a scoring basis, a scoring score, feedback, and a relevance score, but is not limited thereto.

[0126] When using generative AI models, the process by which the generated results were derived is generally not explained, making it difficult to find logical or rational grounds for the outcomes. Due to this characteristic, users often find it difficult to trust the results provided by the AI. However, this post presents the scoring results along with the corresponding rationale. This allows users to verify the explanation and basis for the results, thereby enhancing the reliability of the findings and aiding in the understanding of the AI's judgment.

[0127] In one embodiment, the AI ​​module (1004) for generating evaluation results according to the scoring criteria can generate a scoring basis by evaluating the assignment result according to the scoring elements associated with the predetermined scoring criteria. In one embodiment, the AI ​​module (1004) for generating evaluation results according to the scoring criteria can generate a scoring basis based on the content written in the assignment result by evaluating the assignment result according to the scoring elements associated with the predetermined scoring criteria. For example, the AI ​​module (1004) for generating evaluation results according to the scoring criteria can generate a scoring basis by citing the content written in the assignment result.

[0128] In one embodiment, the AI ​​module (1004) for generating evaluation results according to the scoring criteria evaluates the project results according to detailed scoring elements associated with a predetermined scoring element and can generate a scoring score based on the points associated with the detailed scoring element. In one embodiment, the AI ​​module (1004) for generating evaluation results according to the scoring criteria generates advanced detailed scoring elements based on the detailed scoring elements associated with the scoring element and evaluates the project results according to the advanced detailed scoring elements to generate a scoring score.

[0129] In one embodiment, the AI ​​module (1004) for generating evaluation results based on scoring criteria can evaluate the assignment results according to scoring elements associated with a predetermined scoring criterion and generate feedback. Here, the feedback is an evaluation opinion provided to the person who submitted the assignment results. In one embodiment, the AI ​​module (1004) for generating evaluation results based on scoring criteria can generate advice that is helpful to the person who submitted the assignment results as feedback based on the assignment results and the aforementioned scoring grounds. In one embodiment, the AI ​​module (1004) for generating evaluation results based on scoring criteria can generate advice that is helpful to the person who submitted the assignment results by considering the level of the person who submitted the assignment results. For example, the AI ​​module (1004) for generating evaluation results based on scoring criteria can generate advice that is helpful to the person who submitted the assignment results by considering the person's level based on assignment information (e.g., school level, grade).

[0130] In one embodiment, the AI ​​module (1004) for generating evaluation results based on scoring criteria can generate a relevance score. Here, the relevance score is a numerical value indicating how closely the task result matches the predetermined scoring criteria.

[0131] For example, regarding the assignment "Compare the similarities and differences between villages and cities, and explore the problems and solutions arising in each" in the 4th grade Social Studies class unit on "Diverse Lifestyles and Changes," the evaluation result generation AI module (504) evaluates the assignment results according to the grading criteria (e.g., the grading element is "differences between villages and cities," and the points and detailed grading elements are "exceptional: accurately mentions the differences between villages and cities by item," "good: briefly mentions the differences between villages and cities by item," and "poor: does not know or incorrectly mentions the differences between villages and cities"), and the grading basis ("When comparing the differences between villages and cities, differences were mentioned in several items. Differences were written in terms of 'population' and 'buildings,' and comparisons were also made in the parts of 'work done by people' and 'land use.' However, contextual errors were found and more detailed explanations are needed") and the grading score ("poor: does not know or incorrectly mentions the differences between villages and cities"). It can generate feedback ('It is necessary to consider more diverse factors regarding the differences between villages and cities. For example, it would be good to compare them in terms of education or environment. Also, it is necessary to use more grammatically accurate expressions.') and a relevance score ('50').

[0132] FIG. 11 is a drawing illustrating an exemplary evaluation plan according to one embodiment of the present disclosure, and FIG. 12 is a drawing illustrating an exemplary scoring result generated by evaluating a task result according to the evaluation plan in an AI module that generates evaluation results according to scoring criteria.

[0133] As illustrated in FIG. 11, scoring elements (understanding of the subject and accuracy) are shown in relation to the performance evaluation (evaluation name: water resource conservation method). The scoring elements (understanding of the subject and accuracy) include detailed scoring elements according to each score (score (10), detailed scoring element ('accurately understand the reason for water shortage from the perspective of seawater and freshwater and explain it validly'), (score (8), detailed scoring element ('explain the reason for water shortage from the perspective of seawater and freshwater by including some inaccurate content'), (score (6), detailed scoring element ('explained the reason for water shortage, but failed to explain it from the perspective of seawater and freshwater').

[0134] As illustrated in FIG. 12, the AI ​​module (1004) for generating evaluation results based on scoring criteria can perform an evaluation of scoring elements on a task result to generate a score (1202), a scoring basis (1204), and feedback (1206).

[0135] In the example illustrated in FIG. 12, the AI ​​module (1004) for generating evaluation results based on the scoring criteria evaluates the scoring element ('understanding of the topic and accuracy') for the task result and generates a score ('10 points'), a basis for scoring ('You accurately understood the reason for water shortage from the perspective of seawater and freshwater and explained it validly. In addition, you presented seawater desalination technology as a solution to the water shortage problem and explained its necessity well'), and feedback ('You have made an excellent analysis and suggestion. I hope you will continue to engage in activities such as researching various scientific phenomena or problems in such depth and considering solutions for them. Additionally, it would be good to investigate other water resource-related technologies and evaluate their efficiency or feasibility.').

[0136] Although not shown in FIG. 12, the AI ​​module (1004) for generating evaluation results according to the scoring criteria generates evaluation results for all scoring elements. For example, the AI ​​module (1004) for generating evaluation results according to the scoring criteria evaluates the scoring element ('validity of evidence') for the task result and can generate a score, scoring basis, and feedback for the scoring element ('validity of evidence') in the same format as the scoring result evaluated for the aforementioned scoring element ('understanding of the topic and accuracy').

[0137] In one embodiment, the evaluation result generation AI module (314) may further include an evaluation result generation AI module (1006) based on other criteria. In one embodiment, the evaluation result generation AI module (1006) based on other criteria may evaluate the project results according to predetermined other criteria and generate an evaluation result. In one embodiment, the evaluation result may include an evaluation score, feedback, and / or the basis for evaluation.

[0138] FIG. 13 is an operation flowchart conceptually showing the process of an evaluation result generating AI module generating an evaluation result according to one embodiment of the present disclosure.

[0139] First, in step (S1301), the evaluation result generation AI module (314) may receive a predetermined evaluation plan and at least one task result for the evaluation plan. In one embodiment, the predetermined evaluation plan may include at least one piece of information among a subject name, grade level, grading criteria, and achievement criteria, but is not limited thereto.

[0140] In step (S1303), the evaluation result generating AI module (314) can generate an evaluation result (e.g., a score result) by evaluating the task result using a generative AI model based on the scoring criteria of the evaluation plan. In one embodiment, the scoring criteria include at least one scoring element, a score associated with a predetermined scoring element, and at least one of a detailed scoring element associated with the score, but is not limited thereto. In one embodiment, the evaluation result generating AI module (314) can generate a score result by evaluating the task result using an AI model based on the detailed scoring element. Here, the score result may include at least one of a scoring basis, a score, feedback, or a relevance score, but is not limited thereto. The basis for grading is generated by citing the content listed in the above assignment results, the grading score is generated based on the points associated with the detailed grading elements, the feedback is an evaluation opinion provided to the person who submitted the assignment results, generated by considering the level of the person who submitted the assignment results based on grade information, and the relevance score may be the similarity between the assignment results and the prescribed grading criteria.

[0141] In step (S1305), the evaluation result generating AI module (314) can generate an evaluation result (e.g., achievement result) by evaluating the task result using a generative AI model based on the achievement criteria of the evaluation plan. In one embodiment, the achievement criteria are defined as predetermined competency data, and the predetermined competency data may be defined based on the associated subject name. In one embodiment, the achievement result may include at least one of the part where competency was found, the reason for judging that competency exists, the found competency, and the competency record, but is not limited thereto. In one embodiment, the part where competency was found is generated based on the content written in the task result by evaluating whether there is a part of the task result that matches the competency data, the found competency is one of the predetermined competency data, and the competency record may be generated based on the content of the part where competency was found, the reason for judging that competency exists, and the found competency.

[0142] In another embodiment, the evaluation result generating AI module (314) can generate an achievement result by evaluating a task result using a generative AI model based on the detailed achievement criteria of the evaluation plan. Here, the detailed achievement criteria are related to a grade in which the achievement criteria are subdivided into the degree of achievement according to the grade.

[0143] In steps (S1307) and (S1309), the evaluation result generating AI module (314) displays the scoring result or achievement result, and if there is input of modifications to the scoring result or achievement result from the user, it can receive the modifications and save the modified scoring result or achievement result.

[0144]

[0145] Refer again to Fig. 3.

[0146] In one embodiment of the present disclosure, the comprehensive record generation AI module (316) can generate an evaluation record regarding at least one evaluation conducted in a predetermined subject. In one embodiment, the comprehensive record generation AI module (316) can receive an evaluation result for a predetermined evaluation and generate an evaluation record regarding that evaluation. In one embodiment, the comprehensive record generation AI module (316) can generate a comprehensive record by combining the evaluation records regarding each evaluation.

[0147] In one embodiment, the comprehensive record generation AI module (316) may receive an evaluation plan and generate an evaluation record regarding the evaluation. In one embodiment, the comprehensive record generation AI module (316) may receive an evaluation plan from the evaluation plan module (310) and generate an evaluation record regarding the evaluation. Here, the evaluation plan may include items related to the evaluation, such as an evaluation (plan) name, at least one achievement standard, a scoring standard, and at least one scoring element associated with the scoring standard.

[0148] In one embodiment, the comprehensive record generation AI module (316) may receive a scoring result for a predetermined evaluation from the evaluation result generation AI module (1004) according to the scoring criteria and generate an evaluation record regarding the evaluation. Here, the scoring result may be at least one of a scoring score, a scoring basis, feedback, or a relevance score obtained by evaluating the task result according to at least one scoring element, but is not limited thereto.

[0149] In one embodiment, the comprehensive record generation AI module (316) can receive evaluation plan data from the evaluation plan module (310) and generate an evaluation record regarding the evaluation. Here, the evaluation plan may include at least one of a scoring element, a score, and a detailed scoring element.

[0150] Table 2 shows an example of data received by the comprehensive record generation AI module (316) for a predetermined evaluation (evaluation name: finding a career path through Korean language activities). The comprehensive record generation AI module (316) can receive an evaluation plan (e.g., evaluation name, scoring criteria (scoring elements, points, detailed scoring elements)) and an evaluation result (e.g., scoring score).

[0151] Assessment Title: Finding Career Paths Through Korean Language Activities Scoring Element: 1 Point Detailed Scoring Element: Scoring Score Reading Activity: 10. Read a book in the field of interest to the end and specifically summarize the content linked to the career path in a mini-book. 10. 8. Read a book in the field of interest and summarize the content linked to the career path in a mini-book. 6. Read a book in the field of interest partially and summarize some of the content linked to the career path in a mini-book. Scoring Element: 2 Points Detailed Scoring Element: Scoring Score Career Exploration Writing: 10. Connect the content of the book and interview results to one's career path to construct well-structured content and write with consistency. 4. 8. Connect the content of the book and interview results to one's career path to write with consistency. 4. Connect the content of the book and interview results to one's career path to write. Scoring Element: 3 Points Detailed Scoring Element: Scoring Score Explanation: Systematically plan considering the target audience, purpose of writing, and reader. 5. Explanation: Systematically plan considering the target audience, purpose of writing, and reader. 50. Explanation: Failed to systematically plan considering the target audience, purpose of writing, and reader.

[0152] In one embodiment, the comprehensive record generation AI module (316) can generate a predetermined evaluation record based on detailed scoring elements related to the scoring result in all scoring elements (e.g., scoring elements 1 to 3) included in a predetermined evaluation plan. For example, the comprehensive record generation AI module (316) generates an evaluation record ('reads books in the field of interest to the end and specifically organizes content linked to career paths in a mini-book') for a predetermined evaluation plan (e.g., evaluation name - finding a career path through Korean language activities) based on the detailed scoring element ('reads books in the field of interest to the end and specifically organizes content linked to career paths in a mini-book') related to the score ('10') in scoring element 1 ('reading activities'), the detailed scoring element ('writes by connecting the content of the book and interview results to one's career path') related to the score ('4') in scoring element 2 ('career exploration writing'), and the detailed scoring element ('systematically establishes a plan considering the subject of explanation, the purpose of writing, and the reader') related to the score ('systematically establishes a plan considering the subject of explanation, the purpose of writing, and the reader') in scoring element 3 ('systematically establishes a plan considering the subject of explanation, the purpose of writing, and the reader'). Reflected. Excellent results were achieved in writing activities thanks to a systematic plan established by considering the subject of explanation, the purpose of writing, and the reader. However, in the career exploration writing, a slight lack of consistency was observed in the process of connecting the book's content and interview results to one's career path. It appears that significant growth in future writing activities is expected if more attention is paid to the consistency of content and the structure of the composition. While the ability to actively find materials necessary for career exploration and incorporate them into one's career plan was excellent, it was confirmed that improvement is needed in systematically expressing those ideas through writing.

[0153] In one embodiment, the comprehensive record generation AI module (316) can generate an evaluation record by writing it from a positive perspective when generating the evaluation record, but if negative content is inevitably written (e.g., when the scoring score is low), it can generate the evaluation record by adding a method to supplement it.

[0154] In one embodiment, the comprehensive record generation AI module (316) may receive an achievement result for a predetermined evaluation from the evaluation result generation AI module (1002) according to the achievement standard and generate an evaluation record regarding the evaluation. Here, the achievement result is the result of evaluating a task result according to at least one achievement standard, and may be at least one of a part where competence was found, a reason (basis) for judging that competence exists, a discovered competence, and a competence record, but is not limited thereto.

[0155] In another embodiment, the comprehensive record generation AI module (316) receives an evaluation plan from the evaluation plan module (310) and can generate an evaluation record regarding the evaluation based on the achievement criteria (grade, content) included in the evaluation plan.

[0156] In one embodiment, the comprehensive record generation AI module (316) may generate an evaluation record for each evaluation plan by further including the evaluation results generated by the evaluation result generation AI module (1002) according to the achievement standards. In one embodiment, the evaluation results generated by the evaluation result generation AI module (1002) according to the achievement standards for each evaluation plan may be competency records that evaluated the task results according to at least one achievement standard. For example, the comprehensive record generation AI module (316) may generate a record for each evaluation plan by further including the result ('critical and creative competencies found') generated by the evaluation result generation AI module (1002) according to the achievement standards for each evaluation plan.

[0157] In one embodiment, the comprehensive record generation AI module (316) can generate a comprehensive record by comprehensively summarizing at least one evaluation plan-specific evaluation result record generated based on the evaluation result record generated by the evaluation result generation AI module (1004) according to the scoring criteria for each predetermined evaluation plan. In one embodiment, the comprehensive record generation AI module (316) can highlight, underline, bold text, etc., a portion of the evaluation plan-specific evaluation result record cited when generating the comprehensive record.

[0158] In one embodiment, the comprehensive record generation AI module (316) can generate a comprehensive record by receiving student observation record data from the class management module (308). In one embodiment, the comprehensive record generation AI module (316) can generate a comprehensive record based on student observation record data ('a student who is impressive for concentrating well during class and having a diligent attitude').

[0159] FIG. 14 is a drawing illustrating, in accordance with one embodiment of the present disclosure, an exemplary comprehensive record generated by a comprehensive record generation AI module for a predetermined student.

[0160] As illustrated in FIG. 14, the comprehensive record of each student, for example, student ('Kim Gana (10101)'), and the scoring results and analysis content that served as the basis for it can be displayed. In one embodiment, for each of the multiple evaluation plans (evaluation names: 'Future Society Prediction Scenario', 'Understanding an Integrated Perspective') included in a predetermined evaluation plan, the evaluation score (1402) obtained by the student ('Kim Gana (10101)'), feedback (1404) for each evaluation plan, and student observation records (1406) can be displayed on a part of the screen (e.g., left side). In one embodiment, the comprehensive record generating AI module (316) can receive and display the evaluation score (1402) and feedback (1404) for each evaluation plan. In one embodiment, the comprehensive record generating AI module (316) can receive and display the evaluation score (1402) and feedback (1404) for each evaluation plan from the evaluation result generating AI module (314). In one embodiment, the comprehensive record generation AI module (316) can receive and display student observation records. In one embodiment, the comprehensive record generation AI module (316) can receive and display student observation records from the class management module (308).

[0161] In one embodiment, an evaluation record (1408) and the basis for generating the evaluation record (1410) may be displayed on a part of the screen (e.g., center). In one embodiment, the comprehensive record generation AI module (316) may generate an evaluation record and the basis for generating the evaluation record for each evaluation plan based on the evaluation results of the assignment deliverables submitted by the student for the evaluation plan (e.g., evaluation score obtained by the student, competency record, and feedback) and the evaluation plan (e.g., scoring details associated with the evaluation score).

[0162] In one embodiment, a portion of the screen (e.g., right) may display a total score (1412), a comprehensive record (1414), and the basis for generating the comprehensive record (not shown). In one embodiment, the comprehensive record generating AI module (316) may generate a comprehensive record (1414) by integrating and summarizing evaluation records (1408) for each evaluation plan. In one embodiment, the comprehensive record generating AI module (316) may generate a comprehensive record (1414) by further including student observation records (1406). In one embodiment, the comprehensive record generating AI module (316) may calculate a total score (1412) by summing the evaluation scores for each evaluation plan.

[0163]

[0164] FIG. 15 is an operation flowchart conceptually showing the process of an evaluation result generating AI module generating an evaluation result according to one embodiment of the present disclosure.

[0165] First, in step (1501), the comprehensive record generation AI module (316) receives a command to generate a comprehensive record and prepares to create a comprehensive evaluation record based on the evaluation results received in subsequent steps. This step is an initial step for starting the generation of a comprehensive record, and the module can prepare to process a series of evaluation results after receiving the command.

[0166] In step (1503), the comprehensive record generation AI module (316) receives a scoring result for a predetermined evaluation from the evaluation result generation AI module (904) based on the scoring criteria. This scoring result includes at least one of a scoring score, scoring basis, feedback, or relevance score that evaluated the task result according to at least one scoring element. Based on this, an individual evaluation record for the evaluation is generated, and the scoring basis and feedback can be used to identify the direction of improvement of the evaluation subject and to record it.

[0167] In step (1505), the comprehensive record generation AI module (316) can receive competency evaluation results from the evaluation result generation AI module (902) based on achievement standards and generate evaluation records for each predetermined evaluation plan. The results generated by the evaluation result generation AI module (902) based on achievement standards for each predetermined evaluation plan may be competency records that evaluated task results according to at least one achievement standard.

[0168] In step (1507), the comprehensive record generation AI module (316) can be configured to write the evaluation record from a positive perspective when generating the evaluation record. However, if there are parts where the scoring score is low or is lacking, instead of recording the content negatively, it suggests the possibility of improvement to generate a positive and developmental evaluation record. This allows for meaningful feedback and guidance on the direction of development to be provided to the learner.

[0169] In step (1509), the comprehensive record generation AI module (316) can generate a comprehensive record by comprehensively summarizing at least one evaluation result record for each evaluation plan, which is generated based on the evaluation records generated by the evaluation result generation AI module (904) according to the scoring criteria for each evaluation plan. In this process, the overall evaluation plan is comprehensively summarized based on individual evaluation records to derive the final evaluation result, and this comprehensive record clearly shows the learner's overall performance.

[0170] In step (1511), the comprehensive record generation AI module (316) can highlight, underline, bold, etc., parts of the evaluation result records for at least one evaluation plan cited when generating the comprehensive record.

[0171] In step (1513), the comprehensive record generation AI module (316) can store the comprehensive record.

[0172]

[0173] In the embodiments of the present disclosure described above with reference to FIG. 1 through 15, etc. (and throughout this specification), the user terminal (110) and the evaluation aid device (130) are described as being implemented based on a client-server model, particularly in which the client primarily provides only user input / output functions and most other functions are delegated to the server, but the present disclosure is not limited thereto. It should be noted that according to other embodiments of the present disclosure, the evaluation aid system environment may be implemented with its functions evenly distributed between the user terminal and the server, or rather may be implemented relying more on the application environment installed on the user terminal. Furthermore, it should be noted that when the functions of the evaluation aid system are implemented by distributing them between the user terminal and the server according to one embodiment of the present disclosure, the distribution of each function of the evaluation aid system between the client and the server may be implemented differently depending on the embodiment.

[0174] Additionally, in the embodiments of the present disclosure described above with reference to FIGS. 1 to 3, specific modules are described as performing specific operations for convenience, but the present disclosure is not limited thereto. According to other embodiments of the present disclosure, it should be understood that the operations described above as being performed by a specific module may each be performed by a different, separate module.

Claims

1. A method for assisting in the generation of evaluation results using a generative AI model executed on an evaluation assistance server, Step of receiving a predetermined evaluation design - the said predetermined evaluation design includes subject name and grade information, and grading criteria, and A step of receiving at least one task result; A method for assisting in the creation of evaluation results using a generative AI model, comprising the step of evaluating a task result using a generative AI model based on the above scoring criteria and generating a scoring result.

2. In Paragraph 1, The step of generating an evaluation result by evaluating the project deliverables using a generative AI model based on the above scoring criteria is: The above scoring criteria includes at least one scoring element, at least one point corresponding to the at least one scoring element, and at least one detailed scoring criterion associated with the at least one point. A method for assisting in the creation of evaluation results using a generative AI model, which further includes the step of evaluating project deliverables using an AI model based on the above detailed scoring criteria and generating a scoring result.

3. In Paragraph 1, A method for assisting in the generation of evaluation results using a generative AI model that includes at least one of a scoring basis, a scoring score, feedback, or a relevance score for at least one scoring element.

4. In Paragraph 3, The above scoring basis is generated by citing the contents described in the above assignment deliverable, and The above scoring score is generated based on the points associated with detailed scoring elements, and The above feedback is an evaluation opinion provided to the person who submitted the assignment result, generated based on the above grade information and considering the level of the person who submitted the assignment result, and A method for assisting in the creation of evaluation results using a generative AI model, wherein the above relevance score is the similarity between the task result and a predetermined scoring criterion.

5. In Paragraph 1, A step of receiving the achievement standards of the above-mentioned predetermined evaluation plan - the achievement standards are defined as predetermined competency data, and the predetermined competency data is defined based on related instruction information; and A method for assisting in the creation of evaluation results using a generative AI model, which further includes the step of evaluating task results using a generative AI model based on the above achievement standards to generate achievement results.

6. In Paragraph 5, The step of generating achievement results by evaluating task deliverables using a generative AI model based on the above achievement standards A method for assisting in the creation of evaluation results using a generative AI model, which further includes the step of evaluating assignment results using an AI model based on predetermined competency data defined in the subject of the above-mentioned class information to generate achievement results.

7. In Paragraph 5, The above achievement result includes at least one of the part where competence was discovered, the reason for judging that competence exists, the discovered competence, and the competence record, and The parts where the above capabilities were discovered are generated based on the contents described in the above project deliverables by evaluating whether there are parts that match the above capability data with the above project deliverables, and The above-described capability is one of the above-described capability data, and The above capability record is a method for assisting in the creation of evaluation results by using a generative AI model generated based on the content of the part where the capability was found, the reason for judging that the capability exists, and the discovered capability.

8. In Paragraph 5, The above achievement standards include grades and detailed achievement standards associated with the grades, and A method for assisting in the creation of evaluation results using a generative AI model, which further includes the step of generating achievement results by evaluating task deliverables using a generative AI model based on the above detailed achievement standards.

9. A recording medium for executing on a computing device a method for assisting in the generation of evaluation results using a generative AI model executed by a computing device, wherein the computer-readable recording medium for executing on a computer the method of any one of claims 1 to 8.

10. An apparatus for assisting in the generation of evaluation results using a generative AI model, wherein the apparatus for assisting in the generation of evaluation results using a generative AI model configured to execute the method of any one of claims 1 to 8.

11. A method for assisting in the creation of comprehensive records using a generative AI model executed on an evaluation assistance server, Step of receiving a predetermined evaluation plan - the said predetermined evaluation plan includes at least one of subject name and grade information, achievement standards or grading criteria -; A step of receiving the evaluation results of assignment deliverables submitted by a specified student regarding the above-mentioned predetermined evaluation plan; and A method for assisting in the creation of a comprehensive record using a generative AI model that includes the step of generating an evaluation record corresponding to the above evaluation results.

12. In Paragraph 11, The step of receiving the evaluation result of an assignment result submitted by a specified student regarding the above-mentioned evaluation plan further includes the step of receiving the grading result of the assignment result submitted by the said specified student - said grading result includes at least one of a grading score, a basis for grading, feedback, or a relevance score obtained by evaluating the assignment result according to at least one grading element -; A method for assisting in the creation of a comprehensive record using a generative AI model, wherein the step of generating an evaluation record corresponding to the above evaluation result further comprises the step of generating an evaluation record using a generative AI model based on detailed scoring elements, scoring grounds, and feedback related to the scoring score in the above scoring elements.

13. In Paragraph 12, The step of generating an evaluation record using a generative AI model based on detailed scoring elements, scoring grounds, and feedback related to the scoring score in the above scoring elements is a step performed for all scoring elements included in the above evaluation plan; and A method for assisting in the creation of a comprehensive record using a generative AI model that further includes the step of generating the basis for creating the above-mentioned evaluation record.

14. In Paragraph 13, The step of receiving the evaluation results of assignment results submitted by a specified student regarding the above-mentioned evaluation plan further includes the step of receiving the achievement results of assignment results submitted by the above-mentioned student - the achievement results include competency records -; A method for assisting in the creation of a comprehensive record using a generative AI model, which further includes the step of generating an evaluation record corresponding to the above evaluation result using a generative AI model based on the above competency record.

15. In Paragraph 14, A method for assisting in the creation of a comprehensive record using a generative AI model, wherein the step of creating an evaluation record corresponding to the above evaluation result further includes the step of creating improvement measures when creating an evaluation record if the scoring score is below a predetermined standard.

16. In Paragraph 11, A method for assisting in the creation of a comprehensive record using a generative AI model that further includes the step of integrating and summarizing evaluation records generated according to the above-mentioned predetermined evaluation plans to create a comprehensive record.

17. In Paragraph 16, A method for assisting in the creation of a comprehensive record using a generative AI model that further includes the step of highlighting a portion of at least one evaluation record by evaluation plan cited when creating the above comprehensive record.

18. In Paragraph 16, The step of generating the above comprehensive record is a method of assisting in the creation of a comprehensive record using a generative AI model that is generated by further including student observation records.

19. A recording medium for executing on a computing device a method for assisting in the creation of a comprehensive record using a generative AI model executed by a computing device, wherein the computer-readable recording medium for executing on a computer the method of any one of claims 11 to 18.

20. An apparatus for assisting in the creation of a comprehensive record using a generative AI model, wherein the apparatus for assisting in the creation of a comprehensive record using a generative AI model configured to execute the method of any one of claims 1 to 8. As a method of assisting in the creation of an evaluation plan using a generative AI model executed on an evaluation assistance server, A step of generating an evaluation plan by receiving an evaluation plan generation command; Step of receiving scoring elements; and A method for assisting in the preparation of an evaluation plan using a generative AI model, comprising the step of generating detailed scoring elements using a generative AI model based on the above scoring elements.

21. In Paragraph 20, Step of receiving achievement standards; and A method for assisting in the preparation of an evaluation plan using a generative AI model, further comprising the step of generating detailed scoring elements using a generative AI model based on the above scoring elements and the above achievement standards.

22. In Paragraph 20, Step of receiving subject names; and A method for assisting in the creation of an evaluation plan using a generative AI model, wherein the evaluation plan includes at least one of the subject name, the achievement standard, or the scoring element, and further includes the step of storing the evaluation plan in association with the subject name.

23. In Paragraph 20, A method for assisting in the preparation of an evaluation plan using a generative AI model that further includes a step of receiving a predetermined grade, wherein the grade is stored in association with the detailed scoring elements.

24. In Paragraph 20, Step of receiving an evaluation mode; and A method for assisting in the preparation of an evaluation plan using a generative AI model that further includes the step of generating detailed scoring elements that judge task results by domain and assign scores according to criteria for each scoring element, when the above evaluation mode is an analytical evaluation mode.

25. In Paragraph 20, Step of receiving an evaluation mode; and A method for assisting in the preparation of an evaluation plan using a generative AI model that further includes the step of generating detailed scoring elements that assign scores based on whether each scoring element is satisfied, when the above evaluation mode is a holistic evaluation mode.

26. In Paragraph 21, A method for assisting in the preparation of an evaluation plan using a generative AI model that further includes the step of generating advanced detailed scoring elements based on detailed scoring elements associated with the above-mentioned scoring elements.

27. A recording medium for executing a relation extraction method executed by a computing device on a computing device, wherein the recording medium is computer-readable for executing the method of any one of claims 20 to 26 on a computer.

28. An apparatus for assisting in the preparation of an evaluation plan using a generative AI model, wherein the apparatus for assisting in the preparation of an evaluation plan using a generative AI model configured to execute the method of any one of claims 20 to 26.

Images