Ai learning assessment device based on multidimensional competency analysis and method thereof

Abstract

The AI ​​learning evaluation method based on multidimensional competency analysis of the present invention comprises the steps of: extracting a plurality of evaluation questions from a previously stored question pool based on a learning field and evaluation purpose received from an administrator terminal; mapping the extracted evaluation questions to a previously set evaluation schema to form a final evaluation set; providing the final evaluation set to a learner terminal; collecting response data for each evaluation question from the learner regarding the final evaluation set and executing a learning evaluation; generating a learner competency diagnosis result including the learner's competency level using a competency score calculated based on the response data; and selecting learning content based on weak competency elements identified in the learner competency diagnosis result, which includes the step of generating a personalized learning path and recommending the selected learning content.

Description

Technology Field

[0001] The present invention relates to an AI learning evaluation device and method based on multidimensional capability analysis. Background Technology

[0002] With the recent proliferation of digital learning environments, active attempts are being made across various fields to assess learners' proficiency and provide customized learning content based on the results. These learning assessment systems generally feature a structure in which a large number of assessment questions are pre-built, and the learner's level is determined based on the scores obtained by solving these questions, or assessment results are provided in the form of simple grades or scores.

[0003] Existing learning assessment systems provide fixed sets of questions or fixed assessment templates for specific subjects or fields, and learners are evaluated by taking tests that are standardized according to those templates.

[0004] However, these conventional learning assessment methods have limitations in accurately diagnosing a learner's actual ability. Even if two learners receive the same total score, one may have a strong understanding of concepts but weak application skills, while the other may possess the opposite characteristics; yet, existing systems often fail to adequately distinguish and analyze these differences. Furthermore, if the composition of assessment items is skewed toward specific competency elements or difficulty levels, the evaluation results may fail to accurately reflect the learner's actual abilities and lead to distorted outcomes.

[0005] In conventional learning assessment systems, the process of configuring assessment items or templates was often done manually, relying mostly on the experience and intuition of the operator or instructor. As a result, the quality of the assessment items can vary significantly depending on the individual evaluator's competence, and problems such as specific competency elements being undervalued, difficulty distributions becoming unbalanced, or low-discriminatory items being excessively included can frequently occur.

[0006] Furthermore, despite the fact that core concepts or key competency elements to be evaluated in specific fields are continuously evolving due to the rapid pace of change in knowledge and technology, many existing systems have frequently reused established item pools or templates for extended periods. Consequently, this can lead to problems where evaluation items fail to adequately reflect the latest trends or competencies required in actual business operations, resulting in a decline in the validity and effectiveness of the assessment.

[0007] Furthermore, conventional systems have not sufficiently provided functions to systematically detect or verify the quality of results in situations where the reliability of evaluation results is low, such as when a learner did not participate in the evaluation sincerely, responded randomly, or completed the response within an abnormally short period of time. As a result, unreliable evaluation results may be used as is to determine the learner's level or recommend subsequent learning, which can lead to the problem of suggesting incorrect learning directions. The problem to be solved

[0008] The technical problem that the present invention aims to solve is to provide a multidimensional competency analysis-based AI learning evaluation device and method that allows a manager to directly select evaluation items according to the learning field and evaluation purpose desired by the manager, and to edit the generated evaluation set according to the manager's intentions. means of solving the problem

[0009] A multidimensional competency analysis-based AI learning evaluation method according to an embodiment of the present invention comprises: a step of extracting a plurality of evaluation questions from a previously stored question pool based on a learning field and evaluation purpose received from an administrator terminal; a step of configuring a final evaluation set by mapping the extracted evaluation questions to a previously set evaluation schema; a step of providing the final evaluation set to a learner terminal; a step of executing a learning evaluation by collecting response data for each evaluation question from the learner regarding the final evaluation set; a step of generating a learner competency diagnosis result including the learner's competency level using a competency score calculated based on the response data; a step of selecting learning content based on weak competency elements identified in the learner competency diagnosis result; and a step of recommending the selected learning content by generating a personalized learning path.

[0010] According to an embodiment, the step of extracting evaluation items may selectively extract evaluation items corresponding to the learning field and evaluation purpose by referring to an attribute including process classification information assigned to each evaluation item stored in the item pool.

[0011] According to an embodiment, prior to the step of extracting the evaluation items, the method may further include the step of collecting evaluation item data by linking with an external server, the step of deriving core competency elements by applying a pre-trained artificial intelligence model to the collected evaluation item data, the step of generating new item candidates reflecting the core competency elements, and the step of performing automatic quality verification on the new item candidates and registering them in the item pool.

[0012] According to an embodiment, the step of configuring the final evaluation set may include: loading an evaluation template corresponding to the manager's selection information; creating an evaluation set by placing extracted evaluation items into the evaluation template; creating a competency measurement structure model by aggregating the metadata of the evaluation items included in the evaluation set by competency dimension; creating a correction proposal by comparing the competency measurement structure model with a predefined target standard model; and creating the final evaluation set by editing the evaluation set based on the correction proposal.

[0013] According to an embodiment, after the step of configuring the final evaluation set, the method may further include the step of simulating a virtual test of the final evaluation set through a previously generated virtual learner to derive an expected evaluation result, and the step of replacing items included in the final evaluation set or adjusting the points to reduce the deviation from the target distribution if the expected evaluation result does not match the previously set target distribution.

[0014] According to an embodiment, the step of executing the learning evaluation involves collecting micro-behavioral patterns that occur during the process of solving evaluation questions by the learner, along with the learner's response data for each evaluation question of the final evaluation set, and storing them as a learning behavior log. The micro-behavioral patterns may include at least one of the dwell time per question, the movement trajectory of the mouse cursor, the number of times the answer is modified, and the number of times the browser focus is lost.

[0015] According to an embodiment, the step of generating the learner competency diagnosis result may generate the learner competency diagnosis result that maps the learner's competency level and learning attitude tendency by synthesizing a quantitative competency score calculated by analyzing the response data and an attitude indicator including sincerity and concentration derived by analyzing the learning behavior log.

[0016] According to an embodiment, the method may further include the step of generating visualization data that visually represents the learner competency diagnosis result, generating an analysis report including the visualization data, and providing it to the learner terminal.

[0017] An AI learning evaluation device that performs a learning evaluation using a processor that performs operations on at least one application or program according to another embodiment of the present invention may include: a learning evaluation design unit that extracts a plurality of evaluation questions from a previously stored question pool based on a learning field and evaluation purpose received from an administrator terminal and maps the extracted evaluation questions to a previously set evaluation schema to construct a final evaluation set; a learning evaluation execution unit that collects response data for each evaluation question of a learner regarding the final evaluation set and executes a learning evaluation; an evaluation result analysis unit that generates a learner competency diagnosis result including the learner's competency level using a competency score calculated based on the response data; and a learning content recommendation unit that selects learning content based on weak competency elements identified in the learner competency diagnosis result, generates a personalized learning path, and recommends the selected learning content.

[0018] According to an embodiment, the processor may further include an evaluation question generation unit that collects evaluation question data by linking with an external server, derives core competency elements by applying a pre-trained artificial intelligence model to the collected evaluation question data, generates new question candidates reflecting the core competency elements, and registers the new question candidates in the question pool by performing automatic quality verification. Effects of the invention

[0019] According to the AI ​​learning evaluation device and method based on multidimensional competency analysis according to an embodiment of the present invention, an administrator can directly select evaluation items according to the learning field and evaluation purpose desired by the administrator, and edit and finalize the generated evaluation set according to the administrator's intention.

[0020] In addition, according to the multidimensional competency analysis-based AI learning evaluation device and method of the embodiment of the present invention, by verifying structural bias through a competency measurement structure model and performing a simulated test using a virtual learner, it is possible to predict difficulty and discriminability before actual evaluation and ensure consistency with the target distribution.

[0021] In addition, according to the multidimensional competency analysis-based AI learning evaluation device and method of the present invention, in addition to the learner's response data (correct / incorrect answers), micro-behavioral patterns (stay time, mouse trajectory, focus loss, etc.) occurring during the problem-solving process are collected and analyzed to calculate attitude indicators such as sincerity and concentration, thereby enabling a three-dimensional and comprehensive diagnosis of the learner's competency.

[0022] In addition, according to the multidimensional competency analysis-based AI learning evaluation device and method of the embodiment of the present invention, external data such as news and technical documents are collected through linkage with an external server, and by automatically converting and verifying this into new questions reflecting core competency elements through an AI model and registering them, the problem of question pool depletion can be resolved and evaluation content reflecting the latest trends can be continuously provided. Brief explanation of the drawing

[0023] FIG. 1 is a schematic block diagram of an AI learning evaluation system based on multidimensional competency analysis according to an embodiment of the present invention. FIG. 2 is a schematic block diagram of an AI learning evaluation device based on multidimensional competency analysis according to an embodiment of the present invention. FIG. 3 is a schematic block diagram of a processor according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a multidimensional competency analysis-based AI learning evaluation method according to an embodiment of the present invention. FIGS. 5 and 6 are drawings for explaining an AI learning evaluation method based on multidimensional competency analysis according to an embodiment of the present invention. Specific details for implementing the invention

[0024] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0025] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0026] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0027] The terms “part” or “module” as used in the specification refer to software or hardware components, such as FPGAs or ASICs, and “part” or “module” perform certain roles. However, “part” or “module” is not limited to software or hardware. “Part” or “module” may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, by example, “part” or “module” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules,” or further separated into additional components and “parts” or “modules.”

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0029] FIG. 1 is a schematic block diagram of an AI learning evaluation system based on multidimensional competency analysis according to an embodiment of the present invention.

[0030] Referring to FIG. 1, the multidimensional competency analysis-based AI learning evaluation system (10) according to an embodiment of the present invention analyzes the learning field and evaluation purpose received from the administrator, selects the optimal evaluation item from the item pool, and provides an intelligent learning evaluation service that provides a highly reliable competency diagnosis and a personalized learning path through simulation using a virtual learner and analysis of learning attitude.

[0031] In addition, the multidimensional competency analysis-based AI learning evaluation system (10) according to an embodiment of the present invention continuously generates new evaluation questions reflecting the latest trends by linking with an external data server to automatically expand the question pool, and supports an in-depth analysis environment that goes beyond simple incorrect answer checking to precisely track prerequisite learning elements that are the root cause of weak competencies by referring to a knowledge graph in which the hierarchy between concepts is defined.

[0032] To this end, the AI ​​learning evaluation system (10) includes an AI learning evaluation device (100), an external data source (200), a database (300), an administrator terminal (400), and a learner terminal (500).

[0033] The AI ​​learning evaluation device (100) can preprocess news, technical documents, and academic materials collected from an external data server into a form that can be analyzed by an artificial intelligence model, generate new question candidates by deriving core competency elements through the application of a pre-trained AI model, and systematically register and manage them in a question pool through automatic quality verification.

[0034] The AI ​​learning evaluation device (100) loads evaluation templates corresponding to the administrator's learning field and evaluation purpose to arrange questions, and can generate a competency measurement structure model to verify structural bias of the generated evaluation set. In particular, the AI ​​learning evaluation device (100) can simulate a virtual test through a virtual learner created based on previously accumulated learning history data, and if the expected score distribution does not match the target distribution, it can automatically perform question replacement or score adjustment to ensure the validity of the evaluation in advance.

[0035] In addition, the AI ​​learning evaluation device (100) collects the learner's response data for each item during the evaluation execution, and at the same time tracks minute behavioral patterns such as the time spent on each item, the movement trajectory of the mouse cursor, the number of times the answer was corrected, and the number of times the browser focus was lost in real time and stores them as a learning behavior log, thereby enabling the analysis of learning attitude beyond simple scoring.

[0036] The AI ​​learning evaluation device (100) can generate multidimensional learner competency diagnosis results by combining quantitative competency scores and attitude indicators and provide them as a visualized report, and furthermore, by referencing the Knowledge Graph to track prior learning elements that are the root cause of weak competencies and creating a personalized learning path to recommend customized content, it can dramatically improve learning efficiency.

[0037] According to an embodiment, the AI ​​learning evaluation device (100) may be any one of a PC (personal computer), a smartphone, a tablet PC, a mobile internet device (MID), an internet tablet, an IoT (internet of things) device, an IoE (internet of everything) device, a desktop computer, a laptop computer, a workstation computer, a Wibro (Wireless Broadband Internet) terminal, and a PDA (Personal Digital Assistant), but is not limited thereto and may include all types of communication devices.

[0038] According to another embodiment, the AI ​​learning evaluation device (100) may be an operating server that operates a learning content provision platform, which is a device capable of hosting an online network and network addressing. The AI ​​learning evaluation device (100) may communicate with an external data source (200), a database (300), an administrator terminal (400), and a learner terminal (500) via a network communication network to provide a learning content provision platform.

[0039] Here, the network communication refers to a connection structure capable of exchanging information between an AI learning evaluation device (100), an external data source (200), a database (300), an administrator terminal (400), and a learner terminal (500), and may refer to a local area network (LAN), a wide area network (WAN), the internet (WWW), a wired and wireless data communication network, a telephone network, a wired and wireless television communication network, etc.

[0040] The AI ​​learning evaluation device (100) may be a cloud computing model that provides a Platform as a Service (PaaS) capable of building a learning content provision platform on-premise and running and managing related applications, or a model that provides a Software as a Service (SaaS) capable of using the learning content provision platform service via the internet without installing a separate program.

[0041] The AI ​​learning evaluation device (100) can automatically perform the entire learning evaluation and coaching process, including collecting external data and generating new questions using a pre-trained artificial intelligence model, optimizing evaluation sets through virtual simulation, analyzing attitudes based on learning behavior logs, diagnosing multidimensional competencies, and recommending content based on a knowledge graph.

[0042] Here, the artificial intelligence model may be a deep learning model. A deep learning model (e.g., a deep neural network (DNN)) may refer to a multilayer perceptron that includes multiple hidden layers in addition to the input and output layers. Deep neural networks can be used to identify the latent structures of data. Deep neural networks may include, but are not limited to, convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, and Siamese networks.

[0043] The external data source (200) is a data storage location related to learning evaluation and can provide data necessary for new evaluation questions of the AI ​​learning evaluation device (100). For example, the external data source (200) may include all forms of structured and unstructured data constituting a knowledge system of a specific learning field, such as textbooks and lecture notes published by educational institutions, papers listed in academic databases, technical white papers from the industry, relevant laws and recent precedents, news articles, or online encyclopedias.

[0044] The external data source (200) may be an online service that provides API-based integration, such as a public data portal or an academic information service, and the AI ​​learning evaluation device (100) can collect the latest information in real time by analyzing the API schema of the service provider and performing a data request that takes into account the authentication method, search query parameters, and response format (JSON, XML, etc.).

[0045] Additionally, the external data source (200) may be an existing legacy learning management system or a question bank server. The AI ​​learning evaluation device (100) can search for previously stored evaluation questions and learning history data through database access rights, table structure, and analysis of question metadata, and selectively collect them by converting them to fit the evaluation schema of the present invention.

[0046] According to an embodiment, the external data source (200) includes an electronic library system, a government-operated education portal, a MOOC (Massive Open Online Course) platform, a technology blog, an open source repository (GitHub, etc.), and the AI ​​learning evaluation device (100) can be securely connected to the external data source (200) through security protocols such as OAuth authentication, token-based authentication, and API key authentication.

[0047] The database (300) can store a pool of multiple evaluation questions with various difficulty levels and types. Each evaluation question can be stored in a structured manner along with metadata such as the question body, correct answer, and explanation data, as well as information on the process classification that the question intends to measure, evaluation attributes, difficulty level, and discrimination. Through this, the AI ​​learning evaluation device (100) can quickly search for and extract questions from the database (300) that match the administrator's learning field and evaluation purpose.

[0048] Additionally, the database (300) can store learning content to supplement the learner's weak competencies. The learning content includes various forms of educational materials such as video lectures, concept summary notes, advanced learning materials, and problem-solving videos, and each piece of content can be stored by mapping it to a specific learning concept or competency element.

[0049] The administrator terminal (400) can input or modify selection information including learning fields and evaluation purposes through a web-based or app-based UI, and provides an environment in which the AI ​​learning evaluation device (100) generates a final evaluation set and the learner can take it.

[0050] The administrator terminal (400) can view the list of items in the evaluation set proposed by the AI ​​learning evaluation device (100), and supports an interface that allows the administrator to proactively determine the evaluation configuration by performing a request to select, exclude, or replace specific evaluation items when necessary.

[0051] In particular, the administrator terminal (400) can receive and display on the screen a list of questions in the final evaluation set primarily configured by the AI ​​learning evaluation device (100) and an expected score distribution, which is a simulation result through a virtual learner, in the form of a visual graph, and the administrator can determine in advance the appropriateness of the difficulty and discriminability of the test through this.

[0052] The administrator terminal (400) can visually display the learner competency diagnosis results provided by the AI ​​learning evaluation device (100), the multidimensional competency graph, and the list of recommended learning content through the UI screen.

[0053] The learner terminal (500) receives the final evaluation set distributed from the AI ​​learning evaluation device (100), displays it on the screen, and can provide a web or application-based interface that allows the learner to solve the evaluation questions.

[0054] The learner terminal (500) can transmit the selected or entered answer to the evaluation question in real time to the AI ​​learning evaluation device (100) through the learner's input device. In particular, the learner terminal (500) can detect minute behavioral patterns in real time, such as the learner's dwell time per question, mouse movement trajectory, and browser focus departure, and transmit them to the AI ​​learning evaluation device (100) while the evaluation is in progress.

[0055] According to an embodiment, the administrator terminal (400) and the learner terminal (500) may each represent a PC, a smartphone, a tablet PC, a mobile internet device (MID), an internet tablet, an IoT (internet of things) device, an IoE (internet of everything) device, a desktop computer, a laptop computer, a workstation computer, or a PDA (personal digital assistant), but are not limited thereto.

[0056] FIG. 2 is a schematic block diagram of an AI learning evaluation device based on multidimensional competency analysis according to an embodiment of the present invention.

[0057] Referring to FIG. 2, an AI learning evaluation device (100) according to an embodiment of the present invention includes a processor (110), memory (120), a communication interface (130), and storage (140).

[0058] The processor (110) controls the overall operation of each component of the AI ​​learning evaluation device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphic Processing Unit), or any type of processor (110) well known in the art of the present invention.

[0059] Additionally, the processor (110) can perform operations for at least one application or program for executing the method according to embodiments of the present invention, and the server (100) may have one or more processors (110).

[0060] According to an embodiment, the processor (110) may further include RAM (Random Access Memory, not shown) and ROM (Read-Only Memory, not shown) for temporarily and / or permanently storing signals (or data) processed within the processor (110). Additionally, the processor (110) may be implemented in the form of a system-on-chip (SoC) comprising at least one of a graphics processing unit, RAM, and ROM.

[0061] Memory (120) stores various data, commands, or information. Memory (120) may load a program from storage (140) to execute a method according to various embodiments of the present invention. When a computer program is loaded into memory (120), the processor (110) may perform the method by executing one or more instructions that constitute the computer program. Memory (120) may be implemented as volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0062] The communication interface (130) supports wired or wireless communication of the AI ​​learning evaluation device (100). Additionally, the communication interface (130) may support various communication methods other than internet communication. To this end, the communication interface (130) may be configured to include a communication module well known in the art of the present invention. In some embodiments, the communication interface (130) may be omitted.

[0063] Storage (140) can store computer programs non-temporarily. When providing a content recommendation service through the AI ​​learning evaluation device (100), storage (140) can store various information necessary for generation and processing during the execution of the process.

[0064] The storage (140) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0065] The bus (150) provides communication functions between components of the AI ​​learning evaluation device (100). The bus (150) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0066] FIG. 3 is a schematic block diagram of a processor according to an embodiment of the present invention.

[0067] Referring to FIG. 3, the processor (110) includes an evaluation item generation unit (111), a learning evaluation design unit (112), a learning evaluation execution unit (113), an evaluation result analysis unit (114), a visual material generation unit (115), and a learning content recommendation unit (116). Each component is implemented by a software module, a hardware module, or a combination thereof, and may mean a unit that processes at least one function or operation by being connected through at least one communication path.

[0068] The evaluation question generation unit (111) manages a pool of questions for each field that has been built in advance and can extract evaluation questions corresponding to the learning field and evaluation purpose selected by the manager.

[0069] Here, evaluation items refer to a set of questions generated in advance for various learning fields, such as common competencies, leadership courses, and job competencies, and detailed course classification information, such as digital literacy or management insights, may be assigned as attributes. In other words, evaluation items are data prepared in such a state that questions of different topics are organized and managed within their respective areas without mixing, so that when an administrator selects a desired field using a checkbox on the screen, the system can selectively extract and provide only the corresponding questions.

[0070] First, the evaluation question generation unit (111) can secure a vast amount of past exam questions and example data by linking with an internal database, legacy LMS, or external content partnership server.

[0071] At this time, the evaluation item generation unit (111) is connected to the item database through security protocols such as an access control list (ACL) and security token authentication, and can obtain legitimate access rights to evaluation assets that are copyrighted or require security.

[0072] According to an embodiment, the evaluation question generation unit (111) may be linked with an external data source (200), such as an external learning content provision server, a qualification test operating institution, a publisher's question bank, or a closed question repository within a company, through an API interface. The evaluation question generation unit (111) may perform an authentication procedure by applying at least one of OAuth-based token exchange, API key authentication, or certificate-based mutual authentication according to the security policy required by each external system.

[0073] For example, when collecting the latest past exam questions from a question bank server operated by a specific qualification examination institution, the evaluation question generation unit (111) requests the issuance of an access token from an authentication server using a pre-registered client identifier and secret key, and by calling an API including the issued token in the header, it can safely receive only question data within the scope of authorized authority. At this time, the access token may include authorization information regarding the category of accessible questions, the validity period of use, the number of daily calls allowed, etc.

[0074] According to another embodiment, the evaluation question generation unit (111) can identify the usable scope of the question (e.g., evaluation type, usage period, redistribution availability, etc.) by analyzing license metadata included in the question data received from an external data source (200). The evaluation question generation unit (111) can verify whether the identified license conditions satisfy the internal policy based on a pre-set internal policy, and then selectively include only the verified questions in the question pool.

[0075] For example, if a usage restriction is set in the metadata of a specific question to be for mock test only, the evaluation question generation unit (111) can automatically filter the question so that it is not included in an actual accredited certification test or commercial evaluation set, or isolate and store it in a separate group.

[0076] In addition, the evaluation question generation unit (111) can generate new evaluation questions or supplement existing evaluation questions according to external trend changes while stably providing an existing pool of questions to maximize the efficiency of evaluation question management.

[0077] For example, if the criteria for a specific qualification test field are changed, the evaluation question generation unit (111) can reflect the latest learning environment while maintaining data continuity by selectively updating or creating a new version of only the evaluation questions of a specific category corresponding to the changed criteria and merging them into the existing question pool, instead of regenerating all questions.

[0078] The evaluation question generation unit (111) can preprocess collected external data (e.g., news, technical documents, academic materials, etc.) or internal learning history data into a form that can be analyzed by an artificial intelligence model, and can identify question trends or extract key keywords from unstructured text data and convert them into a standardized data structure.

[0079] For example, the evaluation question generation unit (111) can use text mining technology to extract new terms that frequently appear in the latest technical documents or identify and remove outdated expressions within the existing question text.

[0080] According to an embodiment, the evaluation item generation unit (111) can apply a pre-trained artificial intelligence model to the pre-processed data to derive new core competency elements required for each field and generate new item candidates that reflect the derived core competency elements. At this time, the evaluation item generation unit (111) can generate metadata including the competency elements to be measured by the new item candidates, the required cognitive level, difficulty category, and item type, and configure the new item candidates to fit a structured item schema.

[0081] For example, if recent data analysis results indicate that the importance of the ability to formulate problem-solving strategies in a specific technology field has increased, the evaluation item generation unit (111) can be controlled to prioritize the generation of situation-presentation type items or case-based items that can measure the relevant competency elements, instead of simple knowledge-type items.

[0082] The evaluation item generation unit (111) can perform an automatic quality verification procedure to analyze the ambiguity of expressions within the item, the uniqueness of the correct answer, and whether there is duplication among the options in order to ensure the reliability of the newly generated item candidates. The evaluation item generation unit (111) can quantitatively evaluate the degree of duplication with the existing item pool using a similarity comparison algorithm or a semantic duplication detection algorithm, and can automatically filter out item candidates that do not satisfy the pre-set quality criteria or classify them as targets for regeneration.

[0083] The evaluation item generation unit (111) can generate evaluation items for new item candidates whose quality verification has been completed by structuring the content and metadata of the new item candidates by mapping them to a predefined item schema.

[0084] The learning evaluation design unit (112) can extract an evaluation template based on evaluation attribute information selected by the manager and provide a customized evaluation configuration according to the manager's editing actions through the user interface.

[0085] Here, an assessment template refers to a standardized assessment structure model in which execution conditions, such as item composition ratios, difficulty distribution, and time limits, are predefined to align with a specific learning field, assessment purpose, and assessment type. Assessment templates serve as a foundational framework that reduces the burden on managers of designing assessments from scratch and supports the efficient initiation of design by referencing standard assessment criteria verified by experts.

[0086] When an administrator selects the learning field, purpose of evaluation, and type of evaluation to be evaluated, the learning evaluation design unit (112) loads an evaluation template corresponding to the selection result, and can design the desired evaluation configuration by adding, deleting, or replacing evaluation questions in the evaluation template in response to the administrator's input and adjusting the points for each evaluation question.

[0087] The learning evaluation design unit (112) can perform artificial intelligence-based quality verification on the evaluation set designed by the manager. For example, the learning evaluation design unit (112) can analyze the evaluation items included in the evaluation set and structurally interpret which competency elements are measured with what weight, whether the difficulty distribution is excessively skewed to a specific range, and whether a specific type of evaluation item is under- or over-included.

[0088] The learning evaluation design unit (112) can perform statistical aggregation by extracting metadata (e.g., measurement competency tags, difficulty levels, item type codes, etc.) of individual evaluation items included in the evaluation set and calculating the distribution density by competency dimension. Through this, the learning evaluation design unit (112) can generate a competency measurement structure model that indicates which competency the currently configured evaluation set aims for overall. For example, the competency measurement structure model can be expressed in the form of a topology that indicates where the center of gravity of the evaluation set is located and whether the difficulty variance follows a normal distribution in a multidimensional space composed of multiple competency axes.

[0089] The learning evaluation design unit (112) can interpret structural bias by comparing the generated ability measurement structure model with a predefined target standard model. For example, if the ratio of concept understanding to application ability is set to 5:5 in the target standard model, but is analyzed as 8:2 in the currently generated ability measurement structure model, the learning evaluation design unit (112) can interpret this as a 'structure with under-included application ability measurement items' and generate a correction proposal to correct the deviation.

[0090] If it is diagnosed that there is a lack of evaluation items for application ability in the ability measurement structure model, the learning evaluation design unit (112) can provide a feedback message to the manager saying, "It is recommended to add items to strengthen application ability," or provide alternative evaluation items from the item pool that include the relevant ability element without disrupting the difficulty balance of the current evaluation set, thereby inducing the manager to immediately improve the evaluation set.

[0091] In addition, the learning evaluation design unit (112) can perform a simulated simulation by applying evaluation questions to a virtual learner to predict whether the designed evaluation set will function as intended in the actual learning environment.

[0092] Specifically, the learning evaluation design unit (112) selects a virtual learner with various conditions such as learning ability, occupation, age, and gender for the currently configured evaluation set, and has the selected learner virtually take the evaluation set to generate an expected response to each item (e.g., whether it is correct, time taken to solve, probability of giving up, etc.), and analyzes the generated data to derive an expected score distribution and item response curve, thereby verifying in advance whether the evaluation set has the target difficulty and discriminative power.

[0093] Here, the virtual learner refers to a simulation agent generated based on accumulated learning history data and learner behavior logs, representing a virtual user object parameterized with behavioral characteristics such as the learner's competency level, problem-solving speed, error probability, and guessing tendency. Designed to follow the statistical distribution of the actual learner population, the virtual learner can precisely simulate actual response results by probabilistically calculating the attributes of the input assessment items and its own ability parameters.

[0094] The learning evaluation design unit (112) can analyze the evaluation results of a virtual learner to determine whether the expected score distribution matches the pre-set target distribution. If structural imbalance is detected, such as an excessive concentration of personnel in a specific score range, the learning evaluation design unit (112) can generate and provide feedback to the administrator to improve the completeness of the evaluation set, such as identifying items that need to be replaced to correct the imbalance or suggesting adjustments to the scoring.

[0095] The learning evaluation design unit (112) can create a final evaluation set by confirming the evaluation set that has been edited and verified by the manager, and by structuring the configured evaluation items and metadata (e.g., total score, time limit, section information, scoring rules, etc.) by mapping them to a predefined evaluation schema. The final evaluation set can be converted into a standardized data format, such as JSON format, and stored in the database (300) in a state that allows the subsequent learning evaluation execution unit (113) to immediately distribute and execute it to learners without changing any logic.

[0096] The learning evaluation execution unit (113) can provide the finalized evaluation set as a user interface optimized for the environment of the learner terminal (500) and manage the entire process from the start to the end of the evaluation. By structuring different client environments, such as web browsers, mobile apps, and tablets, into an execution schema with a common format, the learning evaluation execution unit (113) can support the artificial intelligence model in consistently tracking the learner's behavioral patterns during the evaluation process.

[0097] First, the learning evaluation execution unit (113) can be linked with the learner terminal (500) through responsive web technology. The learning evaluation execution unit (113) can identify the identity of the learner through security protocols such as login session verification and group affiliation verification, and can grant access to the evaluation only to authorized learners by verifying the evaluation target group and eligibility to take the exam.

[0098] In addition, the learning evaluation execution unit (113) may apply a state recovery mechanism to maximize the stability of the evaluation process, which saves the learner's response status in real time and resumes from the point of interruption in the event of a network error or terminal failure. For example, if an abnormal termination is detected during the evaluation, the learning evaluation execution unit (113) can prevent data loss and ensure the continuity of the examination experience by loading the log and answer data from the previous saving point and restoring the remaining time and solution status.

[0099] The learning evaluation execution unit (113) can detect cheating by linking with the learner terminal (500) to ensure fairness in the evaluation in an online environment. The learning evaluation execution unit (113) obtains webcam, microphone, and screen capture permissions with the learner's consent and can monitor gaze deviation, detection of multiple people, background noise patterns, etc. through real-time video analysis. In addition, by monitoring the browser's focus status, if deviation behavior is detected during the evaluation, such as opening another tab or running an external messenger program, it can immediately issue a warning or mark and save the log at that time as data suspected of cheating.

[0100] Additionally, the learning evaluation execution unit (113) can collect minute behavioral patterns that occur during the process of a learner solving problems in the form of a learning behavior log. For example, the learning evaluation execution unit (113) can record the dwell time for each question, the movement trajectory of the mouse cursor, the number of corrections made after selecting an answer, the scroll speed, and the hesitation time over a specific option in seconds.

[0101] According to an embodiment, the learning evaluation execution unit (113) may execute a computer-adaptive test logic that dynamically determines the optimal next item based on the learner's real-time response pattern. At this time, the final evaluation set may not include fixed evaluation items, but may include an evaluation item group containing multiple items that meet the evaluation purpose, and an evaluation item extraction policy that determines the items to be presented to the learner from the evaluation item group. That is, the learner may receive evaluation items selected in real-time according to their ability level within a defined range called the final evaluation set.

[0102] The learning evaluation execution unit (113) can determine the learner's potential ability level in real time by performing a Bayesian estimation or maximum likelihood estimation algorithm based on an item response algorithm whenever the learner submits a response to a specific item. Then, the learning evaluation execution unit (113) can generate an individualized test path optimized for each learner by searching for the item with the maximum information function value or the highest discriminative power from the candidate item group and presenting it as the next item based on the determined ability level.

[0103] The learning evaluation execution unit (113) can dynamically determine the end time of the evaluation by monitoring a pre-set termination rule in real time. The learning evaluation execution unit (113) can immediately end the evaluation when at least one of the following conditions is satisfied: when the standard error of the learner's ability estimate value converges to a pre-set threshold value or lower and it is determined that the reliability of the measurement is secured, when the pre-set maximum number of items is reached, or when the time limit expires.

[0104] The learning evaluation execution unit (113) can generate an evaluation result by combining the collected response data for each item and the learning behavior log recorded during the evaluation process when the evaluation is completed or when a command to submit an answer is entered by the learner.

[0105] The evaluation result analysis unit (114) can calculate the learner's competency score based on the evaluation results and analyze the behavior log collected during the evaluation process to diagnose the learner's learning attitude, such as sincerity and concentration.

[0106] First, the evaluation result analysis unit (114) can perform a primary scoring process for quantitative questions with clear correct answers, such as multiple choice or short answer questions. The evaluation result analysis unit (114) can calculate a raw score by applying the correct answer information mapped to each question and the scoring rules, and can calculate a converted score by reflecting the difficulty weight and partial score recognition rules for each question.

[0107] If the evaluation is conducted using a computer-adaptive test method, the evaluation result analysis unit (114) can convert the final converged estimate of the learner's ability into a score and determine a standard score that allows for fair comparison even between learners who have solved different items.

[0108] Additionally, the evaluation result analysis unit (114) can perform AI-based semantic analysis on qualitative evaluation questions that do not have fixed correct answers, such as descriptive, essay, oral interview, and project assignment questions. The evaluation result analysis unit (114) can convert text answers or voice data submitted by the learner into text and perform natural language processing on the converted text to analyze the logical structure of the sentence, whether key keywords are included, and the appropriateness of the expression.

[0109] At this time, the evaluation result analysis unit (114) can calculate a score by calculating the similarity between the predefined evaluation criteria and the answer, or evaluate the creativity and critical thinking skills of the answer using a generative AI model and generate descriptive feedback thereon.

[0110] The evaluation result analysis unit (114) can derive competency scores by classifying the scores for each evaluation item into competency categories tagged with each evaluation item and summing or weighting the averages.

[0111] Additionally, the evaluation result analysis unit (114) can analyze the learning behavior logs collected during the evaluation process to calculate an attitude score that quantifies the learning attitude. For example, the evaluation result analysis unit (114) can classify cases where the time spent on a specific question is significantly shorter than the average time spent on that question and the answer is incorrect as an insincere guess and deduct the sincerity score. Conversely, cases where the time spent is long, the number of answer corrections is frequent, and the mouse cursor repeatedly explores the area around a specific option can be identified as a state of careful deliberation and reflected in the concentration index.

[0112] According to an embodiment, the evaluation result analysis unit (114) can generate a learner competency diagnosis result by mapping the learner's position on a predefined competency system diagram using the derived competency score and attitude indicator.

[0113] According to an embodiment, the evaluation result analysis unit (114) can compare the learner competency diagnosis results with statistical data of a group to be compared, such as the same job group, same age group, or students taking the same learning course, and produce comparative analysis data including the relative position of the learner's score within the group, the deviation from the mean, and position information on the overall group distribution for each item.

[0114] The visual material generation unit (115) can generate visual materials including graphic elements based on the results of the learner competency diagnosis and comparative analysis data, so that the learner can intuitively recognize their competency level.

[0115] The visual material generation unit (115) can generate a competency balance graph in the form of a hexagon model by converting the score rate data for each competency item into multidimensional coordinate values. At this time, the visual material generation unit (115) can generate a reference graph by mapping statistical data of the group to which the learner belongs (e.g., same job group, same age group, average of all test takers, etc.) onto the same coordinate system, and by rendering this overlapping with the learner's individual competency graph, it can support the learner in visually immediately identifying their relative level and the distribution of strengths and weaknesses within the entire group.

[0116] The visual material generation unit (115) can convert and display attitude indicators such as the learner's sincerity and concentration into a time series graph or a gauge chart. For example, the visual material generation unit (115) can visualize the trend of change in the time spent on each item relative to the total time spent solving the items as a line graph to show whether concentration is maintained in the latter part of the evaluation compared to the beginning, or provide attitude indicators in the form of a percentage gauge by combining the number of times the learner left and the rate of insincere responses.

[0117] The visual material generation unit (115) can generate a structured final analysis report by combining competency analysis charts, comparison graphs, attitude indicator visualization data, and customized learning prescription UI. The final analysis report can be provided in the form of a dynamic dashboard that responds to the learner's interaction (e.g., click, hovering, etc.) on a web browser, or converted into a static document format such as PDF or JPG that is easy to print and store and provided to the learner terminal (500).

[0118] The learning content recommendation unit (116) can select learning content that can most effectively supplement the learner's lacking competencies based on the results of the learner competency diagnosis and the identified weak competency elements.

[0119] The learning content recommendation unit (116) can primarily select content that is highly relevant to the learner's weaknesses through content-based filtering. Specifically, the learning content recommendation unit (116) can analyze learning goals, key keywords, and related competency tags included in the content metadata, and select learning content that directly addresses the weak competency elements by matching them with the weak competency elements identified in the evaluation results.

[0120] At this time, the learning content recommendation unit (116) can apply the learner's competency level or job information as a filtering condition to selectively extract only content that matches the learner's level or is one level higher.

[0121] In addition, the learning content recommendation unit (116) can identify not only the weak competency elements but also the prerequisite learning elements that are the root cause of the weak competency elements by utilizing a knowledge graph in which the hierarchical relationships and causal structures between learning concepts are defined.

[0122] Here, a knowledge graph refers to a relational data model constructed by setting concepts, keywords, or competency elements—the smallest units of learning—as individual nodes, and connecting the prerequisite learning relationships, causal relationships, or difficulty hierarchy structures between nodes via edges. In other words, a knowledge graph is not merely a simple list of data, but a knowledge base structured in a network form of sub-concepts that must be understood first to acquire specific knowledge.

[0123] For example, if a specific problem-solving ability is diagnosed as weak but the results of the knowledge graph analysis indicate that understanding the basic theory that forms the basis of the ability must be prioritized, the learning content recommendation unit (116) can arrange content explaining the basic theory rather than content for solving practical problems in a priority learning order.

[0124] According to an embodiment, the learning content recommendation unit (116) may apply a collaborative filtering algorithm that utilizes data from a group of learners with similar learning patterns or competency distributions. The learning content recommendation unit (116) can identify a group of successful learners who previously showed weaknesses similar to the current learner but whose competencies have significantly improved after learning specific content. Furthermore, the learning content recommendation unit (116) can provide high-efficiency content with statistically verified learning effects by extracting a list of content that the group has taken the most or has shown high satisfaction with and recommending it to the current learner.

[0125] The learning content recommendation unit (116) can sort selected content according to learning priority, time required, and content type to create a personalized learning path and provide it to the learner terminal (500).

[0126] FIG. 4 is a flowchart for explaining an AI learning evaluation method based on multidimensional competency analysis according to an embodiment of the present invention, and FIG. 5 and FIG. 6 are drawings for explaining an AI learning evaluation method based on multidimensional competency analysis according to an embodiment of the present invention.

[0127] Referring to FIGS. 4 to 6, the AI ​​learning evaluation device (100) can extract multiple evaluation questions from a previously stored question pool based on the learning field and evaluation purpose received from the administrator terminal (400) (S100). The AI ​​learning evaluation device (100) can primarily select evaluation questions that are most suitable for the learning range and difficulty level selected by the administrator by referring to the metadata of each evaluation question stored in the question pool (e.g., course classification, difficulty level, evaluation attributes, etc.).

[0128] The AI ​​learning evaluation device (100) can configure a final evaluation set by mapping extracted evaluation questions to a pre-set evaluation schema (S110). In this process, the AI ​​learning evaluation device (100) does not merely arrange the questions but can perform a simulation in which a virtual learner, created based on accumulated learning history data, virtually takes the evaluation set. If the expected score distribution derived through the simulation does not match the pre-set target distribution, the AI ​​learning evaluation device (100) can determine a final evaluation set with verified reliability by undergoing an optimization process of replacing questions or adjusting the points to reduce the deviation.

[0129] Additionally, the AI ​​learning evaluation device (100) can provide a final evaluation set to the learner terminal (500) (S120). The AI ​​learning evaluation device (100) transmits the configured evaluation set in accordance with the rendering format of the learner terminal (500) so that the learner can intuitively check and take the questions through a web or app UI, and may also provide an editing interface that allows an administrator to exclude or change some questions as needed.

[0130] After that, the AI ​​learning evaluation device (100) can perform a learning evaluation by collecting the learner's response data for each evaluation item in the final evaluation set (S130). The AI ​​learning evaluation device (100) can collect not only the correct / incorrect answer data entered by the learner, but also the minute behavioral patterns that occur during the solution process in real time and store them as a learning behavior log. The AI ​​learning evaluation device (100) can obtain data on the learner's learning attitude that is difficult to identify from simple score results by tracking the dwell time per item, the movement trajectory of the mouse cursor, the number of times the answer was corrected, and the number of times the browser focus was lost.

[0131] The AI ​​learning evaluation device (100) can generate a learner competency diagnosis result using competency scores calculated based on response data (S140). The AI ​​learning evaluation device (100) can analyze quantitative competency scores and learning behavior logs, and comprehensively analyze attitude indicators including sincerity and concentration derived from the analysis results. Through this, the AI ​​learning evaluation device (100) can generate a diagnosis result that maps the learner's cognitive competency level (achievement) and affective competency level (learning attitude) to a multidimensional axis, and process this into a visualized graph or report form.

[0132] Additionally, the AI ​​learning evaluation device (100) can select learning content based on weak competency elements identified in the learner competency diagnosis results (S150). The AI ​​learning evaluation device (100) can trace back the prerequisite learning elements that are the root cause of the weak competency elements associated with the problems the learner got wrong by referring to a knowledge graph in which hierarchical relationships between concepts are defined. The AI ​​learning evaluation device (100) can select learning content such as lectures, summary notes, or advanced problems optimized to supplement the traced prerequisite learning elements from the database (300).

[0133] Additionally, the AI ​​learning evaluation device (100) can generate a personalized learning path and recommend selected learning content (S160). The AI ​​learning evaluation device (100) can generate a personalized curriculum by structuring the selected content in an order that maximizes learning effectiveness, and can induce self-directed learning by providing this as a recommendation list to the learner terminal (500).

[0134] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0135] 10: AI Learning Assessment System Based on Multidimensional Competency Analysis 100: AI learning evaluation device 110: Processor 111: Evaluation Item Generation Section 112: Learning Assessment Design Department 113: Learning Assessment Implementation Department 114: Evaluation Result Analysis Department 115: Visual Data Generation Unit 116: Learning Content Recommendation Section 120: Memory 130: Communication Interface 140: Storage 150: Bus 200: External data source 300: Database 400: Administrator Terminal 500: Learner terminal

Claims

Claim 1 The AI ​​learning evaluation device performs the following steps: extracting a plurality of evaluation questions from a previously stored question pool based on the learning field and evaluation purpose received from the administrator terminal; loading an evaluation template corresponding to the administrator's selection information received from the administrator terminal; creating an evaluation set by placing the extracted evaluation questions in the evaluation template; generating a competency measurement structure model by aggregating the metadata of the evaluation questions included in the evaluation set by competency dimension; calculating a deviation for at least one of the distribution by competency dimension and the difficulty distribution of the evaluation set by comparing the competency measurement structure model with a predefined target standard model, and generating a correction proposal including the replacement of evaluation questions or adjustment of scores based on the calculated deviation; and configuring a final evaluation set by editing the evaluation set based on the correction proposal and mapping the evaluation questions and metadata included in the edited evaluation set to a pre-set evaluation schema. A multidimensional competency analysis-based AI learning evaluation method comprising: a step in which the AI ​​learning evaluation device provides the final evaluation set to a learner terminal; a step in which the AI ​​learning evaluation device collects response data for each evaluation item of the learner regarding the final evaluation set and executes a learning evaluation; a step in which the AI ​​learning evaluation device generates a learner competency diagnosis result including the learner's competency level using a competency score calculated based on the response data; a step in which the AI ​​learning evaluation device selects learning content based on weak competency elements identified in the learner competency diagnosis result; and a step in which the AI ​​learning evaluation device generates a personalized learning path and recommends the selected learning content. Claim 2 In claim 1, the step of extracting evaluation items is a multidimensional competency analysis-based AI learning evaluation method that selectively extracts evaluation items corresponding to the learning field and evaluation purpose by referring to an attribute including process classification information assigned to each evaluation item stored in the item pool. Claim 3 A multidimensional competency analysis-based AI learning evaluation method according to claim 1, further comprising, prior to the step of extracting evaluation items, a step in which the AI ​​learning evaluation device collects evaluation item data by linking with an external data source; a step in which the AI ​​learning evaluation device applies a previously trained artificial intelligence model to the collected evaluation item data to derive core competency elements; a step in which the AI ​​learning evaluation device generates new item candidates reflecting the core competency elements; and a step in which the AI ​​learning evaluation device performs automatic quality verification on the new item candidates and registers them in the item pool. Claim 4 delete Claim 5 A multidimensional competency analysis-based AI learning evaluation method according to claim 1, further comprising, after the step of configuring the final evaluation set, a step in which the AI ​​learning evaluation device simulates a virtual test of the final evaluation set through a previously generated virtual learner to derive an expected evaluation result; and a step in which, if the expected evaluation result does not match a pre-set target distribution, the AI ​​learning evaluation device replaces items included in the final evaluation set or adjusts the scoring to reduce the deviation from the target distribution. Claim 6 In claim 1, the step of executing the learning evaluation comprises collecting micro-behavioral patterns occurring during the process of solving evaluation questions by the learner, along with the learner's response data for each evaluation question of the final evaluation set, and storing them as a learning behavior log, wherein the micro-behavioral patterns include at least one of the dwell time per question, the movement trajectory of the mouse cursor, the number of answer corrections, and the number of browser focus departures, thereby providing a multidimensional competency analysis-based AI learning evaluation method. Claim 7 In claim 6, the step of generating the learner competency diagnosis result comprises synthesizing a quantitative competency score calculated by analyzing the response data and an attitude indicator including sincerity and concentration derived by analyzing the learning behavior log, thereby generating the learner competency diagnosis result that maps the learner's competency level and learning attitude tendency. This is a multidimensional competency analysis-based AI learning evaluation method. Claim 8 A multidimensional competency analysis-based AI learning evaluation method according to claim 7, further comprising the step of the AI ​​learning evaluation device generating visualization data that visually represents the learner competency diagnosis result, generating an analysis report including the visualization data, and providing it to the learner terminal. Claim 9 An AI learning evaluation device that performs a learning evaluation using a processor that performs operations on at least one application or program, wherein the processor comprises: a learning evaluation design unit that, based on a learning field and evaluation purpose received from an administrator terminal, extracts a plurality of evaluation questions from a previously stored question pool and maps the extracted evaluation questions to a previously set evaluation schema to construct a final evaluation set; a learning evaluation execution unit that collects response data of a learner for each evaluation question regarding the final evaluation set and executes a learning evaluation; and an evaluation result analysis unit that generates a learner competency diagnosis result including the competency level of the learner using a competency score calculated based on the response data. A multidimensional competency analysis-based AI learning evaluation device comprising: a learning content recommendation unit that selects learning content based on weak competency elements identified in the above learner competency diagnosis results, generates a personalized learning path, and recommends the selected learning content; and a learning evaluation design unit that extracts a plurality of evaluation questions from a previously stored question pool, loads an evaluation template corresponding to the administrator's selection information received from the administrator terminal, creates an evaluation set by placing the extracted evaluation questions in the evaluation template, creates a competency measurement structure model by aggregating the metadata of the evaluation questions included in the evaluation set by competency dimension, calculates a deviation for at least one of the distribution by competency dimension and the difficulty distribution of the evaluation set by comparing the competency measurement structure model with a predefined target standard model, generates a correction proposal including replacement of evaluation questions or adjustment of points based on the calculated deviation, edits the evaluation set based on the correction proposal, and maps the evaluation questions and metadata included in the edited evaluation set to a pre-set evaluation schema to construct the final evaluation set. Claim 10 In claim 9, the multidimensional competency analysis-based AI learning evaluation device further comprises an evaluation item generation unit that collects evaluation item data in conjunction with an external data source, applies a pre-trained artificial intelligence model to the collected evaluation item data to derive core competency elements, generates new item candidates reflecting the core competency elements, performs automatic quality verification on the new item candidates, and registers them in the item pool.

Images