System for evaluating learning effect

Abstract

A learning effectiveness evaluation system is provided to address the lack of an objective learning effectiveness evaluation mechanism in learning platforms. The system comprises: a teaching platform having a pre-built question database containing at least one test question, each of the at least one test question having a first preset number of actions and a second preset number of actions; an operation module for a user to perform a response action; a recording module for acquiring a response action record; and an action analysis module for converting the response action record into a number of response actions and calculating an action index using an action analysis standard defined by the relationship between the first preset number of actions, the second preset number of actions, and the number of response actions.

Description

[Technical Field]

[0001] This invention relates to a learning effectiveness evaluation system, and more particularly to a learning effectiveness evaluation system for learning code writing. [Previous Technology]

[0002] With the development of technology and the transformation of educational concepts, self-directed learning has indeed become the mainstream trend in education. With the development of technology and the use of distance learning platforms, the convenience and accessibility of self-directed learning have been further enhanced. In many learning processes, it is often necessary for a tutor to provide appropriate feedback based on the learner's learning status in a timely manner to deepen the learner's understanding. However, learning platforms often only provide teaching materials, test questions and standard answers to the test questions, and only use the learner's answer scores as the basis for evaluating the learning effect. They lack an objective learning effect evaluation system or mechanism, making it impossible for learners or tutors to perceive the learner's learning status in a timely manner and provide appropriate feedback.

[0003] In view of this, the evaluation system for the effectiveness of familiar learning still needs to be improved. [Summary of the Invention]

[0004] To solve the above problems, the purpose of this invention is to provide a learning effect evaluation system that can provide users or managers with objective indicators of the user's learning performance as a reference for adjustments to the learning process.

[0005] A further objective of the present invention is to provide a learning effectiveness evaluation system that can perform relevant analysis on the user's answers or answering behavior to serve as a reference for determining whether the user has plagiarized.

[0006] The use of the quantifiers “a” or “an” for the elements and components described throughout this invention is for convenience and to provide the general meaning of the scope of the invention; in this invention, it should be interpreted as including one or at least one, and the single concept also includes the plural case, unless it clearly means otherwise.

[0007] The term "coupled" as used throughout this invention includes direct or indirect electrical and / or signal connections, which can be selected by those skilled in the art based on usage requirements.

[0008] The term "computer" as used throughout this invention refers to various data processing devices that have specific functions and are implemented in hardware or hardware and software, particularly those with a processor to process and analyze information and / or generate corresponding control information, such as servers, virtual machines, desktop computers, laptop computers, tablet computers, or smartphones, which can be understood by those skilled in the art to which this invention pertains.

[0009] The “character” as described throughout this invention includes at least a single character (which may be Chinese, English or other languages), a single number, and a single symbol.

[0010] The “instructions” described throughout this invention refer to the functional effects produced by specific key combinations (e.g., using the keyboard or left and right mouse clicks) or specific operations in various operating systems.

[0011] The learning effect evaluation system of the present invention includes: a teaching platform having a pre-established question database, the pre-established question database having at least one test question, each of the at least one test question having a first preset number of actions and a second preset number of actions, the value of the second preset number of actions being greater than the value of the first preset number of actions; an operation module providing one of the at least one test question to a user to perform a response action, and generating a response result after the response action is completed; the response action may be composed of at least one of at least one character and at least one instruction, or the response action may be composed of at least one of at least one character and at least one instruction. The response behavior is an empty set; the response result consists of at least one character, or the response result is an empty set; a recording module starts a recording program when the response behavior is triggered to record the response behavior; the recording program ends after the response behavior is completed, and a response behavior record is obtained from the recording program; an action analysis module converts the response behavior record into a response action number, and calculates an action index through an action analysis standard, the action analysis standard being defined by the relationship between the first preset action number, the second preset action number, and the response action number.

[0012] Accordingly, the learning effect evaluation system of the present invention can effectively evaluate the number of actions in the answering behavior through the action analysis module, so as to calculate the user's action index, and can achieve the effect of objectively evaluating the user's answering skills and familiarity with the test questions.

[0013] The action analysis module can convert the answer behavior record into the answer action count through a conversion standard. In the conversion standard, each character and / or each instruction in the answer behavior record is measured as an action count, and the answer action count is defined as the sum of such action counts. In this way, the action analysis module can accurately evaluate the action count in the answer behavior, and can achieve the effect of objectively and accurately evaluating the user's answering skills and familiarity with the test questions.

[0014] The formula for the action analysis standard can be: where Ac: represents the action index; as: represents the number of response actions; a1: represents the first preset number of actions; a2: represents the second preset number of actions. Thus, using this action analysis standard formula, the user's actions in response to the test questions can be quantified into an action index, achieving an objective and accurate assessment of the user's answering skills and familiarity with the test questions.

[0015] Wherein, each of the at least one test question may have a first preset time and a second preset time, the value of the second preset time being greater than the value of the first preset time; the recording program of the recording module records a response time to obtain a response time record; a time analysis module obtains the response time record as a response time and calculates a time index through a time analysis standard, the time analysis standard being defined by the relationship between the first preset time, the second preset time, and the response time. In this way, the time analysis module can accurately evaluate the number of hours in the response time, achieving the effect of objectively and accurately evaluating the user's response speed and familiarity with the test question.

[0016] The formula for the time analysis standard can be: where Tc: represents the time index; ts: represents the response time; t1: represents the first preset time; t2: represents the second preset time. Thus, this time analysis standard formula can quantify the user's response time to the test questions into a time index, achieving an objective and accurate assessment of the user's response speed and familiarity with the test questions.

[0017] The learning effectiveness evaluation system may further include a comprehensive analysis module, which calculates a comprehensive index through a comprehensive analysis standard defined by the relationship between the action index and the time index. Thus, through the aforementioned action index, time index, and comprehensive index, a quantitative standard can be provided to comprehensively evaluate the user's learning strengths, serving as a reference to guide the user in adjusting their learning direction.

[0018] The comprehensive analysis standard is defined as follows: When one of the action indicator and the time indicator is greater than a first threshold and the other is not less than a second threshold, the comprehensive indicator is defined as a first definition; when one of the action indicator and the time indicator is less than the second threshold and the other is not less than the first threshold, or when neither the action indicator nor the time indicator is greater than the first threshold nor less than the second threshold, the comprehensive indicator is defined as a second definition; when one of the action indicator and the time indicator is less than the second threshold and the other is not greater than the first threshold, the comprehensive indicator is defined as a third definition; the first threshold is greater than the second threshold, and the first, second, and third definitions are different from each other. Thus, by defining the first, second, and third definitions of the comprehensive indicator through the relationship between the aforementioned action indicator, time indicator, and comprehensive indicator and the first and second thresholds, the effect of objectively summarizing the user's comprehensive performance through quantitative methods can be achieved.

[0019] Each of the at least one test question has a preset answer, which consists of several characters; the learning effect evaluation system also has a similarity comparison module, which uses a similarity analysis method to analyze the degree of similarity between the user's answer and the preset answer of the at least one test question; the similarity analysis method includes at least one of a syntax tree analysis method, a suffix tree analysis method, a parse tree analysis method, a longest common word analysis method, and a word frequency analysis method. Thus, the similarity comparison module can be used to analyze the degree of similarity between the preset answer and the answer, as a reference for judging whether the user has plagiarized.

[0020] Wherein, when the action index is calculated to be 0, the similarity comparison module is activated to analyze the degree of similarity. In this way, the similarity comparison module can be activated only in specific situations to analyze the degree of similarity as a reference for judging whether the user has plagiarized, which can not only increase the credibility of judging whether plagiarism has occurred, but also reduce the system's operating resources. [Simplified Explanation of the Diagram]

[0050] [Figure 1] A schematic diagram of the system architecture of a preferred embodiment of the present invention.

Implementation Method

[0021] In order to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments of the present invention are described below in conjunction with the accompanying drawings; in addition, those symbols that are marked with the same symbols in different drawings are considered to be the same and their descriptions will be omitted.

[0022] Please refer to Figure 1, which is a preferred embodiment of the learning effect evaluation system of the present invention. It includes a teaching platform 1, an operation module 2, a recording module 3, an action analysis module 4 and / or a time analysis module 5, and preferably also includes a comprehensive analysis module 6 and / or a similarity comparison module 7.

[0023] The teaching platform 1 is executed on a computer and has a pre-established question database 11, which has at least one test question. Optionally, each of the at least one test question has a first preset number of actions a1 and a second preset number of actions a2, the value of the second preset number of actions a2 being greater than the value of the first preset number of actions a1; optionally, each of the at least one test question has a first preset time t1 and a second preset time t2, the value of the second preset time t2 being greater than the value of the first preset time t1; optionally, each of the at least one test question has a preset answer.

[0024] The operation module 2 is coupled to the teaching platform 1 to provide a user with one of the at least one test questions to answer, and to generate an answer result after the answering action is completed. The operation module 2 includes a display device and an input device. The display device provides the user with a visual interface, such as displaying information about the at least one test question and allowing the user to view their answer information. The input device provides the user with input information / to perform the answering action, and the input information can be displayed on the display device. In one example, the operation module 2 may be integrated with the computer of the teaching platform 1; in another example, the operation module 2 may be a separate device from the computer of the teaching platform 1, such as another computer.

[0025] In terms of the data structure, the answering behavior consists of at least one of at least one character and at least one instruction, or the answering behavior is an empty set; the answering result consists of at least one character, or the answering result is an empty set; the preset answer consists of multiple characters. The definition of a character includes a single character (which may be Chinese, English, or other languages), a single number, a single symbol, etc.; the instruction refers to a functional instruction in the operating system, especially a functional instruction in a text input and editing environment, such as a copy instruction (e.g., pressing the Ctrl key and C key simultaneously), a paste instruction (e.g., pressing the Ctrl key and V key simultaneously), or a delete instruction (e.g., pressing the delete key or backspace key) used when writing a program in a Windows™ operating system.

[0026] The recording module 3 initiates a recording program when the answering behavior is triggered to record the answering behavior and / or an answering time; the recording program ends after the answering behavior is completed, and an answering behavior record and / or an answering time record is obtained from the recording program. For example, the answering behavior may be triggered when the user answers for the first time (e.g., inputs any information through the input device), or when the user performs a triggering behavior (e.g., presses a simulated button with a start meaning displayed on the display device screen), at which point the recording program is initiated; the answering behavior may be completed when the user does not answer again within a preset time after triggering the answering behavior, or when the user performs an ending behavior (e.g., presses another simulated button with an ending meaning displayed on the display device screen). Preferably, the answering behavior record is a record of all characters and instructions entered by the user during the answering process, and can also be understood as a record of the answering behavior; the answering time record is the total time spent by the user during the answering process.

[0027] The motion analysis module 4 converts the response behavior record into a number of response actions as, and calculates a motion index Ac through a motion analysis standard. The motion analysis standard is defined by the relationship between the first preset number of actions a1, the second preset number of actions a2, and the number of response actions as. In detail, the motion analysis module 4 converts the response behavior record into the number of response actions through a conversion standard. In the conversion standard, each character and / or each instruction in the response behavior record is measured as a number of actions, and the number of response actions as is defined as the sum of such number of actions.

[0028] Preferably, the formula for the action analysis standard is as follows: where as represents the number of response actions, specifically used to represent that each character and / or each instruction entered by the user when answering a test question is measured as one action; a1 represents the first preset action number, specifically used to represent the minimum reasonable number of actions to complete the test question; a2 represents the second preset action number, specifically used to represent the maximum reasonable number of actions to complete the test question. Preferably, the value of the second preset action number a2 is more than 1.2 times the value of the first preset action number a1, and the adjustment of the numerical multiples between the aforementioned preset action numbers can be based on 0.1 times the value of the first preset action number a1 as the adjustment range.

[0029] Thus, when the number of answer actions as is less than the first preset number of actions a1, which represents the minimum reasonable number of actions, the calculated action index Ac is 0, indicating that the user has engaged in unreasonable answering behavior during the answering process, such as plagiarism by copying and pasting part or all of the answers, causing the value of the number of answer actions as to decrease significantly and unreasonably. When the number of answer actions as is not less than the first preset number of actions a1 and not greater than the second preset number of actions a2, the action index Ac can be calculated according to the formula of the above action analysis standard. Thus, the larger the value of the action index Ac, that is, the fewer answer actions the user has in the test question (but not less than the first preset number of actions a1), the higher the user's familiarity with the test question, and the more actions the user can use to complete the test question. In other words, through the calculation of the action indicator Ac, the user and / or an administrator of the teaching platform (such as a teacher) can control the user's familiarity with the test questions, especially the degree of the user's answering skills (especially in the field of programming), and can also detect whether the user has unreasonable answering behavior.

[0030] The time analysis module 5 obtains a response time from the response time record and calculates a time index Tc through a time analysis standard. The time analysis standard is defined by the relationship between the first preset time t1, the second preset time t2 and the response time ts.

[0031] Preferably, the formula for the time analysis standard is: where ts represents the response time, specifically the total time spent by the user on a test question; t1 represents the first preset time, specifically the minimum reasonable time to complete the test question; and t2 represents the second preset time, specifically the maximum reasonable time to complete the test question. The aforementioned times preferably include both the actual input response time and the dwell time during the thinking process. Preferably, the value of the second preset time t2 is at least 1.2 times the value of the first preset time t1, and the adjustment of the numerical multiples between the aforementioned preset times can be within an adjustment range of 0.1 times the value of the first preset time t1.

[0032] Thus, when the answering time ts is less than the first preset time t1, which represents the minimum reasonable time, the calculated time index Tc is 0, indicating that the user has engaged in unreasonable answering behavior during the answering process, such as plagiarism by copying and pasting some or all of the answers, causing the answering time ts to decrease significantly and unreasonably. When the answering time ts is not less than the first preset time t1 and not greater than the second preset time t2, the time index Tc can be calculated according to the formula of the above time analysis standard. Thus, the larger the value of the time index Tc, that is, the less time the user spends answering the test question (but not less than the first preset time t1), the higher the user's familiarity with the test question, and the less time they can use to complete the test question. In other words, through the calculation of the time index Tc, the user and / or an administrator of the teaching platform (such as a teacher) can control the user's familiarity with the test questions, especially the user's thinking and answering speed, and can also detect whether the user has unreasonable answering behavior.

[0033] The comprehensive analysis module 6 determines a comprehensive index P through a comprehensive analysis standard, which is defined by the relationship between the action index Ac and the time index Tc. Preferably, the comprehensive analysis standard can be defined as shown in Table 1 below.

[0034]

[0035] In other words, in the example of Table 1, the definition of this comprehensive analysis standard can be listed as follows:

[0036] (1) When one of the action index Ac and the time index Tc is greater than a first threshold (such as the value of 0.6 in Table 1) and the other is not less than a second threshold (such as the value of 0.3 in Table 1), the comprehensive index P is defined as a first value or a first definition; optionally, the first value may represent the connotation of the first definition.

[0037] (2) In the case where one of the action index Ac and the time index Tc is less than the second threshold and the other is not less than the first threshold, or in the case where neither the action index Ac nor the time index Tc is greater than the first threshold and not less than the second threshold, the comprehensive index P is defined as a second value or a second definition; optionally, the second value may represent the connotation of the second definition.

[0038] (3) In the case where one of the action index Ac and the time index Tc is less than the second threshold and the other is not greater than the first threshold, the comprehensive index P is defined as a third value or a third definition; optionally, the third value may represent the connotation of the third definition.

[0039] In detail, in the example of Table 1, when the first definition of the comprehensive indicator P is "high," it means that the user's performance in learning (especially for this test question) meets at least one of the following criteria: answering skills (corresponding to the action indicator Ac) and answering speed (corresponding to the time indicator Tc), which is at least excellent (corresponding to the first threshold), and neither is lower than a level that indicates the need for improvement (corresponding to the second threshold). Therefore, in this case, the comprehensive indicator P can reflect that the user has good learning outcomes. Thus, the user or the manager can understand the user's strengths in learning and how to improve through the aforementioned action indicator Ac, time indicator Tc, and comprehensive indicator P, to encourage and affirm the user's learning performance, and to provide the manager with a reference for guiding the user.

[0040] Similarly, in the example in Table 1, when the second definition of the comprehensive index P is "general," it means that the user has the following two situations in learning:

[0041] Scenario 1: While one of the answering skills or answering speed is below the required level, the other meets the highest level. In this case, although the user's speed or skill level is below the required level, the other level is above the excellent level. Based on the strengths of different users, the overall indicator P assesses the user's learning effectiveness as average. Therefore, the user or manager can use the aforementioned action indicator Ac, time indicator Tc, and overall indicator P to understand the user's learning strengths and areas needing improvement. This encourages and affirms the user's exceptional learning performance and reminds the user or manager to focus on areas for improvement.

[0042] Scenario Two: The user's answering skills and speed are neither below the required level nor below the excellent level. In this case, the comprehensive indicator P indicates that the user has average learning performance. Therefore, the user or the manager can understand that the user's overall learning performance is average through the aforementioned action indicator Ac, time indicator Tc, and comprehensive indicator P. This information can serve as a reference for the user to motivate themselves to improve, or as a reference for the manager to guide the user to further improve.

[0043] Similarly, in the example in Table 1, when the third definition of the comprehensive indicator P is "low," it means that at least one of the user's answering skills and answering speed in learning is below the level requiring improvement, and the other is not above the level of excellent performance. Therefore, in this case, the comprehensive indicator P can reflect that the user is lagging behind in learning performance. In this way, the user or the manager can understand the user's learning weaknesses and the potential areas for improvement through the aforementioned action indicator Ac, time indicator Tc, and comprehensive indicator P. This allows the user to recognize their weaknesses as an opportunity for improvement and reminds the manager to provide special guidance or care for such users.

[0044] Based on the above explanation of the comprehensive indicator P in the example of Table 1, it can be seen that through the calculation of the above action indicator Ac, time indicator Tc and comprehensive indicator P, the user's answering skills, answering speed and overall performance in learning can be displayed. In particular, it helps the user or manager to understand the user's strengths and weaknesses, so as to serve as a reference for reward, encouragement and other guidance programs, thereby improving the user's learning performance.

[0045] Wherein, the first threshold of the action indicator Ac and the time indicator Tc may not be equal, and the second threshold of the action indicator Ac and the time indicator Tc may also not be equal; in the case of corresponding to different test questions, the first threshold of the action indicator Ac and the time indicator Tc for each test question may be different from each other, and the second threshold of the action indicator Ac and the time indicator Tc for each test question may also be different from each other. In this way, the individual standards of each test question can be reflected, so as to achieve a more appropriate assessment of the user's learning effectiveness.

[0046] Wherein, the first threshold of the action indicator Ac and the time indicator Tc may not be equal, and the second threshold of the action indicator Ac and the time indicator Tc may also not be equal; in the case of corresponding to different test questions, the first threshold of the action indicator Ac and the time indicator Tc for each test question may be different from each other, and the second threshold of the action indicator Ac and the time indicator Tc for each test question may also be different from each other. In this way, the individual standards of each test question can be reflected, so as to achieve a more appropriate assessment of the user's learning effectiveness.

[0047] The similarity comparison module 7 compares the preset answer with the answer result using a similarity analysis method to analyze the degree of similarity between the user's answer result and the preset answer of the at least one test question; particularly when the user's action index Ac or time index Tc is 0, indicating that the user's number of answer actions is too few or the answering speed is too fast, the similarity comparison module 7 is triggered to compare the preset answer with the answer result to analyze the degree of similarity between the preset answer and the answer result, as a reference for judging whether the user has plagiarized. The similarity analysis method can employ at least one of the known code plagiarism detection techniques, preferably using syntax tree analysis, suffix tree analysis, parsing tree analysis, longest common word analysis, term frequency analysis, etc. Optionally, the similarity comparison module 7 can analyze whether the instructions used in the answer behavior record have a specific instruction and a usage scenario of the specific instruction, and compare it with the answer result to determine the relationship between the usage scenario of the specific instruction and the answer result, so as to serve as a reference for whether the user has plagiarized: preferably, the specific instruction refers to the user's use of "copy" and "paste" related instructions.

[0048] In summary, the learning effectiveness evaluation system of the present invention can evaluate a user's answering skills by analyzing action indicators through a motion analysis module, evaluate a user's answering speed by analyzing time indicators through a time analysis module, evaluate a user's overall learning effectiveness by analyzing comprehensive indicators based on motion and time indicators through a comprehensive analysis module, and analyze the similarity between the answer result and the preset answer through a similarity analysis module as a reference for whether the user has copied. Thus, the learning effectiveness evaluation system of the present invention can evaluate a user's learning effectiveness based on at least one module or at least one indicator, achieving the effect of effectively evaluating a user's learning effectiveness based on objective factors.

[0049] Although the present invention has been disclosed using the above preferred embodiments, it is not intended to limit the present invention. Any modifications and alterations made by those skilled in the art to the above embodiments without departing from the spirit and scope of the present invention shall still fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall include all changes within the meaning and equivalent scope of the appended claims. Furthermore, when the above examples or preferred embodiments can be combined, the present invention includes any combination of embodiments.

Claims

1. A learning effectiveness evaluation system, comprising: a teaching platform having a pre-established question database, the pre-established question database having at least one test question, each of the at least one test question having a first preset number of actions and a second preset number of actions, the value of the second preset number of actions being greater than the value of the first preset number of actions; an operation module providing one of the at least one test question to a user to perform a response action, and generating a response result upon completion of the response action; the response action is composed of at least one of at least one character and at least one instruction, or the response action is... The response is an empty set; the response result consists of at least one character, or the response result is an empty set; a recording module starts a recording program when the response behavior is triggered to record the response behavior; the recording program ends after the response behavior is completed, and a response behavior record is obtained from the recording program; and an action analysis module converts the response behavior record into a response action number, and calculates an action index through an action analysis standard, the action analysis standard being defined by the relationship between the first preset action number, the second preset action number, and the response action number.

2. As in request item 1, a learning effectiveness evaluation system, wherein, The action analysis module converts the response behavior record into the response action count through a conversion standard. In the conversion standard, each character and / or each instruction in the response behavior record is measured as an action count, and the response action count is defined as the sum of such action counts.

3. A learning effectiveness evaluation system as described in any of the requests in item 1, wherein, The formula for the action analysis standard is: where Ac: represents the action index; as: represents the number of response actions; a1: represents the first preset number of actions; a2: represents the second preset number of actions.

4. As in request item 1, a learning effectiveness evaluation system, wherein, Each of the at least one test question has a first preset time and a second preset time, the value of the second preset time being greater than the value of the first preset time; the recording program of the recording module records a response time to obtain a response time record; a time analysis module obtains the response time record as a response time and calculates a time index through a time analysis standard, the time analysis standard being defined by the relationship between the first preset time, the second preset time and the response time.

5. As in request item 4, a learning effectiveness evaluation system, wherein, The formula for this time analysis standard is: where Tc: represents the time index; ts: represents the response time; t1: represents the first preset time; t2: represents the second preset time.

6. The learning effectiveness evaluation system as requested in item 4 further includes a comprehensive analysis module, which calculates a comprehensive index through a comprehensive analysis standard, which is defined by the relationship between the action index and the time index.

7. A learning effectiveness evaluation system as requested in item 6, wherein, The comprehensive analysis standard is defined as follows: When one of the action indicator and the time indicator is greater than a first threshold and the other is not less than a second threshold, the comprehensive indicator is defined as a first definition; when one of the action indicator and the time indicator is less than the second threshold and the other is not less than the first threshold, or when neither the action indicator nor the time indicator is greater than the first threshold nor less than the second threshold, the comprehensive indicator is defined as a second definition; when one of the action indicator and the time indicator is less than the second threshold and the other is not greater than the first threshold, the comprehensive indicator is defined as a third definition; the first threshold is greater than the second threshold, and the first definition, the second definition, and the third definition are different from each other.

8. A learning effectiveness evaluation system as requested in any of items 1-7, wherein, Each of the at least one test question has a preset answer, which consists of several characters; the learning effect evaluation system also has a similarity comparison module, which uses a similarity analysis method to analyze the degree of similarity between the user's answer and the preset answer of the at least one test question; The similarity analysis method includes at least one of the following: syntax tree analysis, suffix tree analysis, parsing tree analysis, longest common word analysis, and word frequency analysis.

9. A learning effectiveness evaluation system as requested in item 3, wherein, in, Each of the at least one test question has a preset answer, which consists of several characters; the learning effect evaluation system also has a similarity comparison module, which uses a similarity analysis method to analyze the degree of similarity between the user's answer and the preset answer of the at least one test question; When the action index is calculated to be 0, the similarity comparison module is activated to analyze the degree of similarity.

Images