A method for improving the teaching effect of simulator operation through random testing

Abstract

The application provides a method for improving the teaching effect of simulator operation by means of examination, and is used for improving the teaching effect. Specifically, a directed acyclic graph is used to describe the knowledge system in the teaching of simulator operation. The knowledge system is composed of multiple knowledge paths, each knowledge path is composed of theoretical knowledge nodes and skill nodes which appear alternately, and the direction of the knowledge path represents the learning sequence. Each node contains multiple test questions. The examination strategy of this time is determined according to the importance of the node and the examination state of the previous node. After completing this examination, the examination state of all nodes is updated to prepare for the next examination strategy. The application can adjust the examination strategy in multiple rounds of examination according to the answer results of the learners, the importance of the knowledge points and the historical examination situation, continuously eliminate the test questions which cannot effectively detect the mastery of the knowledge points through the test question evaluation, optimize the test question library, enhance the pertinence of the teaching, and thus improve the efficiency of the teaching of simulator operation.

Description

Technical Field

[0001] This invention belongs to the field of teaching technology, specifically relating to a method for improving the teaching effectiveness of simulator operation through random sampling. Background Technology

[0002] Simulator-based instruction is commonly used, but it cannot cover all knowledge points at once when teaching theory and practical skills, making it difficult to assess learners' mastery and resulting in suboptimal learning outcomes. Existing simulator-based instruction cannot dynamically adjust testing content based on learners' actual progress, thus hindering its effectiveness. Therefore, a method using random sampling to improve learner learning outcomes is needed. Summary of the Invention

[0003] To address the problem that simulator operation instruction cannot be improved through random sampling, this invention proposes a method to enhance simulator operation instruction through random sampling. This invention can adjust the sampling strategy dynamically based on learners' answers, the importance of knowledge points, and historical sampling data during multiple rounds of sampling. It can also dynamically change the content of subsequent sampling tests, addressing shortcomings in earlier instruction and enhancing the relevance of teaching, thereby improving the efficiency of simulator operation instruction.

[0004] This invention is achieved through the following technical solution:

[0005] A directed acyclic graph is used to describe the knowledge system in simulator operation teaching; the knowledge system consists of multiple knowledge paths, each of which alternates between theoretical knowledge nodes and skill nodes, and the direction of the knowledge path represents the order of learning; each node contains multiple test questions;

[0006] The sampling strategy for this test is determined based on the importance of the nodes and the previous node sampling status.

[0007] After completing this sampling test, update the sampling status of all nodes to prepare for the next sampling strategy.

[0008] Furthermore,

[0009] The importance of a node is equal to the number of edges associated with that node in the knowledge system, divided by the total number of edges in the knowledge system.

[0010] Furthermore,

[0011] The node sampling status includes whether the node was selected and the sampling result of the node.

[0012] Furthermore,

[0013] The sampling strategy for this test includes,

[0014] First, select the nodes that were not selected in the previous sampling;

[0015] Then extract nodes with high importance and low results in the previous sampling.

[0016] Finally, other nodes are randomly selected.

[0017] Furthermore,

[0018] The sampling results of nodes include the sampling results of selected nodes and the sampling results of unselected nodes. The sampling results of selected nodes are the actual assessment results, while the sampling results of unselected nodes are calculated from the sampling results of selected nodes in the current sampling and the historical sampling results of unselected nodes.

[0019] Furthermore,

[0020] The process for extrapolating the sampling results of nodes that were not selected is as follows:

[0021] Get the sampling results of the predecessor nodes of the nodes that were not selected in the current sampling;

[0022] The sampling result of the leader node multiplied by the ratio P between the unsampled nodes and the leader node. 正 The first estimated value Est1 is obtained;

[0023] Get the sampling results of the next level node of the node that was not selected in the current sampling;

[0024] The sampling result of the next-level node is multiplied by the ratio P between the unsampled nodes and the next-level nodes. 反 Thus, the second estimate Est2 is obtained;

[0025] Retrieve the historical sampling results of nodes that were not selected in the current sampling;

[0026] The first estimated value Est1, the second estimated value Est2, and the historical sampling results of the node that was not selected are weighted and summed to obtain the sampling result of the node that was not selected in this sampling; this sampling result will be updated to the historical sampling result of the corresponding node.

[0027] Furthermore,

[0028] Proportional relationship P 正 It is equal to the ratio of the sampling result of node i in the historical sampling to the sampling result of its predecessor node in the historical sampling.

[0029] Proportional relationship P 反 It equals the ratio of the sampling result of node i in the historical sampling to the sampling result of its next-level node in the historical sampling.

[0030] Furthermore,

[0031] In the process of extrapolating the sampling results of nodes that were not selected, if their predecessor nodes or next-level nodes were also not selected, the calculation is performed only based on the predecessor nodes or next-level nodes that have sampling results.

[0032] When a node belongs to multiple knowledge paths, the node has one or more preceding nodes and one or more next-level nodes. In this case, when calculating the sampling results of nodes that were not selected, all preceding nodes and all next-level nodes need to be included.

[0033] Beneficial effects

[0034] 1) The knowledge system is comprehensively depicted using a directed acyclic graph. The theories and skills used in simulator operation are organized into an ordered set of interrelated elements to form an ordered relationship between theoretical knowledge points and skill points, which corresponds to the achievement of learners' ability goals and lays the foundation for accurately evaluating teaching effectiveness.

[0035] 2) By evaluating the sampling results of the selected nodes, the sampling results of the unselected nodes are assessed, thus achieving a comprehensive estimate of the mastery of the complete knowledge system.

[0036] 3) Continuously update historical sampling records and dynamically adjust the proportion P used to characterize the correlation between knowledge mastery among nodes. 正 and P 反 This helps to accurately reflect the actual teaching effectiveness;

[0037] 4) For nodes that are not selected, the previous node, the next level node, and the historical sampling results are used to estimate them, which fully follows the continuity and relevance of knowledge point learning and makes the estimation results more accurate.

[0038] 5) Based on an accurate assessment of the existing teaching effectiveness, and taking into account the importance and comprehensiveness of the knowledge points, the content of subsequent teaching sampling tests is dynamically adjusted, which enhances the relevance of the tests and improves the efficiency of teaching.

[0039] 6) Through test evaluation, continuously eliminate test questions that cannot effectively test the mastery of knowledge points, optimize the test question bank, enhance the pertinence of teaching, and thus optimize the efficiency of teaching using simulators. Attached Figure Description

[0040] Figure 1 A directed acyclic graph representation of a knowledge system;

[0041] Figure 2 The implementation flowchart of the present invention. Detailed Implementation

[0042] This method consists of: knowledge system description, knowledge system evaluation, and development of knowledge point sampling strategies.

[0043] (I) Description of the Knowledge System

[0044] The purpose of describing the knowledge system is to organize the theories and skills used in simulator operation into an interconnected and ordered set, so as to form an ordered relationship between theoretical knowledge points and skill points, corresponding to the achievement of learners' ability goals.

[0045] In the teaching content using simulator operation, the relationship between theoretical knowledge points and skill points can be described as a directed acyclic graph (DAG). The teaching design for simulator operation follows a gradual and alternating progression from theory to practice. Therefore, nodes representing theoretical knowledge and nodes representing skill points alternate in the DAG, with the initial set of nodes always representing theoretical knowledge and all terminal nodes representing skill points. The path from the starting point to a skill point represents a "knowledge" item, corresponding to the learning pattern of operational skills. Each node corresponds to multiple test questions, as shown in [the original text]. Figure 1 As shown, rectangles represent knowledge nodes, and circles represent skill nodes. The vectors from N11 to T21 represent the theoretical basis of knowledge N11 and skill T21.

[0046] Mastering the theoretical foundation is a prerequisite for mastering subsequent skills. However, because there is a gap between theoretical and practical understanding, the ability to infer the mastery of subsequent skills from the mastery of prerequisite knowledge carries a certain degree of probability. Figure 1 As shown, Using the first piece of knowledge N11 as a premise, we can estimate the mastery level of the first skill T21 that follows.

[0047] (II) Knowledge System Assessment

[0048] After the initial assessment, data on learners' responses to all questions will be obtained. This data can be used as a sample to evaluate the current batch of trainees' mastery of simulator operation knowledge and skills. The set of all samples constitutes the assessment history, denoted by mem. The historical sample is updated after each assessment. mem represents the trainers' prior estimate of the learners' performance obtained through simulator operation assessments. Figure 1 In the middle, mem(N) is used 11 ) represents prior knowledge of knowledge point N11, that is, the probability that the learner will answer correctly when N11 is drawn from a past exam.

[0049] The proportional relationship P between the sampling results of node T21 and its predecessor node N11 正 It equals the ratio of the sampling result of node T21 in the historical sampling to the sampling result of its predecessor node N11 in the historical sampling, where P 正As shown in the figure

[0050] The proportional relationship P between the sampling results of node T21 and the sampling results of its next-level node N31 反 It equals the ratio of the sampling result of node T21 in the historical sampling to the sampling result of its next-level node N31 in the historical sampling, where P 反 As shown in the figure

[0051] After completing a random test, first calculate the learner's correct answer rate for the selected knowledge points. Then, using the selected knowledge points, calculate the percentage based on the proportional relationship P. 正 P 反 Estimate the sampling results for knowledge points that were not selected in this sampling test.

[0052] Estimating the sampling results of unselected knowledge points based on the sampling results of preceding and subsequent knowledge points will both result in deviations from the actual sampling results of unselected knowledge points. Furthermore, historical sampling results can also roughly indicate the mastery of corresponding knowledge points. For example, students usually grasp simple knowledge points relatively easily, while they typically struggle with difficult ones. Therefore, for the unselected node T21, the first estimated value obtained from its preceding node N11... The second estimate obtained from its next-level node N31 The sampling result mem(T21) of T21 in this sampling is obtained by weighted summation of the historical sampling results mem(T21) of T21. new The historical sampling result mem(T21) of T21 is updated to mem(T21). new .

[0053] The formula for the above process is as follows: Given that the results of this sampling N11 and N31 are Test(N11) and Test(N31) respectively, two new estimates of T21 are obtained based on the sampling results: and The estimated results of the sampling test for skill knowledge point T21 are then updated as follows:

[0054]

[0055] Where α+β+γ=1, the calculation of (2) is performed on all untested nodes to update the historical test records.

[0056] If its predecessor node or next-level node is also not sampled, then the calculation is performed only based on the predecessor node or next-level node that has been sampled. When a node belongs to multiple knowledge paths, the node has one or more predecessor nodes and one or more next-level nodes. In this case, when calculating the sampling results of the unsampled node, all predecessor nodes and all next-level nodes need to be included.

[0057] (III) Develop a knowledge point sampling strategy

[0058] To improve teaching efficiency, it is necessary to develop effective strategies for sampling knowledge points. Figure 1 Based on the established knowledge system structure, the sampling of learners' learning outcomes is usually based on test questions, which are divided into theoretical test questions and practical test questions, respectively belonging to the corresponding knowledge points and skill points in the knowledge system.

[0059] Based on the characteristics of simulator-based teaching, the question selection scheme is designed according to two indicators: the importance of knowledge points and the status of the previous question selection.

[0060] 1. Importance of knowledge points

[0061] The importance of a knowledge point is calculated from the number of edges associated with a given node in the knowledge system. If the set of all edges is E, then the total number of edges in the knowledge system is card(E). For a given node n, the number of edges associated with it is adj(n), then the score based on the importance of the knowledge point is adj(n) / card(E).

[0062] 2. Status of the previous question selection

[0063] The node sampling status includes whether a node has been selected and the sampling result. When selecting questions from the knowledge system, attention should be paid to the comprehensiveness of the questions in the sampling, so as to fully cover the "knowledge" in the knowledge system. At the same time, the importance of the nodes and the students' mastery level should be considered. To this end, a rule was designed to dynamically adjust the question selection strategy based on the importance of the knowledge points and the previous sampling status.

[0064] The sampling strategy for this test includes,

[0065] First, select the nodes that were not selected in the previous sampling;

[0066] Then, nodes with high importance and low previous sampling results are selected. The sampling results of nodes include the sampling results of selected nodes and the sampling results of unselected nodes.

[0067] Finally, other nodes are randomly selected.

[0068] In addition, the focus of the question selection strategy can be adjusted by using weighting coefficients, or by using other existing technical means, depending on the actual situation.

[0069] This completes the improvement of teaching effectiveness.

[0070] (iv) Test item evaluation and optimization of the question bank

[0071] To further improve teaching efficiency, it is necessary to evaluate test questions and optimize the question bank based on the evaluation results. The quality of test question creation is limited by the subjective ability of educators. Test questions with different learner performances should not be categorized into the same knowledge point. The purpose of test question evaluation is to assess the quality of test questions for the current knowledge point. When comparing test questions from the same group of learners, well-designed test questions should result in similar answer scores.

[0072] The test evaluation and optimization of the question bank further includes, for example, the question set for node T21 consists of 200 questions, and the mean (Mean(Item)) of the accuracy rate of these 200 questions in the sampling history is calculated. T21 ) and variance Var(Item T21 ), calculate the historical accuracy rate of each question in these 200 questions. For example, if question 10 was selected 8 times in the historical sampling, the accuracy rate obtained from the answers to these 8 questions is within the range of {Mean(Item T21 )-2Var(Item T21 Mean(Item) T21 )+2Var(Item T21 If the range is defined as 10, then question 10 is an acceptable question; otherwise, it is replaced. This process is repeated for all questions across all nodes to optimize the question bank. The range can be adjusted based on actual circumstances.

[0073] The evaluation and optimization of the question bank can be carried out after each sampling test, or after a certain number of sampling tests.

[0074] The implementation process of the method of the present invention is as follows: sequentially perform knowledge system description, knowledge point sampling test, knowledge system evaluation, test question evaluation and optimization of question bank.

[0075] Step 1: Constructing a teaching knowledge system. Based on the teaching syllabus, organize the knowledge points according to theory and skills, use a directed acyclic graph as the method for describing the knowledge system, and stratify the organized knowledge points according to the order from theoretical knowledge to operational skills.

[0076] Step 2: Improve the test question bank resources. For each knowledge point in the knowledge system, set up test questions belonging to this knowledge point. Each newly added test question should be assigned an identifier, and the database should record the sampling records of this test question, including the sampling frequency and the number of correct answers.

[0077] Step 3: Assessment Implementation, i.e., knowledge point sampling test. A batch of learners will be assessed on their operational skills using a simulator. Following the standardized knowledge point sampling test method, a batch of test questions will be selected and sent to the simulator to be tested. Learners will answer the questions, and the results will be recorded.

[0078] After each sampling is completed, the number of batches is evaluated. If the batch requirement is not met, the sampling of the next batch of test questions will continue.

[0079] Step 4: Knowledge System Assessment. After the learners in this batch have completed their assessments, the prior knowledge will be updated based on the assessment results. The specific update formula is shown in (2).

[0080] Step 5: Evaluate and optimize the question bank. According to the settings, the questions will be evaluated and the question bank will be optimized after every 20 sampling tests.

[0081] Thus, this method was applied to the teaching and assessment of simulator operation, enabling the evaluation of the effectiveness of sampling tests across the entire knowledge system and improving the efficiency of skills teaching using simulators.

Claims

1. A method for improving the teaching effectiveness of simulator operation through sampling tests, characterized in that: A directed acyclic graph is used to describe the knowledge system in simulator operation teaching; the knowledge system consists of multiple knowledge paths, each of which alternates between theoretical knowledge nodes and skill nodes, and the direction of the knowledge path represents the order of learning; each node contains multiple test questions; The sampling strategy for this test is determined based on the importance of the nodes and the previous node sampling status. In this context, node importance is equal to the number of edges associated with the node in the knowledge system, divided by the total number of edges in the knowledge system. The node sampling status includes whether the node was selected and the sampling result of the node; After completing this sampling test, update the sampling status of all nodes to prepare for the next sampling strategy. The sampling results of nodes include the sampling results of selected nodes and the sampling results of unselected nodes. The sampling results of selected nodes are the actual assessment results, while the sampling results of unselected nodes are calculated from the sampling results of selected nodes in the current sampling and the historical sampling results of unselected nodes. The process for extrapolating the sampling results of nodes that were not selected is as follows: Get the sampling results of the predecessor nodes of the nodes that were not selected in the current sampling; The sampling result of the leader node multiplied by the ratio P between the unsampled nodes and the leader node. 正 The first estimated value Est1 is obtained; Get the sampling results of the next level node of the node that was not selected in the current sampling; The sampling result of the next-level node is multiplied by the ratio P between the unsampled nodes and the next-level nodes. 反 Thus, the second estimate Est2 is obtained; Retrieve the historical sampling results of nodes that were not selected in the current sampling; The first estimated value Est1, the second estimated value Est2, and the historical sampling results of the node that was not selected are weighted and summed to obtain the sampling result of the node that was not selected in this sampling; this sampling result will be updated to the historical sampling result of the corresponding node.

2. The method for improving simulator operation teaching effectiveness through sampling tests according to claim 1, characterized in that: Proportional relationship P 正 It is equal to the ratio of the sampling result of node i in the historical sampling to the sampling result of its predecessor node in the historical sampling. Proportional relationship P 反 It equals the ratio of the sampling result of node i in the historical sampling to the sampling result of its next-level node in the historical sampling.

3. The method for improving simulator operation teaching effectiveness through sampling tests according to claim 2, characterized in that: In the process of extrapolating the sampling results of nodes that were not selected, if their predecessor nodes or next-level nodes were also not selected, the calculation is performed only based on the predecessor nodes or next-level nodes that have sampling results. When a node belongs to multiple knowledge paths, the node has one or more preceding nodes and one or more next-level nodes. In this case, when calculating the sampling results of nodes that were not selected, all preceding nodes and all next-level nodes need to be included.

4. A method for improving simulator operation teaching effectiveness through sampling tests according to claim 1 or 3, characterized in that: Further optimization of the question bank was carried out after sampling, specifically including calculating the question set Item of the i-th node. i Mean(Item) of the accuracy rate in the sampling history i ) and variance Var(Item i If the set of questions is Item, then the set of questions is Item. i The accuracy rate of any question in the test is within {Mean(Item i )-2Var(Item i Mean(Item) i )+2Var(Item i The questions in the interval are considered acceptable, and the rest are replaced. The question bank is optimized by traversing all nodes.

Images