Knowledge and ability state evaluation method based on multi-source learning evidence and confidence

Abstract

The application discloses a knowledge and ability state evaluation method based on multi-source learning evidence and confidence, and belongs to the technical field of knowledge and ability state evaluation. The method acquires a knowledge dependency graph and a learning evidence record set, converts observation results of different sources into standardized evidence with evidence direction and intensity, and performs decay processing according to occurrence time; aggregates effective evidence directly associated, respectively determines knowledge mastery degree and state confidence, and uses the state confidence as a control condition for starting graph propagation reasoning; generates inference evidence only when the state confidence reaches a preset threshold, and merges and updates the state of each knowledge node according to the rule that the weight of direct evidence is higher than that of inference evidence; outputs a knowledge and ability state set, outputs a diagnosis task for an important node with low confidence and meeting a preset condition, and writes back a diagnosis result to trigger state updating, thereby realizing unified processing of multi-source heterogeneous evidence and optimization of state estimation.

Description

Technical Field

[0001] This invention relates to the field of knowledge and ability status assessment technology, and in particular to a method for assessing knowledge and ability status based on multi-source learning evidence and confidence levels. Background Technology

[0002] Computer-based learning systems typically need to estimate learners' mastery of various knowledge points based on their answers, assessments, video learning, and task completion in order to select subsequent learning content. Different sources of records have varying observation scales, reliability, and time freshness. Directly averaging results from different sources can easily lead to unreasonable equivalence between self-assessment records, low-quality behavior records, and formal assessment results. Furthermore, learning evidence also exhibits a weakening effect over time; the explanatory power of the same correct answer differs depending on whether it was given immediately or after a longer period. Different sources of evidence and different knowledge points also correspond to different rates of decay.

[0003] Existing technologies, by only storing the most recent score or cumulative accuracy rate, struggle to simultaneously express the strength of evidence, the freshness of evidence, and the credibility of the state. Knowledge nodes exhibit relationships such as prior, inclusion, similarity, and ambiguity. When a knowledge node lacks direct evidence, inference can be formed using the states of adjacent nodes; however, uncontrolled graph propagation may repeatedly amplify along loops or cause indirect inferences to overwrite the actual direct observations of the target node, resulting in unexplainable ability states. Existing solutions often conflate knowledge mastery and confidence with a single value. When there is no continuous feedback, insufficient data, or conflicting evidence, the system may directly determine a decline in knowledge mastery. In reality, this primarily reduces the observability of the state or the credibility of the judgment, not necessarily indicating a decline in the learner's knowledge level. Furthermore, some learning states only provide a final score, failing to retain the evidence records, inference source nodes, and propagation paths, making it difficult to explain the state's origin and hindering the orderly replacement of earlier inferences when subsequent direct evidence arrives. Summary of the Invention

[0004] To address the above issues, this invention standardizes multi-source evidence and weights it according to source reliability and time decay, and decouples the calculation of knowledge mastery and state confidence. It uses confidence control chart propagation and direct evidence to constrain inference, thereby optimizing the process from low-confidence identification to supplementary evidence and back to state update.

[0005] The technical solution is a knowledge and ability state assessment method based on multi-source learning evidence and confidence levels, which includes the following steps:

[0006] Step 1: Obtain the knowledge dependency graph and learning evidence record set corresponding to the target learner; the knowledge dependency graph includes knowledge nodes and edges connecting knowledge nodes with relational types; each evidence record in the learning evidence record set includes the target knowledge node, evidence source type, observation result, and occurrence time;

[0007] Step 2: Convert the observation results into standardized evidence with evidence direction and evidence strength according to the evidence source type, and then attenuate the evidence strength according to the time interval between the occurrence time and the state assessment time to obtain valid evidence;

[0008] Step 3: For each knowledge node, aggregate the valid evidence directly related to that knowledge node, determine the knowledge mastery level representing the current ability level based on the difference between the strength of positive and negative valid evidence, and determine the state confidence level representing the credibility of the knowledge mastery level based on the number of valid evidence and the degree of evidence consistency. The knowledge mastery level and the state confidence level are determined based on the difference in evidence strength and the sufficiency index of evidence, respectively.

[0009] Step 4: Only when the state confidence reaches a preset threshold, take the knowledge node as the source, determine the propagation direction according to the relationship type of the knowledge node connection edge, multiply each edge along the propagation direction by the corresponding attenuation coefficient, and generate inference evidence of adjacent knowledge nodes within a preset depth.

[0010] Step 5: Weight the directly related valid evidence according to the first weight and the inferred evidence according to the second weight. The first weight is greater than the second weight. Update the knowledge mastery and state confidence of each knowledge node according to the weighted and merged evidence, and mark the state type of the knowledge node as direct observation state, inferred state or unevaluated state.

[0011] Step 6: Output a knowledge ability state set containing knowledge node identifiers, knowledge mastery level, state confidence level, and state type. For knowledge nodes whose state confidence level is lower than the diagnostic threshold and meet preset conditions, generate diagnostic tasks and write the diagnostic results back to the learning evidence record set as new evidence to trigger state updates.

[0012] Furthermore, the relationship types include preconditioning, inclusion, similarity, and confusion; the evidence source types include at least two of the following: entrance diagnostic assessment, daily quiz, stage tests, video learning, planned task completion, spaced repetition, learning retelling, and learner self-assessment; each evidence source type is configured with a source reliability, wherein the source reliability of entrance diagnostic assessment, daily quiz, and stage tests is higher than the source reliability of learner self-assessment.

[0013] Furthermore, the evidence direction in step 2 is determined as positive or negative based on whether the observation result reaches the corresponding pass threshold; the evidence strength is determined jointly based on the degree to which the observation result deviates from the pass threshold, the reliability of the source, the completeness of the data, and the abnormal behavior detection result, wherein the abnormal behavior detection result is only used to reduce the evidence strength; the attenuation processing adopts attenuation parameters corresponding to the evidence source type and the memory attribute of the knowledge node, so that the earlier the evidence occurred, the lower the strength after attenuation.

[0014] Further, in step 3, the knowledge mastery is determined by the weighted difference between the strength of positive and negative valid evidence; the state confidence is determined by the number of valid evidence, the total strength of valid evidence, the degree of evidence consistency, and the time interval between the most recent evidence and the state assessment time. Among these, the number of valid evidence, the total strength of valid evidence, and the degree of evidence consistency are positive influencing factors, while the time interval between the most recent evidence and the assessment time is a negative influencing factor. The state confidence is obtained by weighted summing of the positive influencing factors and then subtracting the product of the negative influencing factors and a preset attenuation coefficient. The preset attenuation coefficient is used to adjust the influence of time obsolescence on the confidence. When the number of valid evidence does not reach a preset quantity threshold or the total strength of valid evidence does not reach a preset strength threshold, the original value of the knowledge mastery is maintained. If there is no original value, the knowledge mastery is set to a preset initial value and the state confidence is set to a value lower than the diagnostic threshold.

[0015] Furthermore, in step 4, the preceding relation propagates unidirectionally along the knowledge sequence, the inclusion relation propagates bidirectionally along the knowledge hierarchy, and the similar relation and easily confused relation propagate bidirectionally along the association direction; the strength of the evidence to be propagated is multiplied by the corresponding attenuation coefficient along the propagation path to obtain the inferred evidence strength of each propagation branch; when the inferred evidence strength is lower than the termination threshold, a knowledge node with direct and valid evidence is reached, a preset depth is reached, or an already visited knowledge node is repeatedly reached, the corresponding propagation branch is terminated.

[0016] Furthermore, in step 5, the first weight is greater than the second weight, and the sum of the two is 1; when there is direct and valid evidence that the target knowledge node occurred later than the inference evidence, the knowledge mastery is updated using the direct and valid evidence and the second weight is reduced; the inference state formed by the inference evidence is maintained as the inference state until direct and valid evidence with strength reaching the confirmation threshold is received and updated to the direct observation state.

[0017] Furthermore, the preset condition in step 6 is that the examination weight of the knowledge node reaches the target weight threshold or the knowledge node is located on the preceding path of an important knowledge node; the diagnostic task includes candidate knowledge nodes, diagnostic resources and triggering reasons, the difficulty of the diagnostic resources is determined according to the knowledge mastery, and the number of questions is determined according to the state confidence; after the diagnostic result is written back, steps 2 to 5 are re-executed.

[0018] Furthermore, the reliability of the sources of the entrance diagnostic assessment, daily answers, and stage tests decreases sequentially; the reliability of the source of learner self-assessment is lower than that of the entrance diagnostic assessment, daily answers, and stage tests; continuous lack of feedback records is only used to reduce the observability parameters of behavior and state confidence, and does not directly reduce the knowledge mastery; the observability parameters of behavior are stored independently of the knowledge mastery.

[0019] Furthermore, the attenuation parameter is obtained by multiplying the attenuation base value of the source type and the attenuation base value of the memory attribute; the attenuation base values ​​of the entrance diagnostic assessment, daily answers, and stage tests are higher than the attenuation base value of the learner's self-assessment; knowledge nodes are pre-classified and marked as long-term memory attributes or short-term memory attributes according to the forgetting characteristics of the knowledge content, and the attenuation base value of long-term memory attribute knowledge nodes is higher than the attenuation base value of short-term memory attribute knowledge nodes; the degree of evidence consistency is obtained by dividing the difference between the reliability weighted strength of positive evidence and the reliability weighted strength of negative evidence within a preset time window by the sum of the two and adding a preset positive number, with a value between negative one and positive one.

[0020] Furthermore, the edge-by-edge decay coefficient of easily confused relations is less than that of inclusion relations; there is a constraint relationship between the preset depth and the termination threshold: the depth and the termination threshold are positively correlated, the greater the depth, the higher the termination threshold, so that if the intensity of the propagation branch has decayed below the threshold before reaching the upper limit of the depth, it will terminate in advance; after the propagation is completed, the source knowledge node identifier and propagation path of the inferred evidence are retained as a source tracing record.

[0021] Due to the adoption of the above technical solutions, the present invention has the following advantages compared with the prior art;

[0022] 1. By mapping observation results to a standardized numerical range through conversion rules corresponding to the evidence source type, observation results from different sources such as entrance diagnostic assessments, daily quizzes, and stage tests are unified into two basic semantics: supporting or not supporting knowledge mastery. This eliminates the obstacle of mixing different dimensions of multi-source evidence. Simultaneously, the evidence strength is attenuated based on the time interval between the occurrence time and the state assessment time, and attenuation parameters are set separately for different evidence source types and knowledge node memory attributes. This ensures that high-reliability evidence from formal assessments and evidence of long-term memory attributes maintain high strength over a longer period, while low-reliability evidence from self-assessments and evidence of short-term memory attributes attenuates rapidly. This solves the problem in existing technologies where fixed attenuation windows cannot distinguish between different evidence types and knowledge characteristics. 2. Knowledge mastery reflects the learner's actual ability level, while state confidence reflects the credibility of the ability estimate. State confidence also acts as a control switch for initiating graph propagation inference; evidence is only allowed to propagate outward when the confidence reaches a threshold, preventing the propagation of unreliable evidence along the knowledge graph by knowledge nodes in low-confidence states, thus avoiding the spread of errors. During state merging, directly related valid evidence has a higher weight than inferred evidence, and outdated inferences are gradually eliminated as new direct evidence arrives. It also outputs diagnostic task signals and writes the diagnostic results back to trigger state updates, while retaining the source knowledge nodes and propagation paths of the inferred evidence as tracing records, ensuring that each knowledge capability state can be traced back to the source evidence and the complete propagation chain. (See attached diagram.)

[0023] Figure 1 This is a flowchart of the present invention.

[0024] Figure 2 This is a flowchart illustrating the standardization and time decay processing of multi-source learning evidence according to the present invention.

[0025] Figure 3 This is a flowchart for determining the knowledge mastery and state confidence levels in this invention.

[0026] Figure 4 This is a flowchart illustrating the generation of inference evidence based on a knowledge dependency graph, as described in this invention.

[0027] Figure 5 This is a flowchart illustrating the merging of execution states in this invention.

[0028] Figure 6 This is a flowchart illustrating the process of generating diagnostic tasks and writing back evidence for this invention. Detailed Implementation

[0029] The foregoing and other technical contents, features and effects of the present invention are described in conjunction with the appendix below. Figures 1 to 6 The detailed description of the embodiments will make this clear. All structural details mentioned in the following embodiments are based on the accompanying drawings.

[0030] Based on existing technology, in Embodiment 1, this embodiment provides a knowledge capability state assessment method based on multi-source learning evidence and confidence. The method involves a state estimation server triggering the process with the target learner identifier and state assessment time. First, it reads the knowledge dependency graph and learning evidence record set corresponding to the target learner. The knowledge dependency graph includes multiple knowledge nodes and relational edges connecting these nodes. Each evidence record in the learning evidence record set includes at least the target knowledge node, evidence source type, observation result, and occurrence time. The server converts the observation result into standardized evidence with evidence direction and strength according to the transformation rules corresponding to the evidence source type. Then, it attenuates the evidence strength based on the time interval between the occurrence time and the state assessment time to obtain valid evidence. This attenuation process reduces the influence of evidence from a more distant time period on the current state estimation, avoiding the problem of excessive influence from historical evidence due to fixed weights.

[0031] After obtaining valid evidence, the server aggregates all valid evidence directly related to each knowledge node. It calculates the knowledge mastery level, representing the current ability level, based on the difference between the strength of positive and negative valid evidence. Simultaneously, it calculates the state confidence level, representing the credibility of this mastery level, based on the number of valid evidences and the degree of evidence consistency. The knowledge mastery level and state confidence level are calculated separately and independently using different criteria. This allows the system to distinguish between two different dimensions: how much the learner has actually mastered and how reliable the judgment is. This solves the decision-making bias problem caused by the traditional method of conflating ability estimates with confidence levels.

[0032] The server only initiates knowledge graph propagation reasoning from a knowledge node when the state confidence reaches a preset threshold. The propagation direction is determined based on the relationship type of the connecting edges of the knowledge node. For each connecting edge along the propagation direction, the evidence strength is multiplied by the attenuation coefficient corresponding to that edge. Inference evidence for adjacent knowledge nodes is generated within a preset propagation depth. This makes the state confidence an intermediate control link connecting direct evidence calculation and graph propagation reasoning. Nodes with low state confidence will not propagate blindly even if they have a high level of knowledge mastery. This avoids the spread of unreliable evidence along the knowledge graph by knowledge nodes with low confidence, thus preventing the spread of errors.

[0033] The server weights and merges directly related valid evidence according to a first weight and inferred evidence according to a second weight, with the first weight being greater than the second weight. Based on the weighted and merged evidence, the server updates the knowledge mastery and state confidence of each knowledge node and marks the state type of the knowledge node as a directly observed state, an inferred state, or an unevaluated state. A directly observed state indicates that the state is mainly formed by the direct valid evidence of the target knowledge node itself. An inferred state indicates that the state is mainly formed by the propagation of the state along the knowledge node connection edges of adjacent knowledge nodes. An unevaluated state indicates that the existing evidence is insufficient to form a state that meets the configuration conditions. By distinguishing between the three state types, users and the system can trace the reasons for the formation of the state of each knowledge node.

[0034] The server ultimately outputs a knowledge capability state set containing knowledge node identifiers, knowledge mastery levels, state confidence levels, and state types. For knowledge nodes with state confidence levels below the diagnostic threshold and meeting preset conditions, a diagnostic task signal is generated. This diagnostic task signal includes candidate knowledge node identifiers, diagnostic resource identifiers, estimated diagnostic duration, and triggering reasons. The server writes the diagnostic results back to the learning evidence record set as new evidence to trigger the re-execution of steps 2 to 5. This forms a closed-loop link from low-confidence identification to supplementary evidence and back to state updates, solving the problems of traditional static assessments being unable to proactively supplement evidence and low-confidence states being unable to improve in the long term. In Example 2, based on Example 1, the relationship types in the knowledge dependency graph include prerequisite relationships, inclusion relationships, similarity relationships, and easily confused relationships. Among them, the prerequisite relationship indicates that one knowledge node is the prerequisite knowledge of another knowledge node; the inclusion relationship indicates that one knowledge node contains another knowledge node at the knowledge level; the similarity relationship indicates that two knowledge nodes are similar in content; and the easily confused relationship indicates that two knowledge nodes are easily confused by learners in terms of concept or expression. The setting of these four relationship types enables the knowledge dependency graph to simultaneously express four different semantic association forms: logical sequence, granular hierarchical structure, content similarity, and easily confused nature between knowledge, providing a graph structure foundation for subsequent differentiated dissemination based on different relationship types.

[0035] The types of evidence sources include at least two of the following: entrance diagnostic assessment, daily quizzes, phase tests, video learning, planned task completion, spaced repetition, learning retelling, and learner self-assessment. Among them, entrance diagnostic assessment is a systematic assessment conducted before learners begin learning; daily quizzes are exercises completed by learners during the learning process; phase tests are summative assessments conducted after completing a certain learning phase; video learning records learners' behavior of watching instructional videos; planned task completion records learners' completion of system-assigned tasks; spaced repetition records learners' completion of reviews with system prompts; learning retelling records learners' self-retelling of the learning content; and learner self-assessment is learners' subjective judgment of their own mastery.

[0036] Each type of evidence source is assigned a source reliability rating Rs, where s represents the evidence source type identifier, and R1 represents the source reliability of the admission diagnostic assessment. This indicates the reliability of the source of daily answers. This indicates the reliability of the source of the phase test. This indicates the reliability of the learner's self-assessment source, and the above source reliability satisfies... And each value takes a value in the interval [0,1].

[0037] In Example 3, during the standardization process, the server first maps the observation results to a preset standardized numerical range [0,1] based on the correct answer rate, task completion rate, review and recall results, or self-assessment scores to obtain standardized observation values. Then, it calculates the difference between the standardized observation values ​​and the corresponding pass thresholds, and determines whether the evidence direction is positive or negative based on whether the difference is non-negative. When the difference is non-negative, it indicates that the observation results have reached or exceeded the pass threshold, thus providing positive support for knowledge mastery. When the difference is negative, it indicates that the observation results have not reached the pass threshold, thus providing negative support for knowledge mastery.

[0038] The server further determines the initial evidence strength based on the absolute value of the difference, source reliability, data integrity, and abnormal behavior detection results. Here, 's' in the source reliability Rs represents the evidence source type identifier, 'c' represents the data integrity of the evidence record (c∈[0,1]), and 'a' represents the abnormal behavior detection result of the evidence record (a∈[0,1]). The initial evidence strength I is equal to the source reliability multiplied by the sum of the absolute value of the difference and the data integrity, multiplied by one minus the abnormal behavior detection value. -a), where Δ represents the difference between the standardized observation and the threshold. The initial evidence strength is limited to [0.01, 1.0], and the specific values ​​of each parameter are fixed values ​​preset before the system runs.

[0039] In the attenuation process, the server sets attenuation parameters according to the combination of evidence source type and knowledge node memory attribute, where v represents the knowledge node identifier and mᵥ represents the memory attribute type of knowledge node v. Each evidence source type is configured with a source type attenuation base value. Furthermore, the base value ranges from 0.85 to 0.995. The base value for formal assessment sources such as entrance diagnostic tests, daily quizzes, and stage tests is higher than that for self-assessment sources such as learner self-assessments. Each knowledge node is pre-marked as either a long-term memory attribute or a short-term memory attribute according to its content attributes, and the base value for the memory attribute of long-term memory attribute knowledge nodes is... For knowledge nodes with attributes higher than short-term memory, the product of the source type decay base value and the memory attribute decay base value is the decay parameter for that combination. .

[0040] The server determines the time decay coefficient based on the time interval t between the occurrence time of each learning evidence record and the state evaluation time, and the corresponding decay parameter. The time decay coefficient is equal to the decay parameter as a power function of the time interval, i.e. It monotonically decreases as the time interval increases and Finally, the product of the initial strength of evidence and the time decay coefficient is determined as the effective strength of evidence. .

[0041] Example 4: Knowledge Mastery The set of positive evidence is determined by the weighted difference between the strength of positive and negative valid evidence, where v represents the knowledge node identifier. The server performs a weighted sum of the strengths of all positive and negative valid evidence for that knowledge node according to the reliability of their respective sources. E_ represents the set of valid evidence with a positive direction at a knowledge node, and E_ represents the set of valid evidence with a negative direction at a knowledge node. The positive weighted result... Negative weighted results ,in Indicates the reliability of the source of evidence i, This represents the valid evidence strength of evidence i, and the knowledge mastery is updated using the difference between the positively weighted result and the negatively weighted result. Then, a clamp function is used to restrict it to a preset knowledge mastery interval [0,1], where the clamp function represents the operation of restricting the input value between a specified minimum and maximum value. ,in d represents the level of knowledge mastery before the update.

[0042] State confidence Based on the number of valid evidence Total strength of valid evidence (i) Evidence consistency parameters The time interval between the most recent evidence time and the state assessment time is used to determine the value. Among them, the number of valid evidences, the total strength of valid evidences, and the evidence consistency parameter are positive influencing factors, while the most recent evidence time interval is a negative influencing factor. The server multiplies the three positive influencing factors by their respective preset weight coefficients α, β, and γ, sums them up, and then subtracts the product of the most recent evidence time interval and the preset attenuation coefficient δ to obtain the median value. ,in Furthermore, all weight coefficients are greater than or equal to 0, δ≥0 is the preset attenuation coefficient, and t represents the time interval between the most recent evidence time and the state assessment time. Finally, this intermediate value is mapped to the range (0,1) through the sigmoid function σ(x,) to obtain the state confidence. ,in exp represents the natural constant An exponential function with base 0. T is the preset center parameter, and T is the preset scale parameter.

[0043] Each weighting coefficient is preset according to the application scenario, and the preset decay coefficient δ is used to adjust the degree of influence of time aging on confidence.

[0044] When the number of valid evidence The preset minimum quantity threshold has not been reached. Or the total strength of valid evidence The preset minimum intensity threshold was not reached. When the server maintains the original value of the knowledge mastery without updating it, if there is no original value, the knowledge mastery is set to the preset initial value of 0.5 and the state confidence is set to a value of 0.1, which is lower than the diagnostic threshold.

[0045] Example 5: The server obtains the state confidence score during the initiation of propagation. Reaching the preset propagation threshold The source knowledge nodes, The value ranges from 0.5 to 0.8.

[0046] The server determines the propagation direction and edge-by-edge attenuation coefficient based on the relationship type of the connection edges between the source knowledge node and adjacent knowledge nodes. Precedence relationships propagate unidirectionally along the knowledge sequence, inclusion relationships propagate bidirectionally along the knowledge hierarchy, and similarity relationships and easily confused relationships propagate bidirectionally along the association direction.

[0047] The server propagates valid evidence along the allowed propagation direction, starting from the source knowledge node. For each knowledge node connection edge encountered, the strength of the current evidence to be propagated is multiplied by the edge-by-edge attenuation coefficient corresponding to that edge. Where r represents the relation type, and the edge-by-edge decay coefficient takes a value between 0 and 1, representing the edge-by-edge decay coefficient of the preceding relation. The value ranges from 0.3 to 0.6, representing the edge-by-edge decay coefficient of the containment relationship. The value ranges from 0.5 to 0.8, representing the edge-by-edge decay coefficient for similarity relationships. The value ranges from 0.4 to 0.7, representing the edge-by-edge decay coefficient for easily confused relationships. The value ranges from 0.2 to 0.4.

[0048] The product of the attenuation coefficients of each side in the propagation path is the cumulative attenuation coefficient Λ of that propagation branch. This cumulative attenuation coefficient decreases monotonically with increasing propagation distance, thus inferring the strength of the evidence. Equal to the valid evidence strength of the source knowledge node Multiply by the cumulative attenuation coefficient Right now .

[0049] The server continuously assesses the termination conditions of each propagation branch during the propagation process, and determines the strength of inferential evidence after propagation. The branch terminates when the propagation threshold β is reached. The propagation process terminates when a knowledge node with direct and valid evidence is reached, and terminates when the propagation depth d reaches the preset propagation depth. End at time, The propagation process terminates when an already visited knowledge node is reached again. The preset propagation depth and propagation termination threshold are configured in pairs. hour , hour , hour .

[0050] Example 6: Before merging, the server obtains direct and inferred evidence of association between each knowledge node and sets a first merging weight for the valid evidence directly associated with the target knowledge node. A second pooling weight is set for inferential evidence. , , and .

[0051] During the merging process, the server performs a weighted summation of the two types of evidence according to their respective weights, and updates the knowledge mastery level based on the comprehensive evidence after the weighted summation. and state confidence During the update process, the server continuously monitors whether there is direct and valid evidence for the target knowledge node whose occurrence time is later than the time of the inference evidence generation. If so, it indicates that a new direct observation has arrived. The server uses this new direct and valid evidence to update the knowledge mastery and adjusts the merging weight of the inference evidence according to the principle that the greater the time difference, the greater the reduction in the second weight. The adjusted second weight is denoted as... And it remains unchanged after adjustment. ,in , To reduce the coefficient, As a preset freshness decay constant, exp represents an exponential function with the natural constant e as the base, and t represents the time difference between new direct evidence and inferred evidence.

[0052] Regarding state types, the inferred state formed by inferred evidence remains in the inferred state until the target knowledge node receives direct and valid evidence that is directly related to that node and whose evidence strength reaches a preset confirmation threshold κ. Only then is the inferred state updated to the direct observation state. and .

[0053] In Example 7, the server first filters out knowledge nodes whose state confidence C is lower than the diagnostic threshold δ from the knowledge capability state set. And the diagnostic threshold is lower than the transmission threshold. ,Right now and .

[0054] The server further determines whether these knowledge nodes meet the preset conditions. The preset conditions are that the examination weight ρ of the knowledge node reaches the target weight threshold σ or the knowledge node is located on the preceding path of the target knowledge node, where σ∈[0.6,0.8] and the examination weight ρ∈[0,1].

[0055] For candidate knowledge nodes that meet the preset conditions, the server determines their knowledge mastery level based on the candidate knowledge node. The diagnostic difficulty L is determined by the interval in which the diagnosis takes place, based on the state confidence level. Determine the number of diagnostic questions Q, where L = And L∈{1,2,3,4,5}, This indicates the rounding up operation. and The number of basic questions.

[0056] The server ultimately generates a list containing candidate knowledge node identifiers, diagnostic resource identifiers, and estimated diagnostic duration. The diagnostic task signal that triggers the problem is t(L), where t(L) is the average answer time for a single question at the corresponding difficulty level L, and it increases with L. If the diagnostic task is not performed by the learner, the server only reduces the behavioral observability parameter Oᵥ and the state confidence Cᵥ, without directly reducing the knowledge mastery. ne w=Oᵥ ol d-μ and μ∈(0,0.1).

[0057] After the learner completes the diagnostic task, the server will write the observation results, occurrence time and evidence quality parameters generated by the diagnostic resources into the learning evidence record set as a new learning evidence record, and trigger the re-execution of steps 2 to 5.

[0058] Example 8: The reliability of the three types of evidence sources—admission diagnostic assessment, daily quizzes, and periodic tests—decreases sequentially. .

[0059] Continuous no-feedback records refer to a sequence of records in which the learner does not produce any learning or response behavior under system prompts. These records are only used to reduce the observability parameter Oᵥ and the state confidence Cᵥ of the corresponding knowledge node, without directly reducing the knowledge mastery Pᵥ. The observability parameter is stored independently of the knowledge mastery and decreases monotonically with a preset step size μ∈(0,0.1).

[0060] Example 9: The attenuation parameter is obtained by multiplying the source type attenuation base value and the memory attribute attenuation base value, i.e. Where 's' represents the evidence source type identifier, Representing knowledge nodes Memory attribute type, Indicates the source type attenuation base value, This represents the base value for memory attribute decay.

[0061] The long-term and short-term memory attributes of knowledge nodes are pre-classified according to the forgetting characteristics of the knowledge content. Factual knowledge, basic knowledge, core concepts and other content that needs to be retained for a long time are marked as long-term memory attributes, while skill knowledge, time-sensitive knowledge, operation steps and other content that is easily forgotten are marked as short-term memory attributes.

[0062] The evidence consistency parameter Kᵥ is determined by a preset time window. The weighted strength of the reliability of positive evidence is obtained by dividing the difference between the weighted strength of the reliability of positive evidence and the weighted strength of the reliability of negative evidence by the sum of the two and adding a preset positive number θ. Let E₊ represent the set of positive evidence for the knowledge node within the set time window, E₋ represent the set of negative evidence for the knowledge node within the set time window, and Rᵢ represent the reliability of the source of evidence i. e (ᵢ) represents the effective strength of evidence i, θ=0.001 is a pre-set positive number, Kᵥ=(Σᵢ∈E₊ Rᵢ·I) e (ᵢ)-Σᵢ∈E₋ Rᵢ·I e (ᵢ)) / (Σᵢ∈E₊ Rᵢ·I e (ᵢ)+Σᵢ∈E₋ Rᵢ·I e (ᵢ)+θ), the length of the preset time window is determined according to the type of evidence source: 7 days for formal assessment sources, 3 days for daily question-and-answer sources, and 1 day for self-assessment sources. The calculated result is between [-1, 1].

[0063] Example 10: The edge-by-edge decay coefficient of easily confused relations is less than that of inclusion relations, i.e. .

[0064] Preset propagation depth There is a positive correlation between the constraint and the propagation termination threshold β, and the two are configured in pairs. hour hour hour .

[0065] After propagation is complete, the server retains the source knowledge node identifier, target knowledge node identifier, and propagation path of the inferred evidence as a traceability record. When new direct evidence arrives, the system can quickly identify which inferred states have been affected by the evidence and recalculate them accordingly.

[0066] In practical use, based on existing technology, this invention first constructs a knowledge dependency graph that includes preconditions, inclusion relationships, similarity relationships, and easily confused relationships. It also sets up multiple types of evidence sources, such as entrance diagnostic assessment, daily quizzes, stage tests, video learning, planned task completion, spaced review, and learner self-assessment. Each type of source is configured with a source reliability, and the reliability of formal assessment sources is higher than that of self-assessment sources.

[0067] Taking learner A's daily learning process on a certain online learning platform for upgrading from junior college to university as an example, after learner A registers, the system first loads a knowledge dependency graph of advanced mathematics for upgrading from junior college to university, pre-constructed by teaching experts. This knowledge dependency graph contains knowledge nodes such as "limit concept", "derivative concept", "derivative calculation", "definite integral concept", and "definite integral application". Preconditions, inclusion relationships, similarity relationships, and easily confused relationships are set between each knowledge node. For example, the "limit concept" node is connected to the "derivative concept" node with a precondition relationship, and the "derivative concept" node is connected to the "derivative calculation" node with an inclusion relationship. The above knowledge dependency... Figure 1 Once configured, it serves as the fixed graph structure for system operation. Learner A first completes the entrance diagnostic assessment, which covers all knowledge nodes. The system records knowledge nodes with an accuracy rate below 60% as negative evidence and those above 60% as positive evidence. Each learning evidence record is associated with the corresponding target knowledge node, evidence source type, observation result, occurrence time, and evidence quality parameters.

[0068] The server uses the state assessment time as a benchmark and processes each learning evidence record according to the transformation rules corresponding to its evidence source type. Specifically, for entrance diagnostic assessment records, the system determines the evidence direction based on the difference between the standardized observation and the passing threshold. If the correct answer rate is higher than the passing threshold, the evidence direction is positive; if it is lower, the evidence direction is negative. The initial evidence strength is determined jointly based on the absolute value of the difference, source reliability, data completeness, and abnormal behavior detection results. Evidence with higher source reliability, more complete data, and more normal behavior receives higher initial evidence strength. Abnormal behavior detection results are only used to reduce evidence strength. Subsequently, the server performs attenuation processing based on the time interval between the occurrence time of each learning evidence record and the state assessment time, significantly reducing the influence of older evidence on the current state estimation.

[0069] For each knowledge node, the server aggregates directly related valid evidence and calculates the knowledge mastery and state confidence levels. Taking the "derivative concept" knowledge node as an example, this node has three directly related learning evidence records: one is an entrance diagnostic assessment record, with the observation result being correct and the source reliability being the highest; one is a daily quiz record, with the observation result being correct and the source reliability being relatively high; and one is a learner's self-assessment record, with the self-assessment score below the passing threshold and the source reliability being the lowest. The server calculates positively weighted and negatively weighted results according to the reliability of their respective sources and updates the knowledge mastery level using the difference between the two. After calculation, the knowledge mastery level of this node is updated to 94%, indicating that learner A has a high level of mastery of the "derivative concept". The state confidence level is jointly determined by the number of valid evidence, the total strength of valid evidence, the evidence consistency parameter, and the time interval between the most recent evidence and the state assessment time. Among them, the number of valid evidence, the total strength of valid evidence, and the evidence consistency parameter are positive influencing factors, while the time interval between the most recent evidence and the state assessment time is a negative influencing factor. The calculated state confidence level of this node is 84%, indicating that the knowledge mastery is supported by sufficient and consistent valid evidence.

[0070] The server checks whether the state confidence of each knowledge node reaches a preset propagation threshold. When the state confidence of the "derivative concept" node reaches the propagation threshold, the system initiates knowledge graph propagation reasoning. Valid evidence is propagated downwards along the inclusion relationship between the "derivative concept" node and the "derivative calculation" node, and upwards along the preceding relationship between the "limit concept" node and the "derivative concept" node. For each knowledge node connection edge, the strength of the evidence to be propagated is multiplied by the edge-by-edge attenuation coefficient, generating inferred evidence for adjacent knowledge nodes. Simultaneously, the server continuously judges the termination conditions of each propagation branch during the propagation process. The branch terminates when the strength of the inferred evidence after propagation is lower than the propagation termination threshold, when a knowledge node with directly valid evidence is reached, when the propagation depth reaches a preset propagation depth, and when an already visited knowledge node is repeatedly reached. All four termination conditions must be met simultaneously for the corresponding propagation branch to terminate.

[0071] When merging states, the server assigns a first merging weight to valid evidence directly associated with the target knowledge node and a second merging weight to inferred evidence. The first merging weight is greater than the second weight, and their sum is one. For example, the "derivative calculation" knowledge node itself lacks sufficient direct evidence, but after receiving inferred evidence from the "derivative concept" node, its state confidence level increases to an acceptable level, and its state type is marked as an inferred state. When this node subsequently receives new direct and valid evidence, the system prioritizes using the new evidence to update the knowledge mastery level and reduces the weight of the inferred evidence. If the strength of the new evidence reaches the confirmation threshold, the state type is updated to a direct observation state, while the original inferred evidence and its propagation path are retained as a source tracing record.

[0072] The server ultimately outputs a knowledge ability status set covering all knowledge nodes, including the knowledge mastery level, status confidence level, status type (direct observation state, inferred state, or unevaluated state), recent evidence time, and evidence record citations for each knowledge node. The system automatically identifies knowledge nodes whose status confidence level is below the diagnostic threshold and meets preset conditions (exam weight reaches the target weight threshold or is located on the path preceding the target knowledge node). For example, the exam weight of the "definite integral application" knowledge node is 75%, reaching the target weight threshold, but its status confidence level is only 25%, below the diagnostic threshold; the system selects it as a candidate knowledge node. The server determines the diagnostic difficulty based on the knowledge mastery level range of the knowledge node and the number of diagnostic questions based on the status confidence level. The lower the knowledge mastery level, the higher the selected diagnostic difficulty to explore the learner's true ability boundaries; the lower the status confidence level, the larger the selected number of diagnostic questions to accumulate sufficient evidence to raise the confidence level to a reliable level. The system generates a diagnostic task signal containing candidate knowledge node identifiers, diagnostic resource identifiers, expected diagnostic duration, and triggering reasons, and pushes it to learner A's learning task list. After learner A completes the diagnostic task, the system writes the new observations, occurrence time, and evidence quality parameters generated by the diagnosis into the learning evidence record set, triggering a re-execution of the evaluation process and updating the knowledge ability status of the corresponding knowledge node.

[0073] The system completes a closed-loop evaluation process, from multi-source evidence acquisition, standardization and time decay, direct state calculation, confidence propagation, state merging to diagnostic output. Throughout this process, knowledge mastery and state confidence are calculated independently using different criteria. State confidence serves both as a quantitative indicator of the reliability of knowledge mastery and as a control switch to initiate knowledge graph propagation, thus optimizing the process from low-confidence identification to supplementary evidence and back to state updates.

[0074] The above description is a further detailed explanation of the invention in conjunction with specific embodiments, and it should not be considered that the specific embodiments of the invention are limited to this. For those skilled in the art to which this invention pertains and related fields, any extensions, operation methods, and data substitutions made based on the technical solution concept of this invention should fall within the protection scope of this invention.

Claims

1. A knowledge and ability state assessment method based on multi-source learning evidence and confidence, characterized in that, Includes the following steps: Step 1: Obtain the knowledge dependency graph and learning evidence record set corresponding to the target learner; The knowledge dependency graph includes knowledge nodes and edges connecting knowledge nodes with relational types; Each evidence record in the learning evidence record set includes the target knowledge node, the type of evidence source, the observation result, and the time of occurrence; Step 2: Convert the observation results into standardized evidence with evidence direction and evidence strength according to the evidence source type, and then attenuate the evidence strength according to the time interval between the occurrence time and the state assessment time to obtain valid evidence; Step 3: For each knowledge node, aggregate the valid evidence directly related to that knowledge node, determine the knowledge mastery level representing the current ability level based on the difference between the strength of positive and negative valid evidence, and determine the state confidence level representing the credibility of the knowledge mastery level based on the number of valid evidence and the degree of evidence consistency. The knowledge mastery level and the state confidence level are determined based on the difference in evidence strength and the sufficiency index of evidence, respectively. Step 4: Only when the state confidence reaches a preset threshold, take the knowledge node as the source, determine the propagation direction according to the relationship type of the knowledge node connection edge, multiply each edge along the propagation direction by the corresponding attenuation coefficient, and generate inference evidence of adjacent knowledge nodes within a preset depth. Step 5: Weight the directly related valid evidence according to the first weight and the inferred evidence according to the second weight. The first weight is greater than the second weight. Update the knowledge mastery and state confidence of each knowledge node according to the weighted and merged evidence, and mark the state type of the knowledge node as direct observation state, inferred state or unevaluated state. Step 6: Output a knowledge ability state set containing knowledge node identifiers, knowledge mastery level, state confidence level, and state type. For knowledge nodes whose state confidence level is lower than the diagnostic threshold and meet preset conditions, generate diagnostic tasks and write the diagnostic results back to the learning evidence record set as new evidence to trigger state updates.

2. The method according to claim 1, characterized in that, The relationship types include preconditions, inclusion, similarity, and confusion; the evidence source types include at least two of the following: entrance diagnostic assessment, daily quizzes, stage tests, video learning, planned task completion, spaced repetition, learning retelling, and learner self-assessment; each evidence source type is configured with a source reliability, wherein the source reliability of entrance diagnostic assessment, daily quizzes, and stage tests is higher than the source reliability of learner self-assessment.

3. The method according to claim 2, characterized in that, The evidence direction in step 2 is determined as positive or negative based on whether the observation result reaches the corresponding threshold. The evidence strength is determined by the degree to which the observation result deviates from the threshold, the reliability of the source, the completeness of the data, and the abnormal behavior detection results. The abnormal behavior detection results are only used to reduce the evidence strength. The attenuation process uses attenuation parameters corresponding to the evidence source type and the memory attribute of the knowledge node, so that the earlier the evidence occurred, the lower its strength after attenuation.

4. The method according to claim 1, characterized in that, The knowledge mastery level mentioned in step 3 is determined by the weighted difference between the strength of positive and negative valid evidence; the state confidence level is jointly determined by the number of valid evidence, the total strength of valid evidence, the degree of evidence consistency, and the time interval between the most recent evidence and the state assessment time. Among them, the number of valid evidence, the total strength of valid evidence, and the degree of evidence consistency are positive influencing factors, and the time interval between the most recent evidence is a negative influencing factor. The state confidence level is obtained by weighted summing of the positive influencing factors and then subtracting the product of the negative influencing factors and a preset attenuation coefficient. The preset attenuation coefficient is used to adjust the degree of influence of time obsolescence on the confidence level. When the number of valid evidences does not reach the preset quantity threshold or the total strength of valid evidence does not reach the preset strength threshold, the original value of the knowledge mastery is maintained. If there is no original value, the knowledge mastery is set to the preset initial value and the state confidence is set to a value lower than the diagnostic threshold.

5. The method according to claim 2, characterized in that, In step 4, the preceding relation propagates unidirectionally along the knowledge sequence, the inclusion relation propagates bidirectionally along the knowledge hierarchy, and the similar relation and easily confused relation propagate bidirectionally along the association direction. The strength of the evidence to be propagated is multiplied by the corresponding attenuation coefficient along the propagation path to obtain the inferred evidence strength of each propagation branch. When the inferred evidence strength is lower than the termination threshold, a knowledge node with direct and valid evidence is reached, a preset depth is reached, or an already visited knowledge node is repeatedly reached, the corresponding propagation branch is terminated.

6. The method according to claim 1, characterized in that, In step 5, the first weight is greater than the second weight, and the sum of the two is 1; when there is direct and valid evidence that the target knowledge node occurred later than the inference evidence, the knowledge mastery is updated using the direct and valid evidence and the second weight is reduced; the inference state formed by the inference evidence is maintained as the inference state until direct and valid evidence with strength reaching the confirmation threshold is received and updated to the direct observation state.

7. The method according to claim 1, characterized in that, The preset conditions in step 6 are that the examination weight of the knowledge node reaches the target weight threshold or the knowledge node is located on the preceding path of an important knowledge node; the diagnostic task includes candidate knowledge nodes, diagnostic resources and triggering reasons, the difficulty of the diagnostic resources is determined according to the knowledge mastery, and the number of questions is determined according to the state confidence; after the diagnostic results are written back, steps 2 to 5 are re-executed.

8. The method according to claim 2, characterized in that, The reliability of the sources of the entrance diagnostic assessment, daily answers, and stage tests decreases in that order; the reliability of the source of learner self-assessment is lower than that of the entrance diagnostic assessment, daily answers, and stage tests; continuous lack of feedback records is only used to reduce the observability parameters of behavior and state confidence, and does not directly reduce the knowledge mastery; the observability parameters of behavior are stored independently of the knowledge mastery.

9. The method according to claim 3, characterized in that, The attenuation parameter is obtained by multiplying the attenuation base value of the source type and the attenuation base value of the memory attribute; the attenuation base values ​​of the entrance diagnostic assessment, daily answers, and stage tests are higher than the attenuation base value of the learner's self-assessment; knowledge nodes are pre-classified and marked as long-term memory attributes or short-term memory attributes according to the forgetting characteristics of the knowledge content, and the attenuation base value of long-term memory attribute knowledge nodes is higher than the attenuation base value of short-term memory attribute knowledge nodes; the degree of consistency of evidence is obtained by dividing the difference between the reliability weighted strength of positive evidence and the reliability weighted strength of negative evidence within a preset time window by the sum of the two and adding a preset positive number, with a value between negative one and positive one.

10. The method according to claim 5, characterized in that, The edge-by-edge decay coefficient of easily confused relationships is less than that of inclusion relationships; there is a constraint relationship between the preset depth and the termination threshold: the depth and the termination threshold are positively correlated, the greater the depth, the higher the termination threshold, so that if the intensity of the propagation branch has decayed below the threshold before reaching the upper limit of the depth, it will terminate in advance. After the propagation is completed, the source knowledge node identifier and propagation path of the inferred evidence are retained as a source tracing record.

Images