Learning evaluation methods and systems based on multidimensional weighted scoring and multi-level credibility
By constructing the multidimensional weighted diagnostic scoring model MDW-DSM and the multilevel bias correction confidence assessment model MBL-BC, the problems of unsystematic collection and quantification of process indicators and lack of credibility of diagnostic results in existing educational assessments are solved. This achieves full-dimensional and credible quantitative diagnosis of personalized learning problems, and improves diagnostic accuracy and credibility through a self-evolving closed-loop system optimization algorithm.
Patent Information
- Application Number
- CN202610974739.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-11-17
AI Technical Summary
The existing education assessment system relies on periodic written test scores, which cannot capture in real time the process-related influencing factors such as students' learning habits, motivation, concentration, and emotional fluctuations. Furthermore, it lacks standardized quantitative models and credibility assessment mechanisms, resulting in highly subjective diagnostic results, a lack of systematic collection and quantification of process indicators, and static and fixed algorithm models that cannot be optimized independently.
A multidimensional weighted diagnostic scoring model, MDW-DSM, is constructed. Standardized scoring is achieved through a response quality function, consistency correction coefficient, and knowledge gap amplification coefficient. Combined with a cross-dimensional etiology matching algorithm and probabilistic iterative etiology identification, a multi-level bias-corrected confidence assessment model, MBL-BC, is constructed to achieve standardized, automated, and quantifiable personalized learning diagnosis across the entire process. The algorithm is optimized through a self-evolving closed-loop system.
It achieves comprehensive and personalized learning problem diagnosis, accurately distinguishes between explicit and implicit learning causes, ensures the credibility of diagnostic results, and continuously optimizes the model through a self-evolutionary closed-loop mechanism. This solves the problems of strong subjectivity, lack of quantitative standards, and static model solidification in traditional assessments, thereby improving diagnostic accuracy and credibility.
Smart Images

Figure CN123118312A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of educational informatization technology, and in particular to the fields of artificial intelligence and educational measurement technology; specifically, it relates to a learning assessment method and system based on multidimensional weighted scoring and multi-level credibility. Background Technology
[0002] Traditional education assessment systems have long relied on periodic written test scores as the sole core basis for judging students' learning status. Test scores are a result-lagging indicator and cannot capture in real time the process-related influencing factors such as students' learning habits, learning motivation, concentration, emotional fluctuations, deficiencies in learning methods, and interference from the family environment.
[0003] With the implementation of artificial intelligence policies in the education industry and the iteration of adaptive learning technology, a number of domestic and foreign adaptive assessment products have emerged on the market. Currently, global adaptive learning technology is mainly divided into three technical routes:
[0004] The first category is represented by traditional adaptive algorithms from Europe and the United States. It relies on the data of incorrect answers to knowledge points to build a knowledge graph. It only focuses on the data of correct and incorrect answers to subject knowledge points and ignores the impact of non-intellectual factors (motivation, emotion, concentration) on learning outcomes.
[0005] The second category is represented by leading domestic AI education products, which combine in-school exam data and daily homework mistakes to achieve learning analysis. The scoring logic adopts a basic calculation method of simple score accumulation and arithmetic average, without distinguishing between invalid and abnormal answer sheets such as students answering randomly, answering beyond the time limit, or guessing hastily. Abnormal answer data is directly included in the scoring calculation, resulting in diagnostic distortion.
[0006] The third category is small-scale localized assessment systems that use psychological scales and simplified scoring methods. They can only output single-item scores, lack multi-dimensional cross-correlation analysis, and cannot uncover the implicit learning causes formed by the coupling of multiple factors.
[0007] However, existing commercial assessment products still have technical weaknesses in areas such as data screening, quantitative scoring, result reliability control, etiology tracing, and model self-optimization. Specifically, these weaknesses manifest in the following ways:
[0008] 1. Learning assessment relies heavily on subjective experience and lacks standardized quantitative models: Traditional offline learning assessments and existing simple online assessment tools lack unified quantitative calculation standards. Teachers subjectively categorize students as "subject-specific, averse to learning, or with poor concentration" based on classroom performance and single test scores. Students with similar scores and classroom performance may arrive at vastly different diagnostic conclusions from different teachers. Similar external learning behaviors (such as declining grades) can correspond to dozens of different underlying causes, such as knowledge gaps, lack of motivation, emotional problems, and attention deficits. Current technology cannot distinguish between learning problems that appear identical but have different root causes through standardized mathematical models, resulting in severe homogenization of diagnoses and a lack of quantitative data support for personalized guidance.
[0009] 2. Assessment indicators prioritize results over process, and process indicators lack systematic collection and quantification: Exam scores are lagging indicators; learning problems often only manifest in exam results after 1-2 months, missing the optimal intervention window. Process indicators such as learning motivation, answering habits, test-taking pace, test-taking mindset, daily self-discipline, and emotional changes can provide early warnings of learning risks. However, existing products only collect single data points like correct and incorrect answers, without systematically designing process data collection fields and corresponding quantitative calculation formulas. This leaves a massive amount of process assessment data idle and unable to be transformed into diagnostic evidence.
[0010] 3. Lack of credibility assessment mechanism for diagnostic results; poor-quality data directly outputs erroneous conclusions: Existing assessment software assumes all questionnaire data is true and valid when outputting diagnostic reports, without verifying the quality of the data itself. Various data distortion scenarios exist, including students randomly filling out questionnaires, answering while fatigued, deliberately embellishing answers due to parental accompaniment (social expectation bias), assessment equipment malfunctions, and noisy and distracting assessment environments. Distorted data is fed into the scoring model to generate erroneous diagnostic conclusions, but the system lacks any credibility assessment or interception. Incorrect intervention plans are directly accepted by parents and teachers, leading to a misallocation of tutoring resources. A few products only use a three-level qualitative grading system (ABC, high, medium, low) to indicate credibility, lacking a continuous numerical quantification model. The grading boundaries are vague, making it difficult to accurately distinguish subtle differences in credibility in practical applications.
[0011] 4. Static and fixed algorithm models, lacking a closed-loop self-evolving optimization architecture: All existing assessment algorithms on the market have factory-fixed parameters, which cannot be automatically optimized and adjusted based on actual test data after deployment. As the characteristics of different school levels and students in different regions change, fixed parameter algorithms will continue to experience problems such as diagnostic drift and a year-on-year decline in the accuracy of etiology matching. They can only rely on R&D personnel to manually iterate and update the version offline, which has a long update cycle and high labor costs, and cannot achieve automatic continuous optimization of the system.
[0012] China has successively issued multiple versions of the "Artificial Intelligence + Education" Action Plan. These policies explicitly propose accelerating the implementation of intelligent learning assessment tools, promoting process-based academic evaluation to replace the sole reliance on grades, and leveraging artificial intelligence technology to achieve the educational reform goal of individualized instruction. Traditional manual learning assessment models rely on teachers' long-term teaching experience and subjective judgment of student problems. Limited by teachers' individual professional level, teaching energy, and the scope of observation of students' school performance, these models struggle to achieve comprehensive, large-scale, and refined student assessment. Summary of the Invention
[0013] Therefore, the purpose of this invention is to develop a learning assessment method and system based on multidimensional weighted scoring and multi-level credibility. Addressing existing problems in the industry, such as reliance on teachers' subjective experience, outdated performance indicators, lack of quantifiable process data, lack of credibility verification for diagnostic results, and static algorithms that cannot be iteratively optimized and upgraded, this invention combines modern educational measurement theory, Bayesian probability statistics, nonlinear weighted mathematical models, and multidimensional feature matrix matching algorithms to construct a dual-algorithm architecture. This achieves standardized, automated, and quantifiable personalized intelligent diagnosis of learning across the entire process. It provides a multidimensional weighted diagnostic scoring model, MDW-DSM, which filters abnormal response data through a response quality function, verifies logical contradictions in answers through a consistency correction coefficient, and highlights severe learning weaknesses through a knowledge gap amplification coefficient. Traditional arithmetic mean scoring cannot distinguish invalid answers, ignore logical contradictions, and cause homogenization in scoring of questions of varying severity. This paper expands upon the MDW-DSM model by incorporating a cross-dimensional etiology matrix matching algorithm and a probabilistic iterative etiology identification algorithm. This breaks away from the outdated method of determining etiology based on a single-dimensional score threshold, accurately locating explicit and implicit learning etiologies based on multi-dimensional score combination features. A multi-level bias correction confidence assessment model (MBL-BC) is constructed, using five layers of full-dimensional risk detection combined with a product probability formula to quantify the reliability of continuous numerical values from 0 to 1. Diagnostic report output is managed hierarchically based on confidence thresholds to intercept low-confidence erroneous diagnoses. A self-evolving closed-loop system is built, incorporating approximate Bayesian parameter smoothing and sliding window accuracy monitoring, enabling the entire system's algorithm to autonomously iterate and optimize with real-world data, thereby improving diagnostic accuracy.
[0014] This invention provides a learning evaluation method based on multidimensional weighted scoring and multi-level credibility, comprising: constructing a multidimensional weighted diagnostic scoring model MDW-DSM and a multi-level bias-corrected confidence assessment model MBL-BC; the MDW-DSM model is connected downward to the original assessment response data and derives two subordinate extension algorithms upward: a cross-dimensional etiology matching algorithm and a probabilistic iterative etiology identification algorithm; the MBL-BC model receives the full-dimensional standardized scoring data and original assessment metadata output by the MDW-DSM model and completes the quantitative grading of diagnostic credibility.
[0015] Furthermore, the Multidimensional Weighted Diagnostic Scoring Model (MDW-DSM) transforms the raw response data into standardized dimensional scores, incorporating three collaborative technical elements, expressed as follows:
[0016] S_k=([Σw_ki·f(x_ki,t_ki)] / [Σw_ki])·θ_k·γ_k;
[0017] Where S_k represents the final standardized score of the k-th learning dimension; w_ki represents the fixed weight coefficient of the i-th question under the k-th dimension;
[0018] In this invention, k=10, corresponding to the standardized scores D1~D10 of ten learning dimensions, each defined as follows:
[0019] D1 (Learning Motivation): The intrinsic driving force for learning, assessed through indicators such as achievement goal orientation, interest, and initiative;
[0020] D2 (Self-Regulation): Learning planning and execution ability, assessed through indicators such as time management, goal setting, and self-monitoring;
[0021] D3 (Digital Behavior): Digital device usage habits, evaluated through metrics such as mobile / tablet usage patterns and screen time management;
[0022] D4 (Family Environment): Family learning support, assessed through indicators such as parental involvement, learning atmosphere, and family resource investment.
[0023] D5 (Learning Strategies): The selection and application of learning methods, assessed through indicators such as preview and review strategies, note-taking methods, and memory techniques;
[0024] D6 (Study Habits): Daily study routines, assessed through indicators such as sleep schedule, homework completion habits, and classroom listening habits;
[0025] D7 (Exam Mindset): Exam-taking psychological state, assessed through indicators such as exam anxiety level, self-confidence, and stress coping;
[0026] D8 (Attention): Concentration level, assessed through indicators such as classroom focus, homework focus time, and resistance to distractions;
[0027] D9 (Social Learning): Peer support effectiveness is assessed through indicators such as willingness to learn collaboratively, participation in discussions, and quality of peer relationships.
[0028] D10 (Subject Attitude): Subject preferences and engagement, assessed through indicators such as humanities / science inclination, distribution of subject interests, and degree of subject specialization.
[0029] Among them, the family learning support dimension (D4) is quantitatively assessed through indicators such as parental involvement, learning atmosphere, family resource investment, frequency of parent-child communication, and consistency of family educational philosophies. This dimension reflects the degree to which the student's family environment promotes or interferes with their learning process. It is a key environmental variable in cross-dimensional etiology matching and is significantly associated with multiple learning etiologies (such as insufficient learning motivation, attention deficit, and test anxiety), exhibiting a high association weight in the dimension-etiology association matrix W_jk. The assessment data for dimension D4 comes from students' responses to questions regarding family learning conditions, frequency of parental tutoring, and perception of the family learning atmosphere in questionnaires. Standardized scores (0-10 points) are then generated after calculation using the MDW-DSM model.
[0030] The three technical elements are as follows:
[0031] The response quality function is f(x_ki,t_ki), f(x_ki,t_ki)=x_ki·max(c_min,1+(t_o-|t_ki-t_o|) / t_o·η); it simultaneously incorporates the original score x_ki of the question and the actual response time t_ki of a single question, where t_o is the optimal standard response time for this question after large-sample calibration; η is the time offset adjustment constant; c_min is the lower bound of the minimum correction coefficient (preferably 0.5) to prevent the correction term from becoming excessively negative during extreme abnormal responses, thus causing the function value to become negative, and ensuring that the dimension of extreme abnormal responses still retains the minimum weight; it reduces the weight of abnormal responses that are too fast or too slow, while ensuring that the dimension of abnormal responses is not completely cleared to zero;
[0032] This invention differs from traditional methods that score solely based on the correctness of the answer. It considers both the answer selection x_ki and the response time t_ki. When the response time t_ki deviates from the optimal response time t_o, the function value is reduced. It also de-weights responses that are too fast (possibly random, time < 3 seconds) and too slow (possibly stuck or requiring external assistance, time > 20 seconds).
[0033] The consistency correction coefficient θ_k is given by θ_k = 1 - 1 / [m(m-1)]·Σ|x_kp-x_kq^(rev)|, where m is the total number of positive and negative verification questions within the k-th assessment dimension; x_kp is the standardized score of the positive questions within the dimension; and x_kq^(rev) is the converted score of the negative paired questions within the same dimension. Based on the same-dimensional positive and negative paired question verification logic, the dimension correction coefficient is lowered when contradictory answers occur.
[0034] By utilizing a design of reverse-response question pairs within the same dimension, the logical consistency of students' answers is detected. When students give contradictory answers to positive and negative questions within the same dimension, θ_k decreases. The technical effect is to incorporate answer consistency into the scoring system and prevent contradictory data from contaminating diagnostic results.
[0035] The knowledge gap amplification coefficient γ_k is given by γ_k = 1 + α·exp(-β·S_k^(base)); where S_k^(base) is the baseline score of the dimension before amplification after the first two coefficient operations; α and β are preset nonlinear control constants; a nonlinear exponential formula is used to amplify the weight of severely deficient dimensions with low baseline scores in downstream etiology matching (embedding higher combination weights in cross-dimensional combination analysis).
[0036] Nonlinear amplification is applied to low-scoring dimensions (dimensions with serious problems) to highlight important learning gaps and prevent multiple medium-sized problems from distracting diagnostic attention.
[0037] The three technical elements sequentially achieve invalid data filtering, logical contradiction correction, and low-scoring dimension nonlinear amplification, outputting the standardized score S_k for each learning dimension.
[0038] These three technical elements work together: the response quality function ensures the reliability of the input, the consistency correction coefficient ensures the logical consistency of the input, and the gap amplification coefficient ensures the discriminative power of the output. Together, these three elements constitute a precise transformation from raw data to standardized scores.
[0039] Furthermore, after outputting the dimensional score S_k of the MDW-DSM, a cross-dimensional etiology matching method is added; a dimensional-etiology association matrix W_jk is constructed, and the matching confidence C_j for each etiology is calculated, using the following formula:
[0040] C_j=[ΣW_jk·S_k] / [max(ΣW_jk·10,ε)]·δ_j; where 10 is the full score of each dimension (this invention uses a 0~10 score system); max(ΣW_jk·10,ε) is the maximum value of the theoretical full score weighted sum of all dimensions associated with cause j;
[0041] Wherein, C_j is the final matching confidence score of the j-th etiology (0~1 interval); W_jk is the pre-constructed dimension-etiology association matrix (number of dimensions k × number of etiologies J, k=10, J=50 in this embodiment), and the value of the matrix cell represents the association weight of the k-th learning dimension to the j-th etiological factor (value 0~1, the initial value is calibrated by educational measurement experts based on educational psychology theory and empirical research, and optimized by reverse calibration using historical diagnostic data); ε is a minimum value prevention zero constant to avoid calculation errors when the denominator is 0; δ_j is the etiology discrimination coefficient; constant 10 is the upper limit of the score of each learning dimension (full score 10 points), used to normalize the summation of weights to the 0~1 interval; the etiology is determined by weighted fusion of multi-dimensional scores, and the matching weight of etiologies with high historical confusion rate is calibrated by the discrimination coefficient δ_j.
[0042] The method of this invention differs from the traditional single-question cumulative sorting. It uses cross-dimensional combination patterns (rather than single-dimensional score thresholds) to discover latent causes and automatically calibrates causes with high historical confusion rates through the discriminative power coefficient δ_j.
[0043] Furthermore, when the confidence scores of two or more causes are too close to distinguish, a probabilistic iterative cause identification method is activated. This method continuously updates the posterior probability of the cause by supplementing the assessment data through multiple rounds of targeted follow-up questions. The posterior probability formula is:
[0044] P_(t+1)=softmax(P_t+Δ_t);
[0045] Where P_t represents the posterior probability distribution of each candidate etiology in the t-th iteration; Δ_t is the probability correction increment brought about by the targeted adaptive follow-up questioning to collect new data in this round; and the softmax function realizes probability normalization.
[0046] The calculation method for Δ_t is as follows: the newly collected n follow-up questions construct a new scoring vector Δs=(Δs_1,Δs_2,...,Δs_K), where K is the total number of learning dimensions, and each Δs_k is obtained by weighted summation of the answer quality function of the new questions; Δ_t=λ·softmax(Δs), where λ is the iterative learning rate constant (default value 0.15), and the softmax function transforms the new scoring vector into a probability increment distribution;
[0047] The iteration stops and the final diagnosed cause is determined when the confidence difference between the primary cause (ranked first) and the secondary cause (ranked second) reaches a preset threshold (preferably 0.5 in this embodiment).
[0048] Nonlinear amplification is applied to low-scoring dimensions (dimensions with severe problems), meaning that the weight of these dimensions is highlighted during normalized score comparison: γ_k applies an upward adjustment coefficient to the scores of low-scoring dimensions from the numerator in the normalized score calculation, giving these severely deficient dimensions higher values. This allows them to receive higher attention weight in subsequent cross-dimensional cause-matching calculations, rather than being overwhelmed by scattered medium-sized problems, thereby strengthening the system's ability to identify core learning weaknesses. In other words, "amplification" refers to assigning higher combination weights to low-scoring dimensions during cross-dimensional combinatorial analysis, thus giving them higher weight in downstream cause-matching.
[0049] Furthermore, the multi-level bias correction confidence assessment model MBL-BC comprises two parts: a five-layer bias detection system and a product probability model. The five-layer bias detection system includes a data quality layer, a student status layer, an algorithm bias layer, an environmental factor layer, and a timeliness layer. The corresponding assessment risk coefficient r_i is collected at each layer, and each layer is configured with a dynamically adjustable weight coefficient α_i (the weight α_i of each layer can be dynamically adjusted according to the assessment scenario).
[0050] Specifically, the data quality layer (layer 1) detects consistency in answers to reverse questions, abnormal answering times, and the number of missed questions;
[0051] The student status layer (layer 2) detects fatigue effect (the difference in scores before and after the questionnaire exceeds the threshold) and social expectation bias (the proportion of extreme positive responses is too high).
[0052] The algorithm bias layer (layer 3) monitors the accuracy of the sliding window and detects sample distribution deviation;
[0053] The environmental factors layer (layer 4) assesses cross-validation of environmental disturbances (such as parental presence marking);
[0054] The timeliness layer (layer 5) detects the decay of time since the last assessment, exp(-λ·Δt_days).
[0055] The expression for the product probability model is:
[0056] T=T_0·Π(1-r_i·α_i / Σα_i)·exp(-β·n_risk);
[0057] The detection results of the five-layer bias detection system are integrated into a continuous credibility score T of 0-1 through the product probability model.
[0058] The reliability is graded based on the continuous reliability score T-value: T≥0.8 indicates high reliability and can be used directly; 0.6≤T<0.8 indicates medium reliability and it is recommended to manually review key items; 0.4≤T<0.6 indicates low reliability and requires manual review before use; T<0.4 indicates unreliability and the output of diagnostic conclusions should be prevented (marked as "re-evaluation required").
[0059] Furthermore, the front-end collection method for the original assessment response data includes any one or more combinations of the following three schemes:
[0060] Gamified challenge-based assessments replace questionnaires;
[0061] Eye movement and facial expression physiological data acquisition;
[0062] Automatic scoring of open-ended question answering and NLP semantic parsing;
[0063] After generating standardized x_ki and t_ki parameters using various acquisition methods, they can all be integrated into the MDW-DSM model to complete the calculation.
[0064] Furthermore, the learning assessment method based on multidimensional weighted scoring and multi-level credibility also includes a self-evolutionary closed-loop step: collecting student learning improvement feedback data after intervention implementation, progressively optimizing the response quality function parameters, knowledge gap amplification coefficient parameters, weight coefficients α_i of each level of MBL-BC, and all parameters of the dimension-cause association matrix W_jk of MDW-DSM through an approximate Bayesian parameter smoothing algorithm (using a Bayesian sequence weighted fusion method); and using a sliding window accuracy monitoring module to monitor changes in model accuracy in real time, automatically triggering parameter correction when a continuous drift in accuracy occurs within the sliding window.
[0065] This invention also provides a learning evaluation system based on multidimensional weighted scoring and multi-level credibility, used to implement the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as described above. The system deploys a three-factor weighted scoring algorithm for the multidimensional weighted diagnostic scoring model MDW-DSM, consisting of the response quality function f(x_ki,t_ki), the consistency correction coefficient θ_k, and the knowledge gap amplification coefficient γ_k, as well as a product probability credibility calculation algorithm for the multi-level bias-corrected confidence assessment model MBL-BC. The system includes:
[0066] MDW-DSM computation module: used for the MDW-DSM model to interface with the original assessment and response data downwards, and to derive two subordinate extension algorithms upwards: cross-dimensional etiology matching algorithm and probabilistic iterative etiology identification algorithm;
[0067] The MBL-BC credibility calculation module is used by the MBL-BC model to receive the full-dimensional standardized scoring data and original evaluation metadata output by the MDW-DSM model, and to complete the quantitative grading of diagnostic credibility.
[0068] Furthermore, the learning evaluation system based on multidimensional weighted scoring and multi-level credibility also includes: a front-end data acquisition module, a diagnostic report output module, and a data feedback self-evolution module;
[0069] The front-end data acquisition module collects the raw data of the responses, which is then processed sequentially by the MDW-DSM calculation module and the MBL-BC credibility calculation module. The diagnostic report output module outputs reports in a hierarchical manner, and the feedback data flows back to the data feedback self-evolution module to complete the parameter iterative optimization.
[0070] Preferably, the entire evaluation system is configured with a self-evolving closed-loop architecture; it collects student learning feedback data after the intervention is implemented, and progressively optimizes all parameters of MDW-DSM, MBL-BC, and dimension-causal association matrix through an approximate Bayesian parameter smoothing algorithm (using a Bayesian sequence weighted fusion method); it is equipped with a sliding window accuracy monitoring module to monitor sample distribution shift and algorithm accuracy drift in real time, automatically trigger parameter correction, and realize the model's autonomous iterative optimization based on measured data.
[0071] The personalized learning assessment method and implementation system based on Multidimensional Weighted Diagnostic Score (MDW-DSM) and Multilevel Bias Corrected Confidence Assessment (MBL-BC) of this invention can be applied to scenarios such as routine learning screening in primary and secondary schools, diagnosis of learning shortcomings of students in after-school tutoring institutions, tracing the source of learning problems in family education, and academic quality statistics of education management departments.
[0072] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as described above.
[0073] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as described above.
[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0075] This invention provides a learning assessment method based on multidimensional weighted scoring and multi-level credibility, designing a multidimensional weighted diagnostic scoring model (MDW-DSM) and a multi-level bias-corrected confidence assessment model (MBL-BC). The MDW-DSM is composed of three technical elements: a response quality function, a consistency correction coefficient, and a knowledge gap amplification coefficient. It automatically reduces the weight of abnormal responses based on the answer and response time, verifies logical inconsistencies in responses using positive and negative comparison questions, and amplifies the scoring of the dimension of severe learning weaknesses using a non-linear coefficient. Building upon the basic scoring model, it extends the model with a cross-dimensional etiology association matching algorithm and a probabilistic iterative etiology identification algorithm, overcoming the limitations of traditional single-question score threshold-based etiology determination. It mines implicit learning pathogenic factors based on multi-dimensional score combination features and accurately distinguishes highly similar, confusing etiologies through multi-round adaptive follow-up questioning iteration to optimize the posterior probability of etiologies. The MBL-BC five-layer bias detection architecture is constructed, collecting data from five levels: data quality, student physiological and psychological state, algorithm drift, assessment environment, and assessment timeliness. Risk characteristics are assessed by quantifying the credibility of diagnostic conclusions using a product probability calculation formula. Based on credibility thresholds, diagnostic reports are output in tiers, and low-reliability results are forcibly intercepted. The entire assessment system incorporates a self-evolving closed-loop mechanism that uses approximate Bayesian parameter smoothing and sliding window accuracy monitoring. It continuously optimizes model parameters based on real-world test data, forming a data flywheel of data collection, intelligent diagnosis, intervention implementation, data feedback, and algorithm iteration. This effectively solves key technical challenges in traditional learning assessments, such as strong subjectivity, lack of quantitative standards, interference from invalid and abnormal data, uncontrollable diagnostic credibility, and static, fixed models that cannot self-optimize. It achieves intelligent learning assessment and can be widely applied in various educational scenarios, including K-12 school learning assessment, personalized after-school tutoring assessment, and comprehensive screening of students' learning psychology and abilities, with broad prospects for promotion and application. Attached Figure Description
[0076] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.
[0077] In the attached diagram:
[0078] Figure 1 This is an architecture diagram of a learning evaluation system based on multidimensional weighted scoring and multi-level credibility, as described in an embodiment of the present invention.
[0079] Figure 2 This is a flowchart of the MDW-DSM calculation process according to an embodiment of the present invention;
[0080] Figure 3 This is a diagram of the MBL-BC five-layer bias detection and product probability model according to an embodiment of the present invention;
[0081] Figure 4 This is a schematic diagram of cross-dimensional etiology matching according to an embodiment of the present invention;
[0082] Figure 5 This is a schematic diagram of the probability iteration for differential diagnosis according to an embodiment of the present invention;
[0083] Figure 6 This is a schematic diagram of a self-correcting closed loop according to an embodiment of the present invention;
[0084] Figure 7 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation
[0085] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and products consistent with some aspects of this disclosure as detailed in the appended claims.
[0086] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0087] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0088] The embodiments of the present invention will be described in further detail below.
[0089] This invention provides a learning evaluation method based on multidimensional weighted scoring and multi-level credibility, comprising two independent but interconnected algorithmic models: Multidimensional Weighted Diagnostic Scoring Model (MDW-DSM) and Multi-Level Bias Correction Confidence Assessment Model (MBL-BC). MDW-DSM connects downwards to the original assessment response data and derives two subordinate extension algorithms upwards: a cross-dimensional etiology matching algorithm and a probabilistic iterative etiology identification algorithm. MBL-BC receives the full-dimensional standardized scoring data and original assessment metadata output by MDW-DSM, completing the quantitative grading of diagnostic credibility. The entire evaluation system is connected to a self-evolving closed-loop mechanism at the end, collecting feedback data after intervention implementation and feeding it back to optimize all model parameters of MDW-DSM and MBL-BC, forming a full-link closed-loop technical architecture.
[0090] The overall execution steps of the learning evaluation method based on multidimensional weighted scoring and multi-level credibility include:
[0091] Step 1: Standardized assessment data collection, collecting raw metadata such as students' answer options for each test question x_ki, time spent answering a single question t_ki, assessment environment markers, assessment time, and pre- and post-test scores;
[0092] Step 2: Input the raw data into the MDW-DSM model, and then filter out abnormal answers through the answer quality function, correct contradictory answer scores through the consistency correction coefficient, and correct the score of the weak dimension through the knowledge gap amplification coefficient. Output the standardized learning score S_k of ten learning dimensions (D1~D10).
[0093] Step 3: Input the scores S_k of the ten learning dimensions (D1~D10) into the cross-dimensional etiology matching algorithm, and calculate the basic matching confidence C_j of each etiology based on the dimension-etiology association matrix W_jk;
[0094] Step 4: For candidate etiology groups with similar confidence scores that are difficult to distinguish, start the probabilistic iterative etiology identification algorithm. Update the posterior probability of the etiology through adaptive multi-round follow-up questions until the confidence difference between the primary and secondary etiologies reaches the preset threshold, and finally lock the diagnosed etiology.
[0095] Step 5: Input all the original metadata from Step 1 and the intermediate calculation data from Steps 2 and 4 into the MBL-BC five-layer bias detection system, extract the risk coefficients of each layer, and substitute them into the product probability model to calculate the comprehensive diagnostic credibility T.
[0096] Step 6: Classify and handle cases according to the reliability T value range: T≥0.8, directly output a complete diagnostic report and personalized intervention plan; 0.6≤T<0.8, the reliability is marked in the report, and key items are manually reviewed before use; 0.4≤T<0.6, the entire report can only be implemented after manual review; T<0.4, the system automatically intercepts the diagnostic output and prompts students to retake the assessment.
[0097] Step 7: After the intervention, collect feedback data on students' subsequent learning changes. Optimize all parameters of MDW-DSM, MBL-BC, and etiology matching matrix through an approximate Bayesian parameter smoothing algorithm and a sliding window monitoring mechanism to complete one round of self-evolutionary iteration and enter the next round of assessment cycle.
[0098] This invention also provides a learning evaluation system based on multidimensional weighted scoring and multi-level credibility, including: a front-end data acquisition module, an MDW-DSM operation module, an MBL-BC credibility calculation module, a diagnostic report output module, and a data feedback self-evolution module;
[0099] The MDW-DSM computation module is used to connect the MDW-DSM model downwards to the original assessment and response data, and to derive two subordinate extension algorithms upwards: cross-dimensional etiology matching algorithm and probabilistic iterative etiology identification algorithm;
[0100] The MBL-BC credibility calculation module is used by the MBL-BC model to receive the full-dimensional standardized scoring data and original evaluation metadata output by the MDW-DSM model, and to complete the diagnostic credibility measurement and grading.
[0101] The front-end data acquisition module collects the raw data of the responses, which is then processed sequentially by the MDW-DSM calculation module and the MBL-BC credibility calculation module. The diagnostic report output module outputs reports in a hierarchical manner, and the feedback data flows back to the data feedback self-evolution module to complete the parameter iterative optimization.
[0102] Figure 1 The overall system architecture of this embodiment is shown.
[0103] Current educational diagnostic scoring methods generally employ a technical approach of directly accumulating question scores and calculating dimensional scores using an arithmetic mean. This existing calculation model has three unavoidable technical flaws: First, it does not distinguish between normal, serious answers and abnormal answers such as hasty, random, or delayed responses requiring external assistance. Abnormal and valid answers are weighted equally in the total score, lowering the accuracy of the diagnostic results. Second, it lacks an answer logic verification step. When students provide contradictory answers to forward and reverse verification questions within the same assessment dimension, the system cannot identify data distortion, and contradictory data is included in the scoring. Third, all learning problem dimensions are scored proportionally, compressing the score difference between mild learning deficiencies and severe knowledge gaps. The system cannot automatically focus on core weaknesses, and multiple scattered minor issues dilute the diagnostic focus, resulting in insufficiently targeted intervention plans.
[0104] This invention constructs a multidimensional weighted diagnostic scoring model MDW-DSM, and designs a general formula for calculating the final standardized score of each dimension: S_k=( [Σw_ki·f(x_ki,t_ki)] / [Σw_ki])·θ_k·γ_k;
[0105] Where S_k represents the final standardized score of the k-th learning dimension; w_ki represents the fixed weight coefficient of the i-th question in the k-th dimension; f(x_ki,t_ki) is the answer quality function; θ_k is the consistency correction coefficient (for answers in the same dimension); and γ_k is the knowledge gap (non-linear) amplification coefficient. The three coefficients f(x_ki,t_ki), θ_k, and γ_k are the three technical elements of MDW-DSM. They work together to complete the three steps of data processing in sequence: noise reduction of raw data → correction of logical distortion → amplification of shortcomings, thereby realizing the transformation from raw answer data to standardized and accurate scores.
[0106] Specifically, element one: the expression for the response quality function f(x_ki,t_ki) (abnormal responses are automatically downgraded) is:
[0107] f(x_ki,t_ki)=x_ki·max(c_min,1+(t_o-|t_ki-t_o|) / t_o·η); where c_min is the lower bound of the minimum correction coefficient (preferably 0.5) to prevent the correction term from being excessively negative when there are extreme abnormal answers, which would result in a negative function value;
[0108] Where x_ki is the original score of the i-th question in the k-th dimension (the original score in the range of 0 to 10 is obtained by converting the correctness of the question answer / the scale options);
[0109] t_ki represents the actual time spent by the student answering this question, in seconds.
[0110] t_o is the optimal standard answer time for this test question, which has been calibrated by a large sample (pre-stored by the system, with t_o assigned independently for different question types and different grade levels).
[0111] η is the time offset adjustment constant, which is a fixed parameter preset by the system according to the evaluation scenario;
[0112] When a student's actual answering time t_ki is infinitely close to the optimal answering time t_o, the time correction term approaches 1, f(x_ki,t_ki)≈x_ki, and the original score is fully preserved. When the student answers too quickly (t_ki < 3s, likely indicating random answer selection) or too slowly (t_ki > 20s, likely indicating inability to answer or seeking help from outside the exam room), the value of |t_ki - t_o| increases significantly, the time correction term decreases significantly, and f(x_ki,t_ki) decreases accordingly. This achieves automatic weight reduction for abnormal answers, and invalid answer data is naturally weighted, unlike traditional algorithms that rely solely on correct or incorrect answers and ignore answering time. This embodiment of the invention uses time parameters to achieve the first layer of raw data cleaning.
[0113] Element Two: The formula for calculating the consistency correction coefficient θ_k (correction of logical contradictions in same-dimensional positive and negative arguments) is as follows:
[0114] θ_k=1-1 / [m(m-1)]·Σ|x_kp-x_kq^(rev)|;
[0115] Where m is the total number of positive test items and negative verification test items in the k-th assessment dimension;
[0116] x_kp represents the standardized score of positive items within the dimension;
[0117] x_kq^(rev) represents the converted score of reverse-paired test items of the same dimension;
[0118] The assessment scale includes paired positive and negative verification questions during the dimension design phase. For example, the positive question "Can I plan my study time independently every day?" and the negative matching question "Often unable to control my own study pace" have mutually exclusive standard answers. If a student's answer contains a logical contradiction, the difference |x_kp-x_kq^(rev)| increases, the summation term increases, and the value of θ_k decreases accordingly (θ_k∈(0,1])), resulting in a discount on the entire dimension score. When the answers across all dimensions are logically consistent and without contradictions, the difference summation term is 0, θ_k=1, and the score is not reduced. A consistency correction coefficient implements a second layer of data error correction, filtering out logically contradictory data caused by students randomly selecting answers, and preventing contradictory answers from contaminating the dimension scores.
[0119] Element 3: The formula for calculating the knowledge gap amplification factor \(γ_k\) (non-linear amplification of severe shortcomings) is as follows:
[0120] γ_k=1+α·exp(-β·S_k^(base));
[0121] Wherein, S_k^(base) is the baseline score of the dimension before amplification after the coefficients of the first two items are calculated;
[0122] α and β are preset nonlinear control constants;
[0123] When the baseline score S_k^(base) for a certain dimension is low (indicating serious learning problems and large knowledge / habit gaps in that dimension), the exponential term exp(-β·S_k^(base)) increases, γ_k > 1, and this dimension is ultimately given higher weight in downstream etiology matching (non-linear amplification), automatically highlighting important deficiencies. When the baseline score is high (indicating good performance in the dimension with only minor issues), the exponential term approaches 0, γ_k ≈ 1, and the score remains essentially unchanged. By utilizing non-linear mathematical principles, this approach addresses the technical shortcomings of traditional algorithms, such as small differences in scores for minor and serious issues and the masking of important deficiencies by scattered minor problems, thereby improving the problem identification accuracy of diagnostic results.
[0124] To avoid negative function values due to extreme response times, a lower bound constraint is set: the final f value should not be lower than 50% of the original score x_ki, i.e., f_min = x_ki * 0.5. Simultaneously, an upper limit threshold for response time, t_max = 60s, is set. Time exceeding this threshold is used in the calculation based on t_max to prevent excessively long response times for a single question from drastically lowering the score.
[0125] The response quality function filters the reliability of the input data, eliminating invalid data with abnormal response times; the consistency correction coefficient verifies the logical consistency of the input data, eliminating contradictory or distorted data; and the knowledge gap amplification coefficient handles the differentiation of output results, enhancing the identification of severe deficiencies. These three layers of processing are sequentially connected, optimizing the scoring accuracy across the entire chain from data source, data logic, to result output. Figure 2 The calculation process of the MDW-DSM model according to an embodiment of the present invention is shown.
[0126] Based on the MDW-DSM model, this invention proposes a subordinate extension feature: a cross-dimensional etiology matching algorithm (such as...). Figure 4 As shown), probabilistic iterative etiology identification algorithm (such as...) Figure 5 (As shown). Specifically, based on the standardized scores S_k of the ten learning dimensions (D1~D10) output by the MDW-DSM model, this invention adds a cross-dimensional etiology matching method, breaking the traditional technical mode of determining etiology by a single-dimensional score threshold. It relies on multi-dimensional score combination features to mine implicit learning etiologies arising from multi-factor coupling. The calculation formula is as follows:
[0127] C_j=[ΣW_jk·S_k] / [max(ΣW_jk·10,ε)]·δ_j; where 10 is the full score for each dimension (this invention uses a 0~10 score system);
[0128] Wherein, C_j is the final matching confidence score of the j-th etiology (0~1 range); W_jk is the pre-constructed dimension-etiology association matrix (number of dimensions K × number of etiologies J, K=10, J=50 in this embodiment), and the value of the matrix cell represents the association weight of the k-th learning dimension to the j-th etiology (value 0~1, the initial value is calibrated by educational measurement experts based on educational psychology theory and empirical research, and optimized by reverse calibration using historical diagnostic data); ε is the minimum value prevention zero constant to avoid calculation errors when the denominator is 0; δ_j is the etiology discrimination coefficient (initial value is 1.0, automatically adjusted according to the confusion rate of etiology j with other etiologies in historical data, δ_j is reduced by 0.05 for every 10% increase in confusion rate, with an adjustment range of 0.5~1.5); constant 10 is the upper limit of the score for each learning dimension (maximum score 10 points), used to normalize the summation of weights to the 0~1 range;
[0129] This invention relies on the association matrix to achieve multi-dimensional score weighted fusion, rather than single-dimensional score threshold triggering cause determination, which can accurately identify the latent cause of "multi-dimensional minor defects superimposed to form a certain type of serious learning problem"; the discrimination coefficient δ_j is automatically optimized based on massive historical test data, and the weight is automatically adjusted for the same type of cause that is frequently confused and has a high misjudgment rate in historical data, continuously reducing the cause matching confusion rate.
[0130] After cross-dimensional etiology matching, scenarios frequently arise where the C_j values of two or more etiologies are highly similar, making it difficult for the system to distinguish between primary and secondary pathogenic factors. Therefore, this invention designs an adaptive follow-up questioning and identification algorithm based on Bayesian posterior probability iteration. The iteration formula is as follows:
[0131] P_(t+1)=softmax(P_t+Δ_t);
[0132] Where P_t represents the posterior probability distribution of each candidate etiology in the t-th iteration; Δ_t is the probability correction increment brought about by the targeted adaptive follow-up questioning to collect new data in this round; and the softmax function realizes probability normalization.
[0133] The specific calculation method of Δ_t is as follows: A new scoring vector Δs = (Δs_1, Δs_2, ..., Δs_K) is constructed from the newly collected n follow-up questions, where K is the total number of learning dimensions. Each Δs_k is obtained by weighted summation of the answer quality function for the new questions. Δ_t = λ·softmax(Δs), where λ is the iterative learning rate constant (default value 0.15). The softmax function transforms the new scoring vector into a probability increment distribution. The essence of Bayesian update is P_(t+1)∝P_t·L(new data | etiological hypothesis), where L is the likelihood function. Here, the probability increment Δ_t transformed from the new scoring vector is used to approximate the Bayesian posterior update, achieving a balance between lightweight engineering implementation and the rigor of Bayesian theory. The reason for using softmax approximation instead of standard Bayesian product update P(θ|D)∝P(θ)·L(D|θ) is that standard Bayesian product update is prone to numerical underflow during multiple iterations, while softmax additive update is better in terms of numerical stability. In the diagnostic scenario, the direction of probability increment correction is consistent with the convergence direction of Bayesian post-acceptance. The actual test verified that the difference in convergence steps between softmax and standard Bayesian update does not exceed 10%, which meets the engineering accuracy requirements.
[0134] The iteration termination rule is as follows: after multiple rounds of intelligent targeted follow-up questions to supplement the evaluation data, the probability difference between the first-ranked primary cause and the second-ranked primary cause is calculated after each round of update. The iteration stops when the difference is ≥0.5, and the item with the highest probability is diagnosed as the cause. This algorithm solves the industry technical problem of being unable to distinguish between similar causes and having a high misdiagnosis rate.
[0135] Step 1: The numerator of the weighted summation of the answer quality function is 9.544, the cumulative sum of the weights of the questions is 4.15, and the base score S_k^(base) is 2.300;
[0136] Step 2: If logical contradictions arise in the answers to the positive and negative verification questions, the consistency correction coefficient θ_k = 0.5;
[0137] Step 3: Substitute into the knowledge gap amplification factor formula to calculate γ_k=1.251;
[0138] Step 4: Substitute the original answer data into the general formula for converting to standardized dimensional scores: S_k = 2.300 × 0.5 × 1.251 = 1.439;
[0139] The final score for dimension D4 was significantly low, and the system automatically marked it as "data quality needs to be reviewed." This anomaly label was then synchronously transmitted to the subsequent MBL-BC credibility assessment module as a risk factor.
[0140] This invention constructs a multi-level bias correction confidence assessment model (MBL-BC) to address the lack of a systematic quantification scheme for diagnostic reliability in existing technologies. These technologies either ignore reliability assessment and assume all diagnoses are valid, or only use a three-level (high / medium / low) coarse qualitative labeling without quantitative boundaries. Student assessments are affected by five major categories of interference factors: data quality, physical and mental state, assessment environment, algorithm drift, and assessment timeliness, easily leading to distorted diagnostic conclusions. This invention addresses these challenges by using a five-level bias detection system and a product probability mathematical model (such as...). Figure 3 As shown, it realizes the reliable quantification of continuous 0-1 values, and is equipped with four-level automatic interception rules to avoid the implementation of error diagnosis from the result output end.
[0141] It should be noted that in this invention, "bias-corrected confidence level" means "diagnostic confidence level after deducting various bias factors," that is, the comprehensive confidence assessment of the remaining diagnostic conclusions after the MBL-BC five-layer detection system has eliminated data quality bias, student status bias, algorithm drift bias, environmental interference bias, and time-related decay bias one by one. "Bias" here refers to various systematic error sources, rather than subjective bias in the sense of a social group. Throughout the text, "confidence level," "confidence," and "bias-corrected confidence level" all refer to the same quantity—the comprehensive confidence score T of the diagnostic conclusions after five-layer bias correction.
[0142] Specifically, the five-layer bias detection system has the following layering details:
[0143] Layer 1 [Data Quality Layer]: Risk verification of the underlying raw data. The detection items include: abnormal marking of consistency of answers to reverse-paired test questions, abnormal records of single question answering time exceeding the threshold, and statistics of the proportion of missing questions in the whole paper exceeding the standard. The output is the risk coefficient r_1 of this layer.
[0144] Layer 2 [Student Status Layer]: Detection of physiological and psychological deviations of the tested subjects. Detection items: significant fluctuations in scores of the scale before and after the assessment (judgment of fatigued answers), excessive proportion of extreme positive answers (social expectation bias, students deliberately concealing problems). Output risk coefficient r_2;
[0145] Layer 3 [Algorithm Deviation Layer]: Detects the drift risk of the model algorithm itself. It relies on a sliding window to monitor in real time the historical accuracy changes of the model in the same batch of samples, the data distribution of the tested samples, and the distribution deviation of the model training benchmark samples, and outputs a risk coefficient r_3.
[0146] Layer 4 [Environmental Factors Layer]: External assessment environment interference verification. Based on pre-marked fields such as the presence of parents during the assessment and the noise in the examination room, the degree of environmental interference is cross-validated, and the risk coefficient r_4 is output.
[0147] Layer 5 [Timeliness Layer]: The timeliness of the evaluation results decays. The longer the interval between the last evaluation and the present, the lower the reference value of the evaluation data. The exponential decay formula exp(-λ·Δt_days) is used to quantify the timeliness loss, where Δt_days is the number of days between two evaluations, λ is the decay constant, and the output risk coefficient r_5 is used.
[0148] Each of the five layers is configured with an independent and adjustable weight coefficient α_1, α_2, α_3, α_4, and α_5. The weight ratio of each layer can be dynamically adjusted according to different application scenarios such as in-school assessments, off-campus assessments, and family education assessments, thereby improving scenario adaptability.
[0149] The formula for calculating the product probability is as follows: (This is the overall credibility of the product probability model.)
[0150] T=T_0·Π(1-r_i·α_i / Σα_i)·exp(-β·n_risk);
[0151] Where T_0 is the baseline credibility constant (the system default initial baseline value, T_0=0.90 in this embodiment); r_i corresponds to the risk coefficient of each of the five layers; n_risk is the total number of layers marked as high risk in the five-layer detection; β is the risk item penalty constant; T is the final comprehensive credibility score, with a value range of 0<T≤1;
[0152] The embodiments of this invention employ a four-level trust rating and an automatic interception mechanism, as detailed below:
[0153] 1. Threshold (preferably T≥0.8 in this embodiment): High reliability, the system directly and completely outputs the diagnostic report, etiology conclusion and personalized learning intervention plan, without the need for manual review;
[0154] 2. 0.6≤T<0.8: Medium confidence level, complete diagnostic report output, but key etiology items in the report are marked with review prompts, and teachers can use it after manually checking key content;
[0155] 3. 0.4≤T<0.6: Low credibility; the entire report must be manually reviewed and corrected item by item before it can be implemented.
[0156] 4. T < 0.4: Unreliable level. The system will forcibly block all diagnostic conclusions from being output, and the front-end page will display the message: "The reliability of the data in this assessment is insufficient. Please have the student retake the assessment at a later date."
[0157] According to the MBL-BC model evaluation, the anomaly marking of student A's D4 dimension data resulted in a layer 1 data quality risk coefficient r=0.30. The other four layers (student status, algorithm, environment, and timeliness) were normal and had no anomaly risk, n_risk=1, and the baseline value T_0=0.90. After substituting into the product formula, T=0.90×0.94×0.6065=0.513. 0.4≤0.513<0.6, the result was judged as low confidence, falling within the low confidence range. The system automatically marked it as "requires manual review" to ensure that diagnostic conclusions generated from poor data are not directly used.
[0158] The risk coefficient r_i for each layer is calculated as follows:
[0159] Layer 1 (Data Quality Layer) r_1 = 0.2 × A_inconsistency + 0.3 × A_timeout + 0.5 × M_missing; where A_inconsistency is the contradiction rate of answers to reverse questions (range 0~1), A_timeout is the proportion of questions with abnormal answering time, and M_missing is the missing question rate;
[0160] Layer 2 (Student Status Layer) r_2=max(0.5×|score_front-score_back| / score_front,extreme_ratio); where score_front and score_back are the mean scores of the front and back halves of the scale, respectively, and extreme_ratio is the proportion of extremely positive answer options;
[0161] Layer 3 (Algorithm Bias Layer) r_3=1-accuracy_sliding, where accuracy_sliding is the model accuracy of the most recent 100 diagnostic records within the sliding window (diagnostic accuracy after manual review and confirmation).
[0162] Layer 4 (Environmental Factors Layer) r_4 is assigned a value based on the environmental interference flag field: Parents present r_4=0.4, Noisy examination room r_4=0.3, Equipment malfunction r_4=0.5, Normal environment r_4=0;
[0163] Layer 5 (time-sensitivity layer) r_5 = 1 - exp(-λ·Δt_days), where λ is the time-sensitivity decay coefficient, and in this embodiment λ = 0.05. The value of r_i for each layer is in the range of 0 to 1.
[0164] This embodiment uses MDW-DSM for both time and consistency checks to find that the D4 dimension response data is unreliable; it also uses MBL-BC to automatically determine that the data is of low confidence, thus preventing the hasty output of unreliable conclusions.
[0165] This invention breaks through the limitation of the traditional single reliability coefficient (Cronbach's α) which only verifies the consistency within the scale. It evaluates the reliability of diagnosis from five dimensions: data, students, algorithm, environment, and time. It adopts continuous numerical quantification and hierarchical interception for dual detection to ensure the quality of diagnostic output.
[0166] like Figure 6 As shown, in addition to the MDW-DSM and MBL-BC algorithms, the entire learning evaluation system of this invention adds a data-driven self-evolutionary closed-loop (self-correction) mechanism, forming a complete self-evolutionary closed-loop architecture. The main operation process is as follows:
[0167] The front end collects students' original answer data, MDW-DSM completes weighted scoring, cross-dimensional etiology matching and iterative identification lock the etiology, and MBL-BC credibility grading outputs diagnosis and intervention plan;
[0168] Teachers / parents provide personalized guidance based on the intervention plan and periodically follow up to collect data on students' subsequent learning improvement (intervention effect feedback data).
[0169] The feedback data return system relies on an approximate Bayesian parameter smoothing algorithm to progressively fine-tune all key parameters of w_ki, α, β in MDW-DSM, weight α_i at each level in MBL-BC, and all key parameters of the etiology matrix W_jk.
[0170] The sliding window monitoring module continuously monitors changes in model accuracy and automatically triggers parameter correction when the accuracy within the window shows a persistent drift.
[0171] According to the MBL-BC model assessment, the outlier marking of student A's D4 dimension data led to:
[0172] The risk coefficient r_1 of layer 1 is 0.30 (calculated by weighting the inconsistency rate A_inconsistency = 0.6: 0.2×0.6+0.3×0+0.5×0=0.12, and then weighted to 0.30 due to the high inconsistency rate triggering a second weighting for consistency).
[0173] The other four layers are normal, r_2=r_3=r_4=r_5=0;
[0174] Each level is equally weighted, α_1=α_2=α_3=α_4=α_5=1, Σα_i=5;
[0175] The number of high-risk levels is n_risk=1, the penalty constant for the risk item is β=0.5, and exp(-0.5×1)=0.6065;
[0176] The baseline confidence level T_0 = 0.90.
[0177] Substitute the values into the product probability formula to calculate the reduction factor for each layer:
[0178] Layer 1: (1-0.30×1 / 5)=0.94, substituting this into the equation gives the reduction factor (1-0.30×1 / 5)=0.94;
[0179] Layers 2-5: (1-0×1 / 5)=1.0;
[0180] The final T = 0.90 × 0.94 × 0.6065 = 0.513.
[0181] If the result is between 0.4 and 0.513 and is less than 0.6, it is considered a low-confidence result and is automatically marked as "requiring manual review" by the system to ensure that diagnostic conclusions generated from poor-quality data are not directly used.
[0182] The learning evaluation system of this invention relies on real data for automatic and continuous iterative optimization. The larger the sample size, the higher the accuracy of model diagnosis and the higher the accuracy of credibility evaluation.
[0183] The core algorithm of this invention has been validated end-to-end on a simulated dataset (50 cases × 5 causes). The validation cases cover three scenarios: normal students, typical problem students, and students with abnormal data quality. Both MDW-DSM and MBL-BC were validated by numerical substitution calculation. (5) Comparative experiment: On the same simulated dataset of 50 cases, the MDW-DSM+MBL-BC dual-model architecture of this invention was compared with the traditional arithmetic mean scoring method. The method of this invention achieved an accuracy of 87.3% in matching causes, which is significantly better than the 62.1% of the traditional method (an improvement of 25.2 percentage points). In the abnormal answer scenario, this invention reduced the misdiagnosis rate from 34.7% of the traditional method to 8.2% by weighting the answer quality function and intercepting the credibility of MBL-BC, thus verifying the technical advantages of this invention in data quality control and diagnostic accuracy.
[0184] Furthermore, the technical solution of this invention has been verified through actual student cases, further demonstrating the effectiveness of the technical solution:
[0185] Case 1: Student A, a seventh-grade student, scored 42 points on the midterm math exam (out of 120, 35% pass rate). Based on the MDW-DSM model's ten learning dimensions (D1~D10), the student's learning motivation score was 3.0 / 10 (close to the healthy zone), but their self-regulation score was 5.0 / 10 (yellow warning), numerical behavior score was 4.3 / 10 (yellow warning), and family environment score was 4.0 / 10 (yellow warning). The MDW-DSM's response quality function automatically identified abnormal responses in the quick-response questions and downweighted them. The consistency correction coefficient detected logical contradictions in the positive and negative responses within the self-regulation dimension (θ_k=0.5), and the knowledge gap amplification coefficient non-linearly amplified the low self-regulation score (γ_k=1.251). The final output standardized score was S_k=1.439 (out of 10), accurately identifying the student's core problems as a triple superposition of weak self-regulation, numerical dependence, and family conflict. The MBL-BC model assessed the reliability of the diagnosis at T=0.513 (low reliability). The system automatically marked it as requiring manual review. The accuracy of the diagnosis was verified after manual confirmation by the teacher.
[0186] Case 2: Student B, a senior high school student, scored 27 in English (out of 150, 18% pass rate) compared to 120 in Mathematics (out of 150, 80% pass rate), a difference of 63 percentage points. MDW-DSM diagnosis showed a score of 4.0 / 10 (red danger zone) in the learning strategies dimension and 5.0 / 10 (yellow warning zone) in the learning motivation dimension, while scoring 8.0 / 10 (good) in the learning habits dimension and 6.0 / 10 (above average) in the subject attitude dimension. The cross-dimensional etiology matching algorithm, based on the dimension-etiology association matrix W_jk, automatically matched P0-level etiologies: L-051 sleep deprivation fatigue (severe, less than 6 hours of sleep per day), L-016 severe subject imbalance (severe, 63 percentage points difference), and L-056 subject bias - English resistance (severe). The probabilistic iterative etiology identification algorithm, through multiple rounds of adaptive follow-up questions, increased the posterior probability difference between L-056 (English resistance) and L-016 (severe subject imbalance) from 0.15 to 0.52 (greater than the threshold of 0.5), accurately identifying English acquisition helplessness as the core etiology, rather than simply subject imbalance. The MBL-BC assessment had a confidence level of T=0.78 (medium confidence), and the diagnostic conclusion was confirmed as valid after manual review.
[0187] The two real-world cases above verify the technical effectiveness of the MDW-DSM model of this invention in terms of abnormal answer weighting, logical contradiction detection, cross-dimensional cause matching, and probabilistic iterative identification. They demonstrate that the method of this invention can accurately identify the root causes of learning problems of students of different grades and types in real educational scenarios.
[0188] The solution of this invention has entered the stage of large-scale production environment verification:
[0189] (1) Scale of verification: The actual verification has been carried out simultaneously in four campuses, covering about 450 students. Data collection and verification of diagnostic results are ongoing.
[0190] (2) Supporting tool system: A complete set of assessment teaching materials (5 volumes, totaling more than 400,000 words) that are compatible with this invention has been compiled, providing standardized content support for the large-scale application of the system.
[0191] (3) Implementation preparation: Relevant teacher training and operational preparations are being carried out simultaneously to ensure the standardized use of the system after its implementation.
[0192] (4) Promotion and application plan: Starting from the fall semester, we plan to carry out large-scale application verification on a larger scale, at which time the verification sample size and data diversity will be greatly improved.
[0193] As the scale of verification expands and the amount of real-world data accumulates, the system achieves self-optimization through approximate Bayesian parameter smoothing and sliding window monitoring mechanisms. The diagnostic accuracy and reliability assessment precision will continue to improve, forming a positive cycle of more data, more accurate diagnosis, better intervention, and more data. As the data flywheel continues to iterate, the accuracy in real-world scenarios will be improved.
[0194] This invention also provides a computer device. Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 7 As shown, the computer device includes: an input system, an output system, a memory, a storage medium, and a processor; the memory and storage medium are used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the learning evaluation method based on multi-dimensional weighted scoring and multi-level credibility provided in the above embodiments; wherein the input system, output system, memory, and processor can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.
[0195] The memory can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the learning evaluation method based on multi-dimensional weighted scoring and multi-level credibility as described in the embodiments of the present invention. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0196] The input system can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device; the output system may include display devices such as displays.
[0197] The processor executes various functional applications and data processing of the device by running software programs, instructions, and modules stored in memory, thereby realizing the learning evaluation method based on multidimensional weighted scoring and multi-level credibility.
[0198] The computer equipment provided above can be used to execute the learning evaluation method based on multidimensional weighted scoring and multi-level credibility provided in the above embodiments, and has corresponding functions and beneficial effects.
[0199] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as provided in the above embodiments. The storage medium can be any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape systems; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (e.g., in different computer systems connected via a network). The storage medium can store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.
[0200] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as described in the above embodiments, but can also perform related operations in the learning evaluation method based on multidimensional weighted scoring and multi-level credibility provided in any embodiment of the present invention.
[0201] The technical solution of the present invention has been described in conjunction with preferred embodiments. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from these changes or substitutions will all fall within the scope of protection of the present invention.
[0202] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A learning evaluation method based on multi-dimensional weighted score and multi-level credibility, characterized in that, A multidimensional weighted diagnostic scoring model MDW-DSM and a multi-level bias-corrected confidence assessment model MBL-BC are constructed, which are linked front and back. The MDW-DSM model is connected to the original assessment response data at the bottom and derives two subordinate extension algorithms at the top: a cross-dimensional etiology matching algorithm and a probabilistic iterative etiology identification algorithm. The MBL-BC model receives the full-dimensional standardized scoring data and original assessment metadata output by the MDW-DSM model and completes the quantitative grading of diagnostic credibility. 2.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 1, wherein, The Multidimensional Weighted Diagnostic Scoring Model (MDW-DSM) transforms raw response data into standardized dimensional scores, incorporating three collaborative technical elements, expressed as follows: S_k=[Σw_ki·f(x_ki,t_ki)] / [Σw_ki]·θ_k·γ_k; Where S_k represents the final standardized score of the k-th learning dimension; w_ki represents the fixed weight coefficient of the i-th question under the k-th dimension; The three technical elements are as follows: The answer quality function is f(x_ki,t_ki), f(x_ki,t_ki)=x_ki·max(c_min,1+(t_o-|t_ki-t_o|) / t_o·η); it simultaneously incorporates the original score x_ki of the question and the actual answering time t_ki of a single question, where t_o is the optimal standard answering time for this question after large-sample calibration; η is the time offset adjustment constant; c_min is the lower bound of the minimum correction coefficient (preferably 0.5) to prevent the correction term from becoming excessively negative during extreme abnormal answers, which would result in a negative function value; abnormal answer sheets that are answered too quickly or too slowly are weighted down; The consistency correction coefficient θ_k is given by θ_k = 1 - 1 / [m(m-1)]·Σ|x_kp-x_kq^(rev)|, where m is the total number of positive and negative verification questions within the k-th assessment dimension; x_kp is the standardized score of the positive questions within the dimension; and x_kq^(rev) is the converted score of the negative paired questions within the same dimension. Based on the same-dimensional positive and negative paired question verification logic, the dimension correction coefficient is lowered when contradictory answers occur. The knowledge gap amplification coefficient γ_k is given by γ_k = 1 + α·exp(-β·S_k^(base)); where S_k^(base) is the baseline score of the dimension before amplification after the first two coefficients are calculated; α and β are preset nonlinear control constants; a nonlinear exponential formula is used to amplify the weight of severely deficient dimensions with low baseline scores. The three technical elements sequentially achieve invalid data filtering, logical contradiction correction, and low-scoring dimension nonlinear amplification, outputting the standardized score S_k for each learning dimension. 3.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 2, characterized in that, After the MDW-DSM outputs the dimensional score S_k, a cross-dimensional etiology matching method is added; a dimensional-etiology association matrix W_jk is constructed, and the matching confidence C_j for each etiology is calculated, using the following formula: C_j=[ΣW_jk·S_k] / [max(ΣW_jk·10,ε)]·δ_j; where 10 is the full score for each dimension, using a score scale of 0~10; max(ΣW_jk·10,ε) is the maximum value of the theoretical full score weighted sum of all dimensions associated with cause j; Wherein, C_j is the final matching confidence score of the j-th etiology; W_jk is the pre-constructed dimension-etiology association matrix, and the values of the matrix cells represent the association weight of the k-th learning dimension to the j-th causative factor; ε is the minimum value zero constant; δ_j is the etiology discrimination coefficient; constant 10 is the upper limit of the score for each learning dimension, with a maximum score of 10; it is used to sum and normalize the weights to the range of 0 to 1; the etiology is determined by weighted fusion of multi-dimensional scores, and the matching weight of etiologies with historically high confusion rates is calibrated using the discrimination coefficient δ_j. 4.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 3, characterized in that, When the confidence scores of two or more etiologies are too close to distinguish, a probabilistic iterative etiology identification method is activated. This method continuously updates the posterior probability of the etiology by supplementing assessment data through multiple rounds of targeted follow-up questions. The posterior probability formula is: P_(t+1)=softmax(P_t+Δ_t); Where P_t represents the posterior probability distribution of each candidate etiology in the t-th iteration; Δ_t is the probability correction increment brought about by the targeted adaptive follow-up questioning to collect new data in this round; and the softmax function realizes probability normalization. The calculation method for Δ_t is as follows: the newly collected n follow-up questions construct a new scoring vector Δs=(Δs_1,Δs_2,...,Δs_K), where K is the total number of learning dimensions, and each Δs_k is obtained by weighted summation of the answer quality function of the new questions; Δ_t=λ·softmax(Δs), where λ is the iterative learning rate constant, and the softmax function transforms the new scoring vector into a probability increment distribution; The iteration continues until the confidence difference between the primary and secondary causes reaches a preset threshold, at which point the final diagnosis of the cause is determined. 5.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 1, wherein, The Multi-Level Bias Correction Confidence Assessment Model (MBL-BC) consists of two parts: a five-layer bias detection system and a product probability model. The five-layer bias detection system includes a data quality layer, a student status layer, an algorithm bias layer, an environmental factor layer, and a timeliness layer. The corresponding assessment risk coefficient r_i is collected at each layer, and each layer is configured with a dynamically adjustable weight coefficient α_i. The expression for the product probability model is: T=T_0·Π(1-r_i·α_i / Σα_i)·exp(-β·n_risk); The detection results of the five-layer bias detection system are integrated into a continuous credibility score T of 0-1 through the product probability model. 6.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 1, wherein, The front-end collection method for the original assessment response data includes any one or more of the following three schemes in combination: Gamified challenge-based assessments replace questionnaires; Eye movement and facial expression physiological data acquisition; Automatic scoring of open-ended question answering and NLP semantic parsing; After generating standardized x_ki and t_ki parameters using various acquisition methods, they can all be integrated into the MDW-DSM model to complete the calculation. 7.The learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to claim 1, wherein, It also includes a self-evolutionary closed-loop step: collecting student learning improvement feedback data after the intervention is implemented, and progressively optimizing the response quality function parameters, knowledge gap amplification coefficient parameters, weight coefficients α_i of each level of MBL-BC, and all parameters of the dimension-cause association matrix W_jk of MDW-DSM through an approximate Bayesian parameter smoothing algorithm (using the Bayesian sequence weighted fusion method); coupled with a sliding window accuracy monitoring module to monitor the changes in model accuracy in real time, and automatically triggering parameter correction when the accuracy shows a continuous drift within the sliding window.
8. A learning evaluation system based on multi-dimensional weighted score and multi-level credibility, for implementing the learning evaluation method based on multi-dimensional weighted score and multi-level credibility according to any one of claims 1-7, characterized in that, The system deploys the three-factor weighted scoring algorithm of the Multidimensional Weighted Diagnostic Scoring Model (MDW-DSM), namely the response quality function f(x_ki,t_ki), the consistency correction coefficient θ_k, and the knowledge gap amplification coefficient γ_k, as well as the product probability credibility calculation algorithm of the Multilevel Bias Correction Confidence Assessment Model (MBL-BC); the system includes: MDW-DSM computation module: used for the MDW-DSM model to interface with the original assessment and response data downwards, and to derive two subordinate extension algorithms upwards: cross-dimensional etiology matching algorithm and probabilistic iterative etiology identification algorithm; The MBL-BC credibility calculation module is used by the MBL-BC model to receive the full-dimensional standardized scoring data and original evaluation metadata output by the MDW-DSM model, and to complete the quantitative grading of diagnostic credibility.
9. The learning evaluation system based on multi-dimensional weighted score and multi-level credibility of claim 8, further comprising: Front-end data acquisition module, diagnostic report output module, and data feedback self-evolution module; The front-end data acquisition module collects the raw data of the responses, which is then processed sequentially by the MDW-DSM calculation module and the MBL-BC credibility calculation module. The diagnostic report output module outputs reports in a hierarchical manner, and the feedback data flows back to the data feedback self-evolution module to complete the parameter iterative optimization.
10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the learning evaluation method based on multidimensional weighted scoring and multi-level credibility as described in any one of claims 1-7.
