Monitoring device and monitoring method for network distance education

Abstract

The invention relates to the technical field of remote education, and discloses a monitoring device and method for network remote education, and the method comprises the steps: collecting micro-motion data, and generating an original data sequence for concentration analysis; micro-action feature vectors are extracted, and time sequence feature vectors are generated; constructing a personalized concentration scoring model, and generating a concentration score and a personalized threshold value based on the time sequence feature vector and the time decay function; behavior drift detection and model adaptation: monitoring a change trend of a learner behavior mode along with time based on a time sequence feature vector, executing behavior drift detection, and updating concentration scoring model parameters according to the behavior drift detection; generating a concentration state evaluation report, performing multi-level classification on the concentration state of the learner, and generating a real-time evaluation report; according to the method, personalized concentration evaluation is realized, and customized concentration monitoring strategies are provided for different learners by constructing an individual historical behavior mode database and a personalized threshold adjustment mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of remote education, and more specifically, to a monitoring device and a monitoring method for network remote education. Background Art

[0002] With the rapid development of Internet technology, online distance education has become an important part of the modern education system. However, due to the lack of face-to-face supervision and interaction, monitoring learners' concentration in distance education has always been a technical problem that needs to be solved urgently.

[0003] The existing distance education monitoring systems mainly have the following technical problems: the existing monitoring systems generally use unified standards to evaluate the concentration status of all learners, ignoring the individual behavioral differences among learners, especially for special learners (such as ADHD, autism, etc.), the concentration assessment is seriously inaccurate; it is difficult to capture tiny attention fluctuations in real time, and it is impossible to effectively capture micro-expressions and micro-movements at the millisecond level, resulting in insufficient granularity in concentration assessment and inability to intervene in learning state changes in a timely manner; the accuracy of concentration judgment is low, and it is difficult to accurately distinguish different degrees of learning engagement. There are few learning state classifications and it is impossible to provide more detailed state distinctions, resulting in the inability to provide teachers with effective learning state feedback; there is a lack of adaptability to changes in learners' behavior patterns, and it is difficult to make adaptive adjustments based on long-term changes in learners' behavior patterns, resulting in a decrease in the accuracy of the system after long-term use.

[0004] Therefore, there is an urgent need for a distance education monitoring technology solution that can adapt to the individual differences of different learners, has the ability to detect small attention fluctuations, provides fine-grained concentration assessment and is self-adaptive. Summary of the invention

[0005] The present invention provides a monitoring device and a monitoring method for network distance education, which solve the technical problems in the related art in terms of personalized evaluation, capturing of small attention fluctuations and accuracy of concentration judgment.

[0006] The present invention provides a monitoring method for network distance education, comprising:

[0007] Micro-motion data acquisition: using a high-frame-rate visual acquisition system to capture learners’ facial and upper limb micro-motion data, generating raw data sequences for concentration analysis;

[0008] Micro-motion feature vector extraction: receiving the original data sequence, applying computer vision algorithm to process the key frame sequence, extracting facial micro-expressions and subtle body movement features, and generating time series feature vectors;

[0009] Construction of a personalized concentration score model. Based on the time series feature vector and the time decay function, a personalized concentration score model is constructed to generate a concentration score and a personalized threshold;

[0010] Behavior drift detection and model adaptation. Based on the time series feature vector, the changing trend of the learner's behavior pattern over time is monitored, behavior drift detection is performed, and the parameters of the concentration score model are updated accordingly to achieve continuous adaptation;

[0011] Generation of a concentration status assessment report. Based on the personalized threshold and the concentration score output by the concentration score model, combined with the results of behavior drift detection, the concentration status of the learner is classified at multiple levels, and a real-time assessment report is generated.

[0012] Furthermore, the micro-action data collection includes the following steps:

[0013] Collection device configuration. Install a high-frame-rate camera on the computer device used by the learner to ensure that the camera position is fixed and can clearly capture the learner's face and upper limb area;

[0014] Data collection preprocessing. The system real-time collects the learner's video stream data and performs face detection, illumination normalization processing, and key point detection;

[0015] Time window segmentation. The continuously collected video stream is segmented according to a fixed time window to form a sliding time window sequence;

[0016] Key frame extraction. For the video sequence within each time window, a key frame extraction algorithm is applied to select representative frames.

[0017] Furthermore, the micro-action feature vector extraction includes the following steps:

[0018] Facial micro-expression feature extraction. For the facial area in each key frame, the feature of eyebrow movement amplitude, the feature of mouth corner change, the feature of pupil dilation, and the feature of eye blink frequency are extracted to form a facial micro-expression feature vector;

[0019] Limb movement feature extraction. For the upper limb area in each key frame, the feature of head posture, the feature of hand activity, and the feature of sitting posture change are extracted to form a limb movement feature vector;

[0020] Time series feature construction. The facial micro-expression feature vector and the limb movement feature vector are combined to construct a multi-modal time series feature matrix;

[0021] Application of the time decay function. The time decay function is applied to weight the time series features so that recent behaviors obtain higher weights.

[0022] Furthermore, the construction of the personalized concentration score model includes the following steps:

[0023] Feature weight learning, in which for the extracted micro-action features, a multi-variable linear regression model is applied to learn the feature weights;

[0024] Concentration score calculation, in which based on the learned feature weights, a concentration score function is calculated;

[0025] Individual historical behavior pattern construction, in which a historical behavior pattern database is maintained for each learner, recording the typical micro-action feature patterns in high-concentration and low-concentration states;

[0026] Personalized threshold adjustment, in which based on the individual historical behavior patterns, a personalized threshold function is calculated.

[0027] Furthermore, the behavior drift detection and model adaptation include the following steps:

[0028] Behavior representation model construction, in which a learner behavior representation model is constructed to map the original micro-action features to a low-dimensional representation space;

[0029] Behavior drift metric calculation, in which based on the behavior representation model, the behavior drift metric between the current time point and the reference time point is calculated;

[0030] Drift threshold adaptive adjustment, in which the system dynamically adjusts the threshold for drift detection according to the historical behavior stability of the learner;

[0031] Incremental learning to update the model, in which when behavior drift is detected, the system triggers an incremental learning algorithm to update the parameters of the micro-action interpretation model.

[0032] Furthermore, the generation of the concentration state evaluation report includes the following steps:

[0033] Concentration state classification, in which the concentration score is mapped to multiple different learning engagement states based on the personalized threshold;

[0034] Concentration fluctuation detection, in which a sliding time window algorithm is applied to monitor the fluctuation of the concentration state;

[0035] State transition event detection, in which the transition events of the learner's concentration state are detected, especially the transition from high concentration to low concentration;

[0036] Evaluation report generation, in which based on the above analysis results, the system generates a concentration state evaluation report.

[0037] Furthermore, in the time window segmentation, the length of each time window is 500 milliseconds, and the overlapping rate of adjacent windows is 50%.

[0038] Further, in the feature weight learning, a regularization technique is applied to prevent overfitting, where the feature weight w i is learned as follows:

[0039]

[0040] where is the feature weight, and i2 is the index of the feature; is the attention label annotated by an expert, which is the sample attention value annotated by the expert; j1 is the index of the sample; is the j1-th sample, which contains all the micro-action feature data at a specific time point; m is the number of samples; represents the summation operation from j1 = 1 to j1 = m, which sums over all training samples; n is the number of features, represents the summation operation from i2 = 1 to i2 = n, which sums over all features, α is the regularization coefficient, which controls the influence intensity of the L1 regularization term, is the L1 regularization term, which is used for feature selection and promotes some weights to approach zero; is the time decay function, which is applied to the j1-th sample, and i1 is the index of the time window, which contains all the micro-action feature data at a specific time point; is the multi-modal time series feature matrix, which is applied to the j1-th sample and contains all the micro-action feature data at a specific time point; is the minimization objective function for solving the feature weight

[0041] Further, in the incremental learning and updating model, to prevent catastrophic forgetting, the system adopts an elastic weight consolidation strategy:

[0042] θ t+1 = γ·θ t+1 + (1 - γ)·θ t ;

[0043] where θ t+1 and θ t respectively represent the updated and pre-updated model parameters, t represents the time point, and γ is the weight coefficient, which controls the fusion ratio of the old and new model parameters.

[0044] A monitoring device for network distance education includes a memory and one or more processors. Executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned monitoring method for network distance education.

[0045] The beneficial effects of the present invention are: realizing personalized concentration assessment, providing customized concentration monitoring strategies for different learners by constructing an individual historical behavior pattern database and a personalized threshold adjustment mechanism, thereby improving the accuracy of concentration assessment, especially for special learners (such as ADHD, autism, etc.).

[0046] It provides millisecond-level attention fluctuation detection capabilities. Using a high-frame rate camera and key frame extraction algorithm, combined with a sliding window mechanism, the system can capture tiny changes in attention and achieve millisecond-level concentration fluctuation detection, effectively supporting teachers to intervene in a timely manner.

[0047] It realizes fine-grained classification of learning status by mapping the continuous score of concentration into 8 discrete states (high concentration, active participation, normal attention, mild distraction, moderate distraction, severe distraction, complete disengagement and absenteeism), providing a more refined learning status assessment compared with the traditional dichotomy that only distinguishes between "focused / inattentive".

[0048] With adaptive learning capabilities, the system can monitor changes in learners’ behavior patterns through behavior drift detection and model incremental update mechanisms, and automatically adjust model parameters to maintain long-term evaluation accuracy.

[0049] The invasiveness of the monitoring system is reduced, and non-contact visual analysis technology is used. There is no need for learners to actively cooperate or wear special equipment, which does not interfere with the normal learning process and improves user experience and system acceptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a monitoring method for network distance education in the present invention;

[0051] Figure 2 It is a micro-motion data collection flow chart of the present invention;

[0052] Figure 3 It is a flow chart of extracting micro-motion feature vectors of the present invention;

[0053] Figure 4 Constructing a flow chart for the personalized concentration scoring model of the present invention;

[0054] Figure 5 The behavior drift detection and model adaptation flow chart of the present invention;

[0055] Figure 6 A flow chart for generating a concentration status assessment report of the present invention. DETAILED DESCRIPTION

[0056] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein, and that changes can be made to the functions and arrangements of the elements discussed without departing from the scope of protection of the content of this specification. Each example may omit, substitute, or add various processes or components as needed. Additionally, the features described in some examples may also be combined in other examples.

[0057] In at least one embodiment of the present invention, a monitoring method for network distance education is disclosed, as Figures 1 to 6 shown, including the following steps:

[0058] Step 1, micro-action data acquisition, using a high-frame-rate vision acquisition system to capture the micro-action data of the learner's face and upper limbs, generating an original data sequence for concentration analysis;

[0059] The specific implementation process includes:

[0060] Step 1.1, acquisition device configuration;

[0061] Install a high-frame-rate camera (sampling rate not less than 60 frames per second) on the computer device used by the learner, ensuring that the camera position is fixed and can clearly capture the learner's face and upper limb area. The video resolution collected by the camera is not less than 720p to ensure the accuracy of subsequent micro-action feature extraction.

[0062] Step 1.2, data acquisition preprocessing;

[0063] The system continuously acquires the video stream data of the learner and performs the following preprocessing:

[0064] Apply a face detection algorithm to locate the face area in the video and determine the region of interest (ROI);

[0065] Perform illumination normalization on the detected face area to reduce the influence of environmental light changes on subsequent analysis;

[0066] Apply a key point detection algorithm to locate the positions of 68 key points on the face and the main joint points of the upper limbs.

[0067] Step 1.3, time window segmentation;

[0068] Segment the continuously acquired video stream according to a fixed time window. Each time window has a length of 500 milliseconds, and the overlapping rate of adjacent windows is 50%, forming a sliding time window sequence; the data within each time window constitutes a basic unit for micro-action analysis.

[0069] Step 1.4, key frame extraction;

[0070] For each video sequence within a time window, apply a key-frame extraction algorithm to select representative frames. The algorithm is based on the frame difference metric D f Calculate:

[0071]

[0072] where D f (i, j) is the difference degree between the i-th frame and the j-th frame. i and j are frame indices within the time window, and F i (p) and F j (p) represent the positions of key point p in the i-th frame and the j-th frame respectively; P represents the set of key points, including the set of 68 facial key points and the main joint points of the upper limbs; ||·||2 represents the Euclidean distance, which is used to calculate the straight-line distance between two points in space; denotes the summation over all key points. The system selects the set of frames with the largest difference degree within the time window as the key-frame sequence for subsequent feature extraction.

[0073] For the key-frame sequence output in this step, the quality and representativeness of the key frames directly affect the accuracy and integrity of feature extraction: Through the 500-millisecond sliding time-window mechanism, the system can capture micro-action changes at the millisecond level, providing sufficient time resolution for subsequent analysis, which is crucial for realizing the micro-attention fluctuation detection ability of the present invention.

[0074] Step 2, Extract micro-action feature vectors. Receive the original data sequence input in Step 1, apply computer vision algorithms to process the key-frame sequence, extract facial micro-expression and subtle limb movement features, and generate a time-series feature vector;

[0075] The key-point detection results (68 facial key points and upper-limb joint points) in Step 1 provide the basic data for the extraction of facial micro-expression features and limb movement features in this step;

[0076] The specific implementation process includes:

[0077] Step 2.1, Extract facial micro-expression features;

[0078] For the facial region in each key frame, extract the following micro-expression features:

[0079] Extract the eyebrow movement amplitude feature, calculate the displacement vector V of the eyebrow key points eb to quantify the degree of eyebrow raising or lowering;

[0080] Extract the mouth-corner change feature, calculate the displacement vector V of the mouth-corner key points mc to capture expression changes such as smiling or frowning;

[0081] Extract the pupil dilation feature and calculate the area change rate ΔA of the pupil region p , which reflects the degree of concentration;

[0082] Extract the eye blink frequency feature. By observing the changes in the open / closed state of the eyes in consecutive frames, calculate the number of blinks f per unit time blink , which is used to evaluate fatigue and concentration.

[0083] Combine these features to form the facial micro-expression feature vector F face , which contains the combined representation of all facial features.

[0084] Step 2.2, Limb movement feature extraction;

[0085] For the upper limb region in each key frame, extract the following movement features:

[0086] Extract the head pose feature: Calculate the three-dimensional rotation angle θ of the head head , which includes the pitch angle, yaw angle, and roll angle, and reflects the degree of concentration and fatigue. Extract the hand movement feature: Calculate the movement speed and acceleration of the hand key points to form the hand movement feature vector F hand , which contains information such as hand movement speed and acceleration. Extract the sitting posture change feature: Calculate the position change of the trunk key points to form the sitting posture feature vector F posture , which represents the position change information of the trunk key points

[0087] Combine these features to form the limb movement feature vector F body , which is the combined representation of all limb features.

[0088] Step 2.3, Temporal feature construction;

[0089] Combine the facial micro-expression feature vector F face extracted in Step 2.1 and the limb movement feature vector F body extracted in Step 2.2 to construct a multi-modal temporal feature matrix

[0090]

[0091] where is the multi-modal temporal feature matrix, i1 is the index of the time window, x is the video sequence within the time window, F face and F body represent the facial and limb feature vectors respectively, and ΔF face and ΔF body represent the time derivatives of the facial and limb features respectively, which are used to capture the changing trend of the features over time.

[0092] Step 2.4, Application of the time decay function;

[0093] Apply the time decay function to weight the temporal features, giving higher weights to recent behaviors:

[0094]

[0095] where is the time decay function, i1 is the index of the time window, x is the video sequence within the time window, e represents the base of the exponential function (the base of the natural logarithm), λ is the time decay function parameter that controls the rate of time decay, t current is the current time, the time point when the system is processing data, is the timestamp of the i2-th feature, and i2 is the index of the feature, recording the specific time when the feature is collected;

[0096] The decay function ensures that the system is more sensitive to changes in the learner's recent behaviors, improving the real-time performance of the concentration assessment.

[0097] This step receives the key frame sequence output in Step 1 as input. By deeply analyzing the facial expressions and body movements in these frames, it extracts micro-action features related to concentration; the extracted multi-modal temporal feature matrix and the weighted features after applying the time decay function are the output; the application of the time decay function ensures that the system can pay more attention to the learner's recent behaviors, making the concentration assessment highly real-time and sensitive, which is crucial for capturing instantaneous attention changes.

[0098] Step 3, Construction of a personalized concentration scoring model. Based on the temporal feature vector and the time decay function, construct a personalized concentration scoring model to generate a concentration score and a personalized threshold;

[0099] The specific implementation process includes:

[0100] Step 3.1, Feature weight learning;

[0101] For the extracted micro-action features, apply a multi-variable linear regression model to learn the feature weights

[0102]

[0103] where is the feature weight, and i2 is the index of the feature; is the concentration label annotated by experts, the sample concentration value annotated by experts, used for model training; j1 is the index of the sample; is the j1-th sample, containing all micro-action feature data at a specific time point; m is the number of samples, which is the total amount of data used to train the model; represents the summation operation from j1 = 1 to j1 = m, summing over all training samples; n is the number of features, which is the total number of dimensions of the extracted micro-action features, represents the summation operation from i2 = 1 to i2 = n, summing over all features. α is the regularization coefficient, which controls the influence intensity of the L1 regularization term, is the L1 regularization term, which is used for feature selection and promotes some weights to approach zero; is the time decay function, which is applied to the j1-th sample. i1 is the index of the time window, containing all micro-action feature data at a specific time point; is the multi-modal temporal feature matrix, which is applied to the j1-th sample and contains all micro-action feature data at a specific time point; is the objective function to be minimized for solving the feature weights

[0104] Step 3.2, calculation of the concentration score;

[0105] Based on the learned feature weights, calculate the concentration score function:

[0106]

[0107] where f(x) is the concentration score function, x is the video sequence within the time window, is the multi-modal temporal feature matrix, is the time decay function, is the feature weight, and i2 is the index of the feature; represents the summation operation from i2 = 1 to i2 = n, summing over all features. n is the number of features, which is the total number of dimensions of the extracted micro-action features. This function performs a weighted combination of the temporal feature matrix and the time decay function to output a concentration score ranging from [0, 1], where 0 indicates completely unfocused and 1 indicates highly focused.

[0108] Step 3.3, construction of the individual historical behavior pattern;

[0109] Maintain a historical behavior pattern database H for each learner u , recording its typical micro-action feature patterns in high-concentration and low-concentration states:

[0110]

[0111] where H u is the individual historical behavior pattern database, respectively represent the multi-modal temporal feature matrices of the 1st, 2nd, and k-th historical records within the time window i1, which are the feature records saved in the historical database; s1, s2, hist represent the corresponding concentration state labels, corresponding to the concentration states of the historical records; k represents the number of historical records, the total number of records saved in the database. The system regularly updates this database to ensure that it reflects the latest behavior patterns of the learner. hist Step 3.4, personalized threshold adjustment;

[0112] Based on the individual historical behavior pattern H

[0113] , calculate the personalized threshold function: u where

[0114]

[0115] is the personalized threshold function for the i1-th time window, T is the preset benchmark threshold, the standard threshold preset by the system; u is the individual learner, and i1 is the index of the time window; α base is the user characteristic adjustment factor, which adjusts the threshold sensitivity according to the degree of fluctuation of the user's behavior pattern, and is calculated by analyzing the variance of the user's historical behavior pattern: u where α

[0116]

[0117] represents the user characteristic adjustment factor, β is the adjustment parameter, which controls the influence intensity of the user characteristics on the threshold adjustment; Var(H u ) is the variance of the historical behavior pattern, which measures the degree of fluctuation of the learner's behavior pattern; u is in fractional form and decreases as the variance increases. For learners with large fluctuations in behavior patterns (such as patients with attention deficit hyperactivity disorder), the system will reduce the threshold sensitivity to reduce misjudgments; for learners with stable behavior patterns, the system will increase the threshold sensitivity to enhance the detection ability of minor attention changes. This step receives the multi-modal temporal feature matrix output in Step 2

[0118] and the time decay weighted result to construct the concentration scoring function f(x) by learning the feature weights ; the individual historical behavior pattern database H constructed in this step and the personalized threshold function u are the outputs.

[0119] ​Step 4, Behavioral Drift Detection and Model Adaptation: Monitor the changing trend of the learner's behavior pattern over time based on the time-series feature vector, perform behavioral drift detection, and update the parameters of the concentration score model accordingly to continuously adapt to the changes in the individual's behavior pattern;

[0120] The specific implementation process includes:

[0121] Step 4.1, Construction of the Behavioral Representation Model;

[0122] Construct the learner's behavioral representation model f θ (u), which maps the original micro-action features to a low-dimensional representation space:

[0123]

[0124] where f θ (u) represents the low-dimensional representation vector of the learner's behavior, u represents the learner individual, and Encoder θ is an encoder network with parameters θ, a neural network structure for performing feature dimensionality reduction, which maps the high-dimensional micro-action feature matrix to a low-dimensional vector representation, represents the multi-modal time-series feature matrix of learner u within the time window i1;

[0125] Through this model, the system can effectively capture the essential features of the learner's behavior pattern and reduce noise interference.

[0126] Step 4.2, Calculation of the Behavioral Drift Metric;

[0127] Based on the behavioral representation model, calculate the behavioral drift metric between the current time point t and the reference time point t - Δ:

[0128]

[0129] where Drift(u, t) represents the behavioral drift metric between learner u at the current time point t and the reference time point t - Δ, and respectively represent the behavioral representation models at the current time point and the reference time point for behavioral drift calculation, Δ is the time interval parameter, defining the interval between the reference time point and the current time point, and ||·||2 represents the Euclidean distance. This metric reflects the degree of change in the learner's behavior pattern within the time period Δ.

[0130] Step 4.3, Adaptive Adjustment of the Drift Threshold;

[0131] The system dynamically adjusts the drift detection threshold τ drift (u) according to the historical behavior stability of the learner:

[0132]

[0133] where τ drift (u) represents the drift threshold of learner u, represents the set of drift metrics at the past N time points, and t py represents the py-th time point among the past N time points. Median represents the median, a robust statistic that is not affected by extreme values. MAD represents the median absolute deviation, a robust measure of dispersion. μ and σ represent the baseline and sensitivity of the drift threshold respectively;

[0134] This threshold adjustment method based on robust statistics can adapt to the behavioral fluctuation characteristics of different learners.

[0135] Step 4.4, update the model incrementally;

[0136] When behavioral drift is detected, i.e.:

[0137] Drift(u,t) > τ drift (u);

[0138] The system triggers the incremental learning algorithm to update the parameters θ of the micro-action interpretation model t :

[0139]

[0140] where θ t+1 and θ t represent the model parameters at the current time point and the reference time point respectively. η is the learning rate, which controls the step size of model parameter update. is the gradient of the loss function with respect to the parameter θ, which guides the direction of parameter update. D recent is the recently collected data sample, which is the new data for incremental learning;

[0141] To prevent catastrophic forgetting, the system adopts the elastic weight consolidation strategy:

[0142] θ t+1 = γ·θ t+1 +(1 - γ)·θ t ;

[0143] where γ is the weight coefficient, which controls the fusion ratio of the old and new model parameters;

[0144] This incremental update mechanism ensures that the model can gradually adapt to the slow changes in the learner's behavior pattern while maintaining the memory of the historical pattern.

[0145] This step and step 3 form a feedback loop relationship: when this step detects behavior drift, it triggers an incremental update of the parameters of the concentration scoring model in step 3 to ensure that the model can adapt to the long-term changes in the learner's behavior pattern; this step not only updates the parameters of the behavior representation model, but also indirectly adjusts the model output of step 3 by affecting the calculation of feature weights and the personalized threshold function , which forms an adaptive learning closed-loop and continuously improves the evaluation accuracy of the system.

[0146] The features extracted in step 2 also provide the original data for the behavior representation model in step 4, enabling the system to monitor the changing trend of the learner's behavior pattern over time;

[0147] In addition, the behavior drift detection result of this step also affects the judgment of the concentration state transition event in step 5, helping to distinguish the state transition caused by the real change in concentration and the judgment deviation caused by the change in behavior pattern;

[0148] This multi-level association ensures that the system can maintain high accuracy during long-term operation and effectively cope with the natural changes in the learner's behavior pattern.

[0149] Step 5, generation of the concentration state evaluation report, based on the personalized threshold and concentration score output by the concentration scoring model, combines the behavior drift detection result, classifies the learner's concentration state at multiple levels, and generates a real-time evaluation report;

[0150] The real-time evaluation report provides detailed learning status feedback for teachers; the specific implementation process includes:

[0151] Step 5.1, concentration state classification;

[0152] Map the concentration score f(x) to 8 different learning engagement states, based on the personalized threshold

[0153]

[0154] where State(u) represents the concentration state of learner u; are 7 personalized thresholds, which divide the continuous concentration score into 8 discrete states. Here, i1 represents the index of the time window, Classifier is the classifier, and maps the continuous concentration score f(x) to 8 discrete states: highly focused (S1), actively engaged (S2), normally focused (S3), mildly distracted (S4), moderately distracted (S5), severely distracted (S6), completely disengaged (S7), and out-of-seat state (S8).

[0155] The classifier adopts a rule-based decision tree structure and determines the current state category according to the concentration score and the set of thresholds.

[0156] Step 5.2, concentration fluctuation detection;

[0157] Apply the sliding time window algorithm to monitor the fluctuation of the concentration state:

[0158]

[0159] where Fluctuation(u, t) represents the concentration fluctuation of learner u at time point t;

[0160] represents the sequence of concentration scores at the past k fluc time points; f(x t ) represent the concentration scores at the (t - k) fluc th, (t - k) fluc +1th, and tth time points respectively, and t fluc represents the number of historical time points considered, which is used to determine the time window size for fluctuation detection;

[0161] stddev represents the standard deviation, a statistic that measures the degree of data dispersion.

[0162] The system sets a fluctuation threshold τ fluc , and when Fluctuation(u, t) > τ fluc , it is determined as a fluctuation and requires special attention; among them, the fluctuation threshold τ fluc is used to judge whether the concentration fluctuates.

[0163] Step 5.3, state transition event detection;

[0164] Detect the state transition event of the learner's concentration state, especially the transition from high concentration to low concentration:

[0165]

[0166] where Transition(u, t) represents the state transition event of learner u at time point t, and State(u) t and State(u) t-1 represent the concentration states of learner u at time points t and t - 1 respectively, and State(u) t ≠State(u) t-1 indicates that the concentration states of learner u at time points t and t - 1 have changed.

[0167] The system records the time points, durations, and types of state transitions, providing teachers with key information on changes in learners' attention.

[0168] Step 5.4, generation of the assessment report;

[0169] Based on the above analysis results, the system generates a concentration state assessment report, including the following content:

[0170] Current concentration state: Displays the classification results of the learner's current concentration;

[0171] Concentration historical trend chart: Shows the change curve of the concentration score over a period of time in the past;

[0172] State distribution statistics: Counts the proportion of time in different concentration states;

[0173] Key event reminders: Marks the key time points of concentration fluctuations and state transitions;

[0174] Personalized suggestions: Provide targeted learning intervention suggestions according to the learner's concentration pattern.

[0175] The report is presented in the form of visual charts, supports real-time updates, enabling teachers to intuitively understand the changes in the learner's concentration state and adjust teaching strategies in a timely manner.

[0176] This step is the final output link of the entire system, directly relying on the concentration scoring function f(x) calculated in Step 3 (construction of the personalized concentration scoring model) and the personalized threshold function The individual historical behavior pattern database H constructed in Step 3 u and the personalized threshold function are the key bases for classifying the concentration state in this step, enabling the system to provide customized concentration assessment results according to the behavioral characteristics of different learners.

[0177] By mapping continuous concentration scores to 8 discrete states, this step provides a fine-grained concentration state classification, greatly improving the accuracy of the assessment of learning engagement;

[0178] At the same time, the concentration fluctuation detection and state transition event detection in this step can timely capture the subtle changes and important turning points of the learner's attention, providing key opportunities for teachers to intervene. In addition, the concentration state assessment report generated in this step is not only a display of the system operation results, but also provides feedback data for the entire system - the intervention effect based on the report by teachers can be used to verify and improve the assessment accuracy of the system, forming a complete closed loop. Finally, this step works in coordination with Step 4 to ensure that the system can maintain high accuracy during long-term use and continuously provide valuable learning state feedback for teachers.

[0179] A monitoring device for online distance education includes a memory and one or more processors. Executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned monitoring method for online distance education.

[0180] Here, the present invention provides an embodiment 1: a concentration monitoring system in a remote university classroom;

[0181] In a certain university's online education platform, the concentration monitoring method of the present invention is applied, and the following data and processes are recorded:

[0182] System configuration and data collection process;

[0183] The online education platform of this university deployed a concentration monitoring system for 100 online learners. A 120fps high-definition camera was uniformly installed on the learners' devices. The system required the camera to be fixed above the monitor, and the pixel resolution of the collected video was 1080p. Before the learners started their online learning activities, the system required the learners to first go through a 5-minute calibration process. During the calibration process, the system collected the facial micro-expressions and body movements of the learners in the focused state and the distracted state for the training of subsequent personalized models.

[0184] Actual data shows that the system obtains 120 frames of images from the camera per second. After being screened by the key frame extraction algorithm, 30 frames are retained for analysis, and 15 key frames are extracted within an average time window of every 500 milliseconds. For special learners (such as those with attention deficit hyperactivity disorder), the system automatically adjusts the key frame extraction density, and 20 key frames are extracted within an average window of every 500 milliseconds to more precisely capture their rapidly changing micro-movements.

[0185] Feature extraction and personalized modeling effect;

[0186] During the facial micro-expression feature extraction process, the system established a facial micro-expression vector F containing 32-dimensional features for each learner face , including the eyebrow movement amplitude (8 dimensions), the change of the corner of the mouth (8 dimensions), the pupil dilation (4 dimensions), and the eye blinking frequency (4 dimensions), etc.; during the body movement feature extraction process, a body movement vector F containing 24-dimensional features was constructed body , including the head posture (9 dimensions), hand activities (6 dimensions), and sitting posture changes (9 dimensions).

[0187] For the collected data, the system trained a personalized concentration scoring model for each learner. Statistical data shows that for ordinary learners, the weight ratio of micro-expression features and body movement features in the model is approximately 60:40, while for special learners, this ratio is adjusted to 80:20, highlighting the importance of facial micro-expressions in concentration assessment.

[0188] In practical applications, the parameter λ of the time decay function is dynamically adjusted according to the course type: in theoretical courses that require continuous attention, λ is set to 0.8, making the system more sensitive to changes in attention; in practical operation courses, λ is reduced to 0.5, allowing learners more room for activities.

[0189] Behavior drift detection and model adaptation results;

[0190] During the one-semester (16 weeks) application process of the system, the behavior patterns of each learner were continuously monitored. The data shows that the behavior patterns of learners changed significantly in the early stage of the semester (weeks 1 - 4), triggering model updates 3 to 5 times per week on average; while in the middle and late stages of the semester (weeks 5 - 16), the behavior patterns tended to be stable, triggering model updates once every two weeks on average.

[0191] For learners with large fluctuations in behavior patterns, the system automatically increased the dimension of the behavior representation model from the default 16 dimensions to 24 dimensions to capture more complex changes in behavior patterns. During the incremental learning process, the weight merging coefficient γ is dynamically adjusted according to the model stability. During the stable period of the behavior pattern, γ remains in the range of 0.2 - 0.3, while when the behavior pattern changes significantly, γ is increased to the range of 0.5 - 0.6 to accelerate the model's adaptation to the new behavior pattern.

[0192] Evaluation report generation and teaching feedback;

[0193] The concentration status evaluation report generated by the system for the teacher side includes various visualization forms:

[0194] Real-time concentration status level meter, showing the overall concentration of the current class;

[0195] Individual concentration heat map, showing the concentration status of each learner in different colors;

[0196] Status conversion reminder. When more than 30% of the learners change from a high-concentration state to a low-concentration state, the system automatically sends a reminder to the teacher.

[0197] Teaching practice data shows that based on the concentration evaluation report provided by the system, teachers adjusted their teaching strategies 2 to 3 times per 45-minute course on average, such as adding interactive sessions, changing intonation, or playing visual stimulus materials, etc. These targeted adjustments increased the overall concentration of the course by 25% on average and improved the learning effect evaluation by 18%.

[0198] Here, the present invention provides an implementation example 2: a distance education system for special learning needs;

[0199] In an online education platform specifically for special learning needs (such as autism, attention deficit hyperactivity disorder, etc.), the method of the present invention has achieved results after special adaptation:

[0200] System adaptation for autistic learners;

[0201] For the distance education scenario of 30 mildly autistic learners, the system has particularly strengthened the eye movement tracking function in the micro-action feature extraction step, added an eye movement trajectory analysis module, and captured the typical visual attention patterns of autistic learners. The system has established a special analysis model for each autistic learner, including features such as eye movement hotspots, fixation duration distribution, and saccade frequency.

[0202] Data shows that the eye movement characteristics of autistic learners and their concentration states present a unique correlation pattern: when focusing on interesting content, the eye movement hotspots are concentrated and the fixation duration increases; while in a distracted state, the eye movement trajectory shows an obvious repetitive scanning pattern. The system adjusted the weights in the concentration score formula according to these characteristics, increased the weight of eye movement characteristics to 40%, and reduced the dependence on facial expression changes (down to 25%).

[0203] Threshold dynamic adjustment for learners with attention deficit hyperactivity disorder;

[0204] For 25 learners with attention deficit hyperactivity disorder (ADHD), the system applied a more flexible threshold dynamic adjustment strategy in the behavior drift detection step. System analysis shows that the action frequency of ADHD learners is on average 2.5 times higher than that of ordinary learners, but this does not necessarily mean inattentiveness.

[0205] By comparing the action patterns of ADHD learners in different concentration states, the system found that even in a highly concentrated state, these learners still show a certain frequency of small actions, but these actions have predictable periodicity and consistency.

[0206] Based on this, the system introduced an action consistency index C motion :

[0207]

[0208] where C motion is the action consistency index, and are action feature vectors within a continuous time window, n y is the number of action feature vectors within the time window, i y and iy +1 is the index of the action feature vector within the time window; cos() represents the cosine similarity, which is used to calculate the similarity degree of the action feature vectors in consecutive time windows; represents the summation of the action feature vectors of all consecutive time windows;

[0209] When C motion When the value is high, it indicates that the action patterns are consistent, and even if the action frequency is high, it may be in a focused state.

[0210] The actual application data shows that after using dynamic threshold adjustment, the recognition accuracy of the concentration state of ADHD learners has increased from the original 53% to 81%, significantly reducing misclassifications and avoiding the interference of the frequently triggered attention reminders of the system on learning.

[0211] Long-term effect evaluation and model evolution;

[0212] During the 6-month application process of the system, the evolution of the behavior patterns of special learner groups was recorded. The data shows that as the usage time extends, the behavior patterns of special learners gradually tend to be stable, and the prediction accuracy of the system for their concentration states steadily increases.

[0213] For the autism learner group, the average prediction accuracy in the first two months was 65%, and it increased to 88% in the sixth month; for the ADHD learner group, the average prediction accuracy in the first two months was 58%, and it increased to 82% in the sixth month. This progress is mainly attributed to the incremental learning ability of the system, which continuously adapts to and learns the behavior patterns of special learners.

[0214] It is particularly worth noting that by analyzing the evolution of the model parameters learned by the system, the researchers discovered the common behavior pattern characteristics of special learning groups, such as the phenomenon that the eye movement stay time of autism learners increases in the focused state, and the characteristics that although ADHD learners have small movements when focused, the consistency between the movements is high. These findings provide new research data and theoretical basis for the field of special education.

[0215] The above describes the embodiments of the present invention, but these embodiments are not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of this embodiment.

Claims

1. A monitoring method for network distance education, characterized in that Including: Micro-action data collection, using a high-frame-rate vision acquisition system to capture the micro-action data of the learner's face and upper limbs, and generating an original data sequence for concentration analysis; Micro-action feature vector extraction, receiving the original data sequence, applying computer vision algorithms to process the key frame sequence, extracting facial micro-expression and subtle limb movement features, and generating a time-series feature vector; Construction of a personalized concentration scoring model, based on the time-series feature vector and the time decay function, constructing a personalized concentration scoring model, and generating a concentration score and a personalized threshold; Behavior drift detection and model adaptation, monitoring the change trend of the learner's behavior pattern over time based on the time-series feature vector, performing behavior drift detection, and updating the parameters of the concentration scoring model accordingly to continuously adapt to the change of the individual behavior pattern; Generation of a concentration state evaluation report, based on the personalized threshold and the concentration score output by the concentration scoring model, combining the results of behavior drift detection, classifying the learner's concentration state at multiple levels, and generating a real-time evaluation report.

2. The monitoring method for network distance education according to claim 1, characterized in that, The micro-action data collection includes the following steps: Collection device configuration, installing a high-frame-rate camera on the computer device used by the learner, ensuring that the camera position is fixed and can clearly capture the learner's face and upper limb areas; Data collection preprocessing, the system real-time collects the video stream data of the learner, and performs face detection, illumination normalization processing, and key point detection; Time window segmentation, segmenting the continuously collected video stream according to a fixed time window to form a sliding time window sequence; Key frame extraction, for the video sequence within each time window, applying a key frame extraction algorithm to select representative frames.

3. The monitoring method for online distance education according to claim 1, wherein, The micro-action feature vector extraction includes the following steps: Facial micro-expression feature extraction, for the facial area in each key frame, extracting the feature of eyebrow movement amplitude, the feature of mouth corner change, the feature of pupil dilation, and the feature of eye blink frequency, to form a facial micro-expression feature vector; Limb movement feature extraction, for the upper limb area in each key frame, extracting the head pose feature, the hand activity feature, and the sitting posture change feature, to form a limb movement feature vector; Time-series feature construction, combining the facial micro-expression feature vector and the limb movement feature vector to construct a multi-modal time-series feature matrix; Application of the time decay function, applying the time decay function to weight the time-series features, so that recent behaviors obtain higher weights.

4. A monitoring method for online distance education according to claim 1, characterized in that, The construction of the personalized concentration scoring model includes the following steps: Feature weight learning, in which for the extracted micro-action features, applying a multi-variable linear regression model to learn the feature weights; Concentration score calculation, in which based on the learned feature weights, calculating a concentration scoring function; Construction of an individual historical behavior pattern, in which maintaining a historical behavior pattern database for each learner, recording the typical micro-action feature patterns in their high-concentration and low-concentration states; Personalized threshold adjustment, in which based on the individual historical behavior pattern, calculating a personalized threshold function.

5. A monitoring method for online distance education according to claim 1, characterized in that, The behavior drift detection and model adaptation includes the following steps: Construction of a behavior representation model, in which constructing a learner behavior representation model to map the original micro-action features to a low-dimensional representation space; Behavior drift metric calculation, where based on the behavior representation model, the behavior drift metric between the current time point and the reference time point is calculated; Drift threshold adaptive adjustment, where the system dynamically adjusts the threshold for drift detection according to the historical behavior stability of the learner; Incremental learning to update the model, where when behavior drift is detected, the system triggers an incremental learning algorithm to update the parameters of the micro-action interpretation model.

6. The monitoring method for online distance education according to claim 1, characterized in that, The generation of the concentration state evaluation report includes the following steps: Concentration state classification, where the concentration score is mapped to multiple different learning engagement states based on a personalized threshold; Concentration fluctuation detection, where the sliding time window algorithm is applied to monitor the fluctuation of the concentration state; State transition event detection, where the transition event of the learner's concentration state is detected, especially the transition from high concentration to low concentration; Evaluation report generation, where based on the above analysis results, the system generates a concentration state evaluation report.

7. A monitoring method for online distance education according to claim 2, characterized in that, In the time window segmentation, the length of each time window is 500 milliseconds, and the overlapping rate of adjacent windows is 50%.

8. A monitoring method for online distance education according to claim 4, characterized in that, In the feature weight learning, regularization techniques are applied to prevent overfitting, Among them, the learning of the feature weight w i is expressed as: wherein is the feature weight, and i2 is the index of the feature; is the concentration label annotated by the expert, which is the sample concentration value annotated by the expert; j1 is the index of the sample; is the j1-th sample, which contains all the micro-action feature data at a specific time point; m is the number of samples; denotes the summation operation from j1 = 1 to j1 = m, summing over all training samples; n is the number of features, denotes the summation operation from i2 = 1 to i2 = n, summing over all features, α is the regularization coefficient, controlling the influence intensity of the L1 regularization term, is the L1 regularization term, used for feature selection, promoting some weights to approach zero; is the time decay function, applied to the j1-th sample, i1 is the index of the time window, containing all micro-action feature data at a specific time point; is the multi-modal time series feature matrix, applied to the j1-th sample, containing all micro-action feature data at a specific time point; is the objective function to be minimized, solving for the feature weights 9. A monitoring method for network distance education according to claim 5, characterized in that, In the incremental learning to update the model, to prevent catastrophic forgetting, the system adopts the elastic weight consolidation strategy: θ t+1 = γ·θ t+1 + (1 - γ)·θ t ; where θ t+1 and θ t represent the updated and pre-updated model parameters respectively, t represents the time point, and γ is the weight coefficient that controls the fusion ratio of the old and new model parameters.

10. A monitoring device for network distance education, characterized in that, Comprising a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, it is used to implement a monitoring method for online distance education according to any one of claims 1-9.

Images