Student classroom psychological health early warning method and system based on multiple modes

Abstract

The invention provides a multi-mode-based student classroom psychological health early warning method and system. The method comprises the steps that the activity type of a current classroom is detected in real time through a plurality of sensors; an individualized participation standard is generated; the participation behaviors of the students in the classroom are monitored in real time, difference analysis is carried out on the participation behaviors and the personalized participation demand model, whether abnormal behaviors exist or not is judged, and if yes, marking is carried out; generating layered mental health early warning according to a difference analysis result; constructing an intelligent feedback model according to the type of the psychological health early warning and the activity type of the classroom, and feeding back the psychological health early warning to the teacher for intervention according to the type of the psychological health early warning; the individualized participation demand model is optimized according to the feedback of the teacher and the subsequent behaviors of the students, so that the model adapts to the long-term performance of the students, and closed-loop optimization is formed. According to the invention, the accuracy of mental health early warning is improved, the classroom management efficiency is improved through an effective intervention mechanism, and the method has a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the technical field of mental health early warning, and in particular to a multimodal student classroom mental health early warning method and system. Background Art

[0002] With the development of society and the advancement of educational concepts, the importance of students' mental health in the field of education has become increasingly prominent. Mental health not only affects students' learning efficiency, but also has a profound impact on their long-term development. Therefore, how to monitor and warn students' mental health problems in real time during the teaching process has become an important topic in educational research. At present, mental health monitoring systems have begun to be widely used in classroom environments. Common systems mainly rely on the monitoring of emotions, behaviors and physiological signals. For example, through facial expression recognition, voice emotion analysis, body language monitoring and other means, students' emotional state is evaluated, or wearable devices are used to collect students' physiological data, such as heart rate, skin conductance, etc., to infer their psychological stress and anxiety. These systems can identify students' abnormal emotional fluctuations and stress signals to a certain extent, and help teachers and psychological counselors to intervene in students' mental health at an early stage.

[0003] However, the existing mental health monitoring systems have some obvious shortcomings in practical applications. First, most systems use static and unified monitoring standards, lacking effective consideration of individual differences among students. For example, introverted students who speak less in class may be misjudged by the system as having abnormal mental states, while when extroverted students are silent in certain situations, the system may ignore their real mental health problems. Secondly, the existing systems have poor adaptability to dynamic environmental changes in the classroom. Classroom activities are usually diverse, covering different links such as explanations, discussions, group assignments, and independent learning. Each link has different effects on students' mental health, and a single monitoring strategy cannot accurately capture students' psychological changes in different activities. Traditional systems are prone to false alarms or omissions in these dynamic scenarios, and cannot adjust the warning standards in real time according to changes in classroom situations.

[0004] In addition, most existing systems lack intelligent feedback mechanisms for teachers, making it difficult for teachers to learn about students’ mental health status in class, and they do not receive timely intervention suggestions. Especially in large classes, teachers often fail to notice the emotional and behavioral changes of all students, resulting in potential mental health problems being ignored. These technical defects not only limit the application effect of the system, but also increase the burden on teachers in actual operation, reducing the accuracy and effectiveness of early warnings. Summary of the invention

[0005] The purpose of the present invention is to design a multimodal student classroom mental health warning method and system, a mental health monitoring system that combines individual differences among students, dynamically adapts to the classroom environment, and provides real-time intelligent feedback for teachers. This system should not only solve the problems of false alarms and missed alarms in traditional systems under unified standards, but also be able to flexibly adjust the warning rules according to different classroom activities, and provide teachers with effective intervention guidance, so that teachers can pay attention to students' mental health status in a timely manner without interfering with teaching. By overcoming these technical shortcomings, the new system will greatly improve the accuracy, real-time nature and intervention effect of mental health monitoring, and provide better support for the healthy development of students.

[0006] In order to achieve the above object, the present invention provides a multimodal student classroom mental health early warning method in a first aspect, the method comprising the following steps:

[0007] S1. Use multiple sensors to detect the current classroom activity type in real time, including:

[0008] S101, collecting multimodal data of the current classroom through multiple sensors, and extracting and fusing multimodal features to obtain an overall feature vector; the multimodal features include video features, audio features and environmental features;

[0009] S102, inputting the overall feature vector into a classification model to determine the activity type of the current class;

[0010] S2. Generate a dynamically adjusted personalized participation demand model based on the activity type and each student’s personalized data, and generate individualized participation criteria, including:

[0011] Define the historical average participation level of students under different activity types as the participation baseline of each student, and build a personalized participation demand model in combination with the current classroom activity type, wherein the personalized participation demand model represents the expected participation performance of the current student in the current time period;

[0012] S3. Monitor students’ participation behaviors in class in real time and analyze the differences with the personalized participation demand model. Determine whether there are abnormal behaviors based on the differences and mark them if there are any abnormal behaviors, including:

[0013] S301, forming a multidimensional vector with the expected participation performance of each student in the personalized participation demand model, and forming a real-time participation behavior vector with the participation behavior data collected in real time;

[0014] S302, calculating the difference vector between the real-time participation behavior vector and the multidimensional vector, which is expressed as the difference between the student's actual performance and the expected performance, and correcting the difference vector, dynamically adjusting the acceptance of the difference according to the student's historical performance, and obtaining a corrected difference vector;

[0015] S303, analyzing the corrected difference vector to determine whether the student's behavior deviates from the normal range. If the second norm of the difference vector exceeds a threshold, the student's participation behavior at this moment is abnormal behavior;

[0016] S4. Generate stratified mental health warnings based on the difference analysis results, including mild, moderate and severe warnings, including:

[0017] S401, dynamically weighting the corrected difference vector and calculating the total abnormality score of the student in combination with the activity type of the current class;

[0018] S402. Design an adaptive warning threshold based on historical data, students’ participation performance and real-time adjustment of the classroom environment. At the same time, during the design process, design an abnormality score regularization term in the total abnormality score to target the situation where students’ short-term behavioral fluctuations lead to false alarms, so as to ensure that the warning mechanism is sensitive to long-term behavioral abnormalities but has tolerance for short-term fluctuations.

[0019] S403, generating corresponding mental health warnings according to the total abnormality score with the regularization term added, including: no warning, mild warning, moderate warning and severe warning, and feeding back the final warning signal to the teacher for display;

[0020] S5. Build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning;

[0021] S6. Optimize the individualized participation demand model based on teacher feedback and students’ subsequent behavior, so that the model can adapt to students’ long-term performance and form a closed-loop optimization.

[0022] Furthermore, the video feature is to extract the dynamic features of students using the optical flow method; the audio feature is to reflect the voice interaction mode in the classroom by analyzing the voice features of teachers and students; the environmental feature is the background noise level in the classroom environment extracted by noise analysis; the classification model dynamically adjusts the decision boundary of the model through incremental learning, which is expressed as follows:

[0023]

[0024] Among them, C i represents the i-th classroom activity type, i = 1, 2, ..., k; F(t) represents the classroom feature vector at the current time t; β i represents the feature weight vector corresponding to each activity type; P(C i |F(t)) indicates that the current class belongs to type C i The probability of jRepresents the feature weight vector corresponding to the current activity type.

[0025] Furthermore, in S2, the designed participation demand coefficient λ is added to the personalized participation demand model. s (t), dynamically reflects the students’ current participation needs, and the participation demand coefficient λ s (t) is expressed as follows:

[0026] λ s (t) = γ(C final (t))·B s (t)

[0027] Among them, λ s (t) represents the participation requirement standard of student s at time t, γ(C final (t)) represents the adjustment factor based on the type of classroom activities, B s (t) represents the student's historical participation baseline;

[0028] The personalized participation demand model is expressed as follows:

[0029] M s (t) = λ s (t)+δ s

[0030] Among them, M s (t) represents the individualized participation demand model of student s at time t; δ s represents a correction factor to account for individual performance deviations in special cases.

[0031] Furthermore, the correction of the difference vector is based on the correction term δ s (t), which is used to dynamically adjust the acceptance of differences based on students’ historical performance, is expressed as follows:

[0032] D s ′(t)=D s (t)+λ s ·δ s (t)

[0033] Among them, D s (t) represents the difference vector, D s ′(t) represents the difference vector after adding the correction term; λ s represents the adjustment parameter, which is used to control the impact of the correction term and is dynamically adjusted due to different individual characteristics of students; s (t) represents the difference correction term, which includes dynamic adjustment factors in multiple dimensions;

[0034] The anomaly detection formula for determining whether a student's behavior deviates from the normal range is expressed as follows:

[0035] ||D s ′(t)||2>θ s (t)

[0036] Among them, ||D s ′(t)||2 represents the second norm of the difference vector, which indicates the total deviation of the student from the expected value in all dimensions; θ s (t) represents the adaptive anomaly detection threshold, which is flexibly set according to the individual differences of students and the current classroom activities to prevent misjudgment caused by a unified standard. When the second norm of the difference vector exceeds the threshold, the student's participation behavior at this moment is abnormal.

[0037] Furthermore, the total anomaly score is calculated as follows:

[0038]

[0039] Among them, S s (t) represents the total abnormal score of student s at time t, which is used to assess the abnormality of the student’s mental health status; w k (C final (t)) represents the kth dimension in the current classroom activity type C final (t), which changes dynamically with different classroom activities; D s ' k (t) represents the corrected difference in participation of student s in the kth dimension.

[0040] Furthermore, the adaptive warning threshold is calculated as follows:

[0041]

[0042] in, Indicates the basic threshold preset based on historical data, which is used to judge different levels of warnings; s represents an adjustment coefficient, which is personalized according to the student's mental health profile and historical participation behavior; δ s (t) represents the fluctuation of current classroom activities and recent participation behaviors, reflecting the changes in students’ mental health in the short term and used to dynamically adjust the threshold;

[0043] Add the anomaly score regularization term, expressed as follows:

[0044] S s ′(t)=S s (t)+Ω s (t)

[0045] Among them, S s ′(t) is the total anomaly score after regularization; is a regularization term based on the variance of historical scores, which represents the variance of abnormal scores of students in a short period of time. N is the observation window, and β is the regularization coefficient term Ω s (t).

[0046] According to the total anomaly score S after regularization s ′(t), generates the corresponding mental health warning, which is expressed as follows:

[0047]

[0048] in, are the adaptive thresholds of student s under mild, moderate and severe warnings respectively.

[0049] Furthermore, the S5 specifically includes:

[0050] S501, generating a feedback matrix according to the abnormal feedback value of each student in each dimension, wherein the feedback matrix reflects the abnormal situation of the student in each dimension, and adjusting the feedback weight in combination with the type of classroom activity;

[0051] S502: Generate an intervention suggestion matrix based on the feedback matrix, wherein the intervention suggestion matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current warning level;

[0052] S503: The generated feedback matrix and intervention suggestion matrix are presented to the teacher through the smart terminal device in the form of feedback and suggestions, and dynamic adjustments are made according to the teacher's feedback and suggestions.

[0053] Furthermore, the abnormal feedback value of each student in each dimension is expressed as follows:

[0054]

[0055] in, represents the feedback value of the kth dimension, reflecting the severity of the abnormality in this dimension; the feedback matrix is

[0056] Furthermore, the S6 specifically includes:

[0057] S601, integrate teacher feedback, student historical data and current classroom behavior to generate a personalized feedback learning matrix L s (t), used to dynamically adjust the personalized participation demand model M s (t); The personalized feedback learning matrix L s (t) is expressed as follows:

[0058] L s (t) = α·F s(t)+β·H s (t)+γ·I s (t)

[0059] Among them, L s (t) represents the feedback learning matrix of student s at time t, which integrates the current feedback and historical behavior data; α, β, γ represent weight parameters, which respectively represent the influence of current feedback, historical data and intervention suggestions on the learning matrix; F s (t) represents the feedback matrix; H s (t) represents the student's historical behavior matrix, representing long-term data; I s (t) represents the intervention suggestion matrix, which reflects the intervention measures taken by teachers and their effects;

[0060] S602, using adaptive learning strategy and dynamic learning rate to personalize students' participation demand model M through feedback learning matrix s (t) Perform dynamic optimization; the adaptive learning strategy is expressed as follows:

[0061] M s ′(t+1)=M s (t)+η(t)·L s (t)

[0062] Among them, M s ′(t+1) represents the optimized personalized participation demand model, which is used for the next class monitoring; M s (t) represents the personalized participation demand model at the current moment; η(t) represents the dynamic learning rate at time step t, which is used to control the learning matrix L s (t) Impact on model updating;

[0063] The dynamic learning rate is calculated as follows:

[0064]

[0065] Among them, Δ s (t) represents the abnormal change of student s at time t; λ represents the adjustment factor, which is used to control the change amplitude of the learning rate; μ s It represents the mean of student s’ long-term participation behavior, reflecting the average participation level of students;

[0066] S602. Introduce a long-term fluctuation control mechanism to enable the personalized participation demand model to adapt to long-term changes and perform long-term optimization.

[0067] In a second aspect of the present invention, a multimodal student classroom mental health early warning system is provided, the system comprising:

[0068] A multimodal acquisition unit, used to detect the type of activities in the current classroom in real time through multiple sensors;

[0069] A personality analysis unit, which is used to generate a dynamically adjusted personalized participation demand model based on the activity type and the personalized data of each student, and to generate individualized participation criteria;

[0070] The abnormal analysis unit is used to monitor the participation behavior of students in the classroom in real time, and conduct difference analysis with the personalized participation demand model. It determines whether there is abnormal behavior based on the difference sharing, and marks it if there is an abnormality;

[0071] A psychological warning setting unit is used to generate hierarchical psychological health warnings based on the difference analysis results, including mild, moderate and severe warnings;

[0072] The early warning intervention unit is used to build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning;

[0073] The system optimization unit is used to optimize the individualized participation demand model based on the teacher's feedback and the students' subsequent behavior, so that the model can adapt to the students' long-term performance and form a closed-loop optimization.

[0074] The beneficial technical effects of the present invention are at least as follows:

[0075] (1) The present invention establishes an individualized participation demand model for each student, comprehensively considers their personality characteristics, historical participation data and mental health records, and dynamically adjusts the participation monitoring standards for each student. In different parts of the class, the system makes warning adjustments based on the unique behavior patterns of each student. For example, in the discussion session, extroverted students should have a higher degree of participation, while introverted students can remain relatively silent. Through the personalized model, the system can carry out refined management based on the specific needs of each student, avoiding the problems of false alarms and missed alarms under unified standards, and greatly improving the accuracy of the warning.

[0076] (2) The present invention automatically adjusts the monitoring strategy of mental health warning through real-time detection of classroom activities (such as teacher explanations, interactive discussions, silent self-study, etc.). For example, in the explanation mode, the system focuses on monitoring the students' concentration, while in the discussion mode, it focuses on monitoring the students' interaction frequency and the number of times they speak. Each classroom activity corresponds to a different mental health monitoring standard. The system can flexibly adjust the warning rules according to changes in the classroom situation to ensure the accuracy and real-time nature of the warning.

[0077] (3) Through the intelligent feedback mechanism, the student's mental health warning results are fed back to the teacher in real time in an appropriate manner. For example, the system uses a visual interface or appropriate reminders to let teachers know in a timely manner which students have abnormal participation or obvious mood swings. At the same time, the system combines the student's individual participation demand model to provide teachers with specific intervention suggestions. For example, if a student shows long-term low participation, the system will suggest that the teacher intervene by calling on the student or by individual communication after class. This feedback and intervention mechanism greatly enhances the teacher's classroom management ability, helping teachers to pay attention to students' mental health problems in a timely manner without disrupting normal teaching, and take appropriate actions to intervene. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0079] Figure 1 This is a flow chart of a multimodal student classroom mental health early warning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0080] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0081] In one or more embodiments, Figure 1 As shown, a multimodal student classroom mental health early warning method is disclosed, and the method includes steps 1 to 6, including:

[0082] S1. Use multiple sensors to detect the activity type of the current class in real time.

[0083] Specifically, in the classroom dynamic environment detection, firstly, multiple sensors are used to collect classroom audio and video data in real time. Set the video frame sequence V(t) and the audio signal A(t), where t is the time series. For each frame V(t), the present invention extracts multimodal features:

[0084] Video features: Use optical flow method to extract dynamic features such as students’ body movements and posture changes. v (t), captures the overall activity of students in the classroom.

[0085] Audio features: By analyzing the voice features of teachers and students, the audio signal features F are extracted a(t) Includes volume, pitch, speaking speed, etc., which can reflect the voice interaction pattern in the classroom.

[0086] Environmental characteristics: Through noise analysis, the background noise level N(t) in the classroom environment is extracted to judge the quietness of the classroom (for example, the noise level is lower in self-study mode and higher in discussion mode).

[0087] Furthermore, these multimodal data are combined into the overall feature vector F(t) of classroom activities through the formula:

[0088] F(t)=α1F v (t)+α2F a (t)+α3N(t)

[0089] Among them, α1, α2, and α3 are weight parameters that measure the importance of video features, audio features, and noise features, respectively. These parameters are adjusted according to different types of classroom activities (for example, in self-study mode, α3 is higher because quietness is more important; in discussion mode, α2 has the main weight because voice interaction is more frequent).

[0090] Furthermore, the above feature vector F(t) is input into the classification model, and the model will judge the current classroom activity type based on these features. The present invention proposes a decision mechanism based on dynamic feature fusion, which dynamically adjusts the decision boundary of the model through incremental learning. The core formula of classification is as follows:

[0091]

[0092] Among them, C i represents the i-th type of classroom activity (such as explanation, discussion, self-study, etc.), i = 1, 2, ..., k. F(t) is the classroom feature vector at the current time t. β i is the feature weight vector corresponding to each activity type. i |F(t)) means the current class belongs to type C i probability.

[0093] Through continuous iteration, update β i , enabling it to adapt to the characteristic changes of different classroom modes, ensure the accuracy of the classification model in long-term teaching, and capture the dynamic changes of classroom activities.

[0094] Furthermore, in order to avoid misclassification caused by short-term fluctuations or noise, a sliding window smoothing strategy is proposed. The sliding window size is defined as w, the time step is Δt, and the activity classification results in the past w time steps are weighted averaged. The formula is as follows:

[0095]

[0096] Through this smoothing mechanism, the system can effectively remove temporary classification fluctuations, enhance the system's robustness under short-term interference, and ensure accurate identification of long-term trends in the classroom environment.

[0097] Furthermore, the smoothed result C final (t) represents the final classroom activity type at the current time point t. final (t) By selecting the activity type with the highest probability from the classification probabilities within multiple time steps, it represents the most likely activity scenario in the current class (such as explanation, discussion, self-study, etc.). The specific explanation is as follows:

[0098] C final (t) is the final output of the system, which indicates the type of classroom activity recognized by the system at the current moment.

[0099] The output of the system at each time step t is C final (t) It will serve as input for the next step to guide the adjustment of the student individual participation needs model and ensure that the corresponding standards are used in different types of classroom activities.

[0100] Therefore, C final (t) is the final classroom activity type label determined by the system at the current time point t. This activity type will directly affect the dynamic adjustment of the subsequent student individual demand model.

[0101] S2. Based on the activity type and combined with each student’s personalized data, a dynamically adjusted personalized participation demand model is generated, and individualized participation standards are generated.

[0102] In step 1, the output classroom activity type C final (t) (such as explanation, discussion, and self-study) is used as the input of this step. According to different activity types, the system will initialize the corresponding individualized participation demand model for each student s, which reflects the expected participation of the student in the current classroom activity.

[0103] Set a baseline for each student's participationB s (t), which is defined as the final The historical average level of participation of students in the classroom (t). This baseline is constructed from the students' historical performance data and represents the frequency of participation in different classroom settings. For example, an extroverted student s may have a high baseline in the discussion setting, but a low baseline in the self-study setting.

[0104]

[0105] Among them, B s(t) is the participation baseline of student s at time t. n is the number of samples in the historical data. s (C final (t i )) for students in historical time t i Corresponding activity type C final (t i ) of a student. This formula is used to generate a baseline participation model based on a student's historical performance.

[0106] Furthermore, the system is based on the student's participation baseline B s (t), combined with the current classroom activity type C final (t) Dynamically adjust individual participation needs. In order to flexibly reflect students’ current participation needs, a participation demand coefficient λ is set. s (t), used to adjust students’ real-time participation criteria.

[0107] λ s (t) = γ(C final (t))·B s (t)

[0108] Among them, λ s (t) is the participation requirement standard of student s at time t. γ(C final (t)) is an adjustment factor based on the type of classroom activities. Different classroom activities have different requirements for participation. For example, in the discussion mode, the γ value is larger, while in the self-study mode, the γ value is smaller. s (t) is the student’s historical participation baseline. This dynamically adjusted model ensures that the system flexibly adjusts the participation needs of each student according to the specific type of classroom activities.

[0109] Furthermore, according to the dynamically adjusted participation demand coefficient λ in step 2 s (t), generates the individualized participation demand model of students at the current time t. This model reflects the expected behavioral standards of current students in the current classroom scenario.

[0110] M s (t) = λ s (t)+δ s

[0111] Among them, M s (t) represents the individualized participation demand model of student s at time t. s is a correction factor used to account for individual performance deviations in special cases. For example, factors such as students’ mood and recent performance may lead to small deviations in participation requirements, and this value can be adjusted in real time through the machine learning model.

[0112] Finally, the system generates an individualized participation demand model M for each student. s (t) Will be used in subsequent steps (such as real-time behavior monitoring and differential analysis) to ensure that each student’s participation needs criteria are updated in real time based on their individual characteristics, historical participation, and type of classroom activities, and can flexibly respond to changes in needs in different scenarios.

[0113] This model will serve as a standard for subsequent behavioral monitoring to determine whether students’ actual performance meets expectations and provide a basis for mental health early warning.

[0114] S3. Monitor students’ participation behavior in class in real time, and conduct difference analysis with the personalized participation demand model. Determine whether there is abnormal behavior based on the difference sharing, and mark it if there is any abnormality.

[0115] Specifically, in the previous step (step 2), the individualized participation demand model M generated s (t) Initialization has been completed based on each student's historical participation data, classroom activity type and individual characteristics. Now we enter the stage of real-time participation behavior monitoring.

[0116] The model M s (t) represents the expected participation performance of student s in time period t. It includes different dimensions, such as speaking frequency, eye contact, body movements, etc., which reflect the degree of participation that students are expected to show in class. For each student s, the present invention defines these expected participation as a multidimensional vector:

[0117]

[0118] in, It represents the expected value of the kth participation dimension (such as speaking frequency, eye contact, etc.) of student s at time t, and d is the number of participation dimensions.

[0119] Next, the system collects students’ real-time participation behavior data through sensors and generates students’ real-time participation behavior vector A. s (t). This vector contains the actual behavior data of each student on different participation dimensions. In form:

[0120]

[0121] in, represents the actual performance value of student s on the kth participation dimension at time t. These data come from a variety of sensors, such as cameras to capture body movements, microphones to record the number and duration of speeches, and eye tracking systems to analyze students' concentration.

[0122] Further, receiving real-time participation behavior data As (t) After that, the system needs to compare it with the individual participation demand model M s (t) is compared to calculate the difference between the student’s actual performance and the expected performance. The difference vector D s (t) is calculated as follows:

[0123]

[0124] Among them, D s (t) is the participation difference vector of student s at time t, and each component represents the deviation between the student’s actual performance and the expected value on a certain participation dimension. Through this difference vector, the system can determine the degree of deviation between the student’s actual performance on each participation dimension and the model expectation.

[0125] Furthermore, in order to cope with the fluctuations in individual student behavior and unpredictable factors in classroom activities, the system introduces difference control terms and regularization terms to enhance the robustness of the model. The present invention defines a new correction term δ s (t), whose function is to dynamically adjust the acceptance of differences based on students’ historical performance.

[0126] D s ′(t)=D s (t)+λ s ·δ s (t)

[0127] Among them, D s ′(t) is the difference vector after adding the correction term. s It is an adjustment parameter used to control the impact of the correction term and is dynamically adjusted according to the individual characteristics of students. s (t) is a difference correction term, which includes dynamic adjustment factors in multiple dimensions, taking into account factors such as students' personality, mood swings, and long-term performance deviations. This correction term is generated through students' long-term performance curves and personality characteristics to avoid excessive impact of short-term fluctuations on the results. For example, if a student often has mood swings during certain periods of time, the system will increase tolerance for these differences based on their historical fluctuations to reduce false positives.

[0128] Furthermore, once the present invention obtains the corrected difference vector D s ′(t), the system will analyze the vector to determine whether the student’s participation behavior deviates from the normal range. At this stage, the present invention introduces an adaptive participation abnormality threshold θ s (t), the threshold is dynamically adjusted based on the student’s historical performance, mental health profile, and current classroom activities. The anomaly detection formula is as follows:

[0129] Abnormal conditions: ||D s′(t)||2>θ s (t)

[0130] Among them, ||D s ′(t)||2 is the bi-norm of the difference vector, which represents the total deviation of the student from the expected value in all dimensions. s (t) is the adaptive anomaly detection threshold, which is flexibly set according to the individual differences of students and the current classroom activities to prevent misjudgment caused by unified standards. When the second norm of the difference vector exceeds the threshold, the system will judge the student's participation behavior at this moment as abnormal behavior.

[0131] Furthermore, based on the results of difference calculation and anomaly detection, the system will output the difference analysis conclusions for each student. Specifically, the system will mark which participation dimensions have anomalies and form a report of abnormal behavior. These reports will serve as input for the next step (generation of mental health warning), helping the system to better judge the mental state of students and conduct warning processing.

[0132] In short, the system generates a participation behavior vector A by monitoring students' participation behavior in real time. s (t). Combine real-time behavioral data with individualized participation demand model M s (t) Perform difference calculation to obtain the difference vector D s (t). Introducing the innovative correction term δ s (t) and adaptive threshold θ s (t), enhance the system's ability to adapt to individual differences and behavioral fluctuations, and ensure accurate detection of abnormal behaviors. Finally, the difference analysis results are output, and abnormal participation behaviors are marked as the basis for subsequent mental health warning generation.

[0133] S4. Generate stratified mental health warnings based on the results of differential analysis, including mild, moderate and severe warnings.

[0134] In step 3, the output difference vector D s ′(t) represents the difference between the participation behavior of student s at time t and the correction of the expected model. This difference vector combines the individual student's participation performance, emotional fluctuations, and individualized participation needs, and adjusts the accuracy of anomaly detection through correction terms and adaptive thresholds.

[0135] In step 4, the goal of the present invention is to generate a mental health warning for students based on these difference data, ensuring that the system can identify students' potential psychological problems and inform teachers in a timely manner. In order to enhance the system's innovation and adaptability to complex classroom scenarios, this step will be processed through the following stages to ultimately generate a multi-level mental health warning.

[0136] Furthermore, in a complex classroom environment, students’ behavior and mental state may be affected by multi-dimensional participation characteristics. The impact of abnormalities in each participation dimension (such as speech, body language, eye contact, etc.) on students’ mental state is not the same. Therefore, before the system generates an early warning, it is necessary to weight the differences in different dimensions. Let D s ′(t)=[D s ' 1 (t), D s ' 2 (t), ..., D s ' d (t)] is the modified difference vector of student s at time t. The system weights these difference values ​​and combines them with the activity type C of the current class final (t), and obtain the student’s total abnormality score S s (t).

[0137]

[0138] Among them, S s (t) is the total abnormal score of student s at time t, which is used to assess the abnormality of the student’s mental health status. k (C final (t)) is the kth dimension in the current classroom activity type C final (t). This weight will change dynamically depending on the classroom activities. For example, in the lecture mode, the weight of body movements and eye contact may be larger, while in the discussion mode, the weight of speaking frequency is higher. s ' k (t) is the corrected difference in participation of student s in the kth dimension.

[0139] Furthermore, in order to enhance the accuracy of multi-dimensional abnormality scoring in complex situations, the present invention introduces a dynamic weighting mechanism, which uses the current classroom activity type as a weight adjustment factor. Through this mechanism, the system can flexibly adjust the weight of each dimension according to specific classroom activities (such as discussion, explanation, and self-study) to more accurately reflect the current mental health status of students. For example, in a discussion mode, w 发言 The weight of will be larger, and the system focuses on monitoring whether students participate in the discussion; in the self-study mode, w 目光接触 The weighting is higher to ensure that students' attention is not distracted for long periods of time.

[0140] Furthermore, different students have very different performances in class, so a unified warning standard is not suitable for all students. To solve this problem, the system generates adaptive warning thresholds for each student, which are not only set based on historical data, but also adjusted in real time with the student's participation performance and classroom environment.

[0141] set up are the adaptive thresholds of student s under mild, moderate and severe warning respectively. The calculation formula is:

[0142]

[0143] in, It is a basic threshold preset based on historical data and is used to determine different levels of warnings. s is an adjustment factor that is personalized based on the student's mental health profile and historical participation behavior. s (t) is the fluctuation term of current classroom activities and recent participation behaviors, which reflects the changes in students’ mental health in the short term and is used to dynamically adjust the threshold.

[0144] This warning threshold ensures that the warning generation for each student is personalized, avoiding false positives and false negatives caused by unified standards.

[0145] Furthermore, in order to avoid too many false positives caused by short-term behavioral fluctuations of students (such as occasional distraction), the system introduces an abnormal score regularization term Ω s (t). This regularization term can control the growth rate of the anomaly score, ensuring that the early warning mechanism is sensitive to long-term behavioral anomalies but has a certain tolerance for short-term fluctuations.

[0146] S s ′(t)=S s (t)+Ω s (t)

[0147] Among them, S s ′(t) is the total anomaly score after regularization. is a regularization term based on the variance of historical scores, which represents the variance of abnormal scores of students in a short period of time. N is the observation window and β is the regularization coefficient.

[0148] When students' behavior deviates from expectations for a long time, Ω s (t) will increase, increasing the cumulative effect of abnormal scores; if students have short-term fluctuations but recover quickly, Ω s (t) will be reduced, thus avoiding false positives.

[0149] Furthermore, according to the regularized total anomaly score S s ′(t), the system will generate the corresponding mental health warning. The warning is divided into four levels: no warning, mild warning, moderate warning, and severe warning. The specific calculation is as follows:

[0150]

[0151] No warning: S s′(t) is lower than the mild warning threshold, indicating that the student’s participation behavior is normal or fluctuates within an acceptable range.

[0152] Mild warning: The student’s total abnormality score is slightly high, and there may be minor mental health problems. It is recommended that teachers pay mild attention or observe their subsequent performance.

[0153] Moderate warning: If the student's total abnormality score is significantly higher than the mild threshold, the system will remind the teacher to pay attention to the student's mental health and recommend appropriate intervention in class (such as asking questions, encouraging interaction, etc.).

[0154] Severe warning: If a student's abnormal scores exceed the severe warning threshold, the system will strongly recommend that the teacher have a private conversation with the student after class or seek help from a professional psychological counselor.

[0155] Furthermore, finally, the system will display the generated warning information through the teacher terminal. The warning results include the following aspects:

[0156] Warning level: Displays the current warning status of the student (mild, moderate, severe).

[0157] Abnormal dimension analysis: List in detail which participation dimensions have abnormalities, such as speech, body language, attention, etc., and give specific difference data.

[0158] Recommended intervention measures: The system will provide teachers with personalized intervention suggestions based on the warning level and abnormal dimension. For example, a mild warning may suggest that teachers encourage students to interact by asking questions; a moderate warning may suggest that teachers communicate with students after class.

[0159] Through this real-time feedback, teachers can quickly understand the mental health status of students and take appropriate actions in time to prevent problems from worsening.

[0160] S5. Build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning.

[0161] In step 4, the system generates a score based on the student's abnormal score S s ′(t) multi-level mental health warning, including mild, moderate, and severe warnings. Each warning result contains the abnormal level (mild, moderate, severe) and the difference analysis of specific participation dimensions (for example, speech, eye contact, body language, etc.). This information is used as the input of this step to build an intelligent feedback model. The core input variables are:

[0162] Warning level: The level of mental health status determined by the system.

[0163] Dimensional anomaly analysis: reflects the degree of abnormality of each participation dimension and represents the deviation of students in different dimensions.

[0164] Based on these inputs, the system needs to generate effective and actionable feedback and provide teachers with targeted intervention suggestions.

[0165] Furthermore, in order to provide personalized and accurate feedback, the system generates a feedback matrix F based on the abnormal dimension analysis results of each student. s (t). This matrix reflects the abnormal situation of students in each dimension and adjusts the feedback weight in combination with the type of classroom activities. The formula of the feedback matrix is:

[0166]

[0167] in, in Represents the feedback value of the kth dimension, reflecting the severity of the abnormality in this dimension. k (C final (t)) is the weight of the kth dimension, indicating the current classroom activity type C final (t), the importance of the dimension. For example, in the discussion mode, the weight of speech w 发言 will be larger; in the self-study mode, the weight of eye contact w 目光接触 The feedback matrix F s (t) is the core data of the system feedback, which shows the teacher the specific abnormal performance of the student. The feedback value of each dimension is dynamically adjusted according to the abnormal degree of the participating behavior.

[0168] Furthermore, the system not only generates a feedback matrix, but also associates it with the type of classroom activities, so that the feedback priority in different scenarios can be flexibly adjusted. For example, in discussion mode, speaking frequency is the key dimension to be monitored, while in explanation mode, students' attention (such as eye contact) is more important. Through this scenario-based weight adjustment, the system can provide targeted feedback in different teaching activities.

[0169] Furthermore, in order to further help teachers take appropriate intervention measures, the system generates an intervention suggestion matrix I based on the feedback matrix s (t). The matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current warning level (mild, moderate, severe). The formula for the intervention recommendation matrix is:

[0170]

[0171] in, It is an intervention suggestion for the kth dimension, combining the feedback value and the warning level. The generation formula is:

[0172]

[0173] in, is an intervention generating function based on the feedback value Current warning level and student mental health history profile s Generate targeted intervention recommendations. It is the intervention suggestion generated for the kth dimension, which may include questioning interaction, classroom nomination speech, after-class communication or seeking psychological counseling.

[0174] Furthermore, the intensity of intervention suggestions varies at different warning levels. In a mild warning, the system may suggest that teachers motivate students to participate by asking simple questions; in a moderate or severe warning, the system may suggest that teachers communicate with students one-on-one or contact a psychological counselor.

[0175] Understandably, intervention recommendations rely not only on the current feedback matrix, but also on the student's historical mental health profile. s For example, if a student has shown similar behavior patterns in past courses, the system could suggest that teachers pay closer attention to that student, or recommend long-term coaching measures.

[0176] Furthermore, the generated feedback matrix F s (t) and intervention suggestion matrix I s (t) will be displayed to teachers through smart terminal devices so that teachers can intuitively understand the students' current mental health status and the intervention measures recommended by the system. To ensure that teachers can quickly understand the information, the system provides a variety of visualization tools:

[0177] Feedback Dashboard: Displays each student’s feedback matrix F s (t) Highlight abnormal dimensions through visual charts (such as heat maps and radar charts). For example, when students show abnormalities in certain dimensions (such as speaking frequency or concentration), the system will use color coding to mark these dimensions to help teachers quickly locate the problem.

[0178] Feedback value for each dimension The data will be displayed in a graphical way, so teachers can see at a glance which dimensions need to be focused on. If the value is significantly high, the system will highlight the dimension and prompt the teacher to give priority to student participation.

[0179] Intervention suggestion panel: The system also displays the generated intervention suggestion matrix I s(t), providing specific operational suggestions for each abnormal dimension. Suggestions under different warning levels will have corresponding operational guidelines to ensure that teachers can quickly apply them in actual teaching. For example, under a mild warning, the system may suggest that teachers motivate students to participate by asking questions; under a severe warning, the system will suggest that teachers communicate individually after class or contact a psychological counselor.

[0180] Furthermore, in addition to the intelligent feedback generated by the system, the system also allows teachers to dynamically adjust feedback and suggestions. This mechanism enables teachers to optimize suggestions based on actual classroom conditions and feed them back to the system, forming a closed loop of teacher-system interaction. Teachers can adjust the following:

[0181] Intervention intensity adjustment: If the teacher thinks the system's intervention suggestions are too radical or insufficient, they can adjust them according to the actual situation in the classroom. For example, the teacher can adjust moderate intervention to mild intervention, or upgrade mild intervention to moderate intervention. Intervention intensity adjustment formula:

[0182]

[0183] Where ψ is the teacher feedback adjustment function, which dynamically adjusts the intervention measures according to the teacher's feedback. Through this mechanism, the system not only provides standardized intervention suggestions, but also learns and improves through teacher feedback.

[0184] Confirmation and correction of feedback content: If the teacher believes that the system's abnormal warning for students is not in line with reality, the system can mark specific dimensions and submit feedback. The system will adjust the subsequent monitoring strategy for the student based on the teacher's feedback and update its individualized participation demand model M. s (t).

[0185] Furthermore, the teacher's feedback will be recorded by the system and used as an important basis for subsequent system optimization. Each adjustment and correction made by the teacher to the system feedback will affect the student's individualized participation demand model M. s (t), so that the system can more accurately judge the psychological state of students in future monitoring and avoid false positives or negative positives.

[0186] Through this adaptive learning mechanism, the system can not only improve the ability to monitor mental health in the current classroom, but also gradually optimize the personalized needs model of each student in the long run, thereby improving the accuracy and effectiveness of overall mental health management.

[0187] S6. Optimize the individualized participation demand model based on teacher feedback and students’ subsequent behavior, so that the model can adapt to students’ long-term performance and form a closed-loop optimization.

[0188] In step 5, the system collects the teacher’s intervention feedback and the student’s real-time behavior performance through the intelligent feedback mechanism. These data provide key inputs for the dynamic optimization of the individualized demand model. At this point, the inputs include:

[0189] Teacher feedback data: Post-intervention results of teachers on students’ engagement behaviors based on the feedback matrix and intervention recommendations.

[0190] Student History Data s (t): Records of students’ past participation behaviors, including class participation, early warning information, and intervention effects.

[0191] These inputs will serve as the basis for optimizing the individualized demand model. The dynamic optimization of the model can not only better reflect the real-time performance of students, but also make adaptive adjustments based on students' long-term participation behaviors to improve the accuracy and personalization of long-term monitoring.

[0192] Furthermore, in order to fully integrate teacher feedback, students’ historical data, and current classroom behavior, the system generates a personalized feedback learning matrix L s (t), this matrix will be used to dynamically adjust the individual demand model M s (t).

[0193] The generation formula of the feedback learning matrix is ​​as follows:

[0194] L s (t) = α·F s (t)+β·H s (t)+γ·I s (t)

[0195] Among them, L s (t) is the feedback learning matrix of student s at time t, integrating the current feedback and historical behavior data. α, β, and Y are weight parameters, which respectively represent the impact of current feedback, historical data, and intervention suggestions on the learning matrix. These weights can be adjusted dynamically, depending on the performance of different students. For example, for students with long-term behavioral fluctuations, the β value is larger to enhance the weight of historical data, while for students who are sensitive to feedback, α will be increased accordingly. F s (t) is the feedback matrix, from step 5, representing the teacher’s feedback results for each participation dimension. s (t) is the student’s historical behavior matrix, representing long-term data, such as participation fluctuations, mental health changes over the past few months, etc. s (t) is the intervention suggestion matrix, which reflects the intervention measures taken by teachers and their effects. s(t) It can effectively capture anomalies in students’ participation in the current class, while referring to historical performance to ensure that the model update takes into account both short-term fluctuations and long-term trends.

[0196] Furthermore, the system learns the matrix L through feedback s (t) Model M for individual needs of students s (t) is dynamically optimized. The optimization process adopts an adaptive learning strategy, which enables the model to be flexibly adjusted to adapt to students' short-term behavioral changes and long-term participation trends. The optimization formula for defining the individual demand model of student s at time t+1 is:

[0197] M s ′(t+1)=M s (t)+η(t)·L s (t)

[0198] Among them, M s ′(t+1) is the optimized individualized demand model, which is used for the next classroom monitoring. s (t) is the individualized demand model at the current moment. η(t) is the adaptive learning rate at time step t, which is used to control the learning matrix L s (t) Impact on model updating.

[0199] Furthermore, the system introduces a dynamic learning rate η(t) to adaptively adjust the step size of model optimization. In order to make the model more sensitive to short-term fluctuations of students while maintaining adaptability to long-term trends, the system calculates the dynamic learning rate using the following formula:

[0200]

[0201] Among them, Δ s (t) is the amount of abnormal change in student s at time t (e.g., rapid change in participation). λ is a regulation factor used to control the magnitude of the change in learning rate. μ s It is the long-term mean of student s’ participation behavior, reflecting the average level of student participation.

[0202] The design of the dynamic learning rate ensures that when students’ abnormal behaviors increase dramatically (e.g., participation drops sharply), η(t) increases, allowing the model to quickly adapt to the changes; and when the changes are small, η(t) is small, maintaining a smooth update of the model.

[0203] Furthermore, in order to ensure that the optimization of the individualized demand model not only relies on short-term feedback but also adapts to long-term changes, the system introduces a long-term fluctuation control mechanism to limit the overfitting and short-term fluctuation interference of the model through regularization terms.

[0204] The present invention introduces a regularization term Ωs (t) is used to balance the update amplitude of the model and control the excessive impact of short-term fluctuations on the model:

[0205] M s ′(t+1)=M s (t)+η(t)·L s (t)-λ r ·Ω s (t)

[0206] Among them, Ω s (t) is a regularization term based on the long-term student participation fluctuations, which controls the excessive amplitude of model updates. r is the regularization strength coefficient, which determines the influence of the regularization term. If the student's participation performance is stable in the long term, λ r Smaller; if the student's long-term fluctuation is large, λ r Larger to suppress overfitting short-term behavior. Regularization term Ω s The calculation of (t) takes into account the fluctuations in student participation over multiple time periods in the past to ensure that the model can balance short-term behavior with long-term trends. The specific calculation is as follows:

[0207]

[0208] Here, N is the historical window size, which means that the system refers to the participation data in the past N classes. It is the average of the student individual demand model in this window period, reflecting the long-term participation performance. This design can effectively suppress excessive model adjustments caused by short-term fluctuations and ensure that the model can absorb stable data in the long-term trend when it is updated.

[0209] Furthermore, after completing the optimization of the model, the system generates a new individualized demand model M s ′(t+1) is used for classroom monitoring in the next cycle and enters a closed feedback loop. The long-term goal of the system is to enable the model to more and more accurately reflect the students' participation needs through each optimization, thereby improving the accuracy of monitoring and the effectiveness of intervention.

[0210] The final output is the optimized model M s ′(t+1), and the system will continue to monitor the difference between the model and the actual behavior in subsequent classes to form a complete closed-loop optimization mechanism.

[0211] In one or more embodiments, the present invention discloses a multimodal student classroom mental health early warning system, the system comprising:

[0212] A multimodal acquisition unit, used to detect the type of activities in the current classroom in real time through multiple sensors;

[0213] A personality analysis unit, which is used to generate a dynamically adjusted personalized participation demand model based on the activity type and the personalized data of each student, and to generate individualized participation criteria;

[0214] The abnormal analysis unit is used to monitor the participation behavior of students in the classroom in real time, and conduct difference analysis with the personalized participation demand model. It determines whether there is abnormal behavior based on the difference sharing, and marks it if there is an abnormality;

[0215] A psychological warning setting unit is used to generate hierarchical psychological health warnings based on the difference analysis results, including mild, moderate and severe warnings;

[0216] The early warning intervention unit is used to build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning;

[0217] The system optimization unit is used to optimize the individualized participation demand model based on the teacher's feedback and the students' subsequent behavior, so that the model can adapt to the students' long-term performance and form a closed-loop optimization.

[0218] Some preferred embodiments of the present invention are shown, but they cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A multimodal student classroom mental health early warning method, characterized in that: The method comprises the following steps: S1. Use multiple sensors to detect the current classroom activity type in real time, including: S101, collecting multimodal data of the current classroom through multiple sensors, and extracting and fusing multimodal features to obtain an overall feature vector; the multimodal features include video features, audio features and environmental features; S102, inputting the overall feature vector into a classification model to determine the activity type of the current class; S2. Generate a dynamically adjusted personalized participation demand model based on the activity type and each student’s personalized data, and generate individualized participation criteria, including: Define the historical average participation level of students under different activity types as the participation baseline of each student, and build a personalized participation demand model in combination with the current classroom activity type, wherein the personalized participation demand model represents the expected participation performance of the current student in the current time period; S3. Monitor students’ participation behaviors in class in real time and analyze the differences with the personalized participation demand model. Determine whether there are abnormal behaviors based on the differences and mark them if there are any abnormal behaviors, including: S301, forming a multidimensional vector with the expected participation performance of each student in the personalized participation demand model, and forming a real-time participation behavior vector with the participation behavior data collected in real time; S302, calculating the difference vector between the real-time participation behavior vector and the multidimensional vector, which is expressed as the difference between the student's actual performance and the expected performance, and correcting the difference vector, dynamically adjusting the acceptance of the difference according to the student's historical performance, and obtaining a corrected difference vector; S303, analyzing the corrected difference vector to determine whether the student's behavior deviates from the normal range. If the second norm of the difference vector exceeds a threshold, the student's participation behavior at this moment is abnormal behavior; S4. Generate stratified mental health warnings based on the difference analysis results, including mild, moderate and severe warnings, including: S401, dynamically weighting the corrected difference vector and calculating the total abnormality score of the student in combination with the activity type of the current class; S402. Design an adaptive warning threshold based on historical data, students’ participation performance and real-time adjustment of the classroom environment. At the same time, during the design process, design an abnormality score regularization term in the total abnormality score to target the situation where students’ short-term behavioral fluctuations lead to false alarms, so as to ensure that the warning mechanism is sensitive to long-term behavioral abnormalities but has tolerance for short-term fluctuations. S403, generating corresponding mental health warnings according to the total abnormality score with the regularization term added, including: no warning, mild warning, moderate warning and severe warning, and feeding back the final warning signal to the teacher for display; S5. Build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning; S6. Optimize the individualized participation demand model based on teacher feedback and students’ subsequent behavior, so that the model can adapt to students’ long-term performance and form a closed-loop optimization.

2. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The video feature is the dynamic feature of students extracted by optical flow method; the audio feature is the voice interaction mode in the classroom by analyzing the voice features of teachers and students; the environmental feature is the background noise level in the classroom environment extracted by noise analysis; the classification model dynamically adjusts the decision boundary of the model through incremental learning, which is expressed as follows: Among them, C i represents the i-th classroom activity type, i=1,2,...,k; F(t) represents the classroom feature vector at the current time t; β i represents the feature weight vector corresponding to each activity type; P(C i |F(t)) indicates that the current class belongs to type C i The probability of j Represents the feature weight vector corresponding to the current activity type.

3. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: In S2, the designed participation demand coefficient λ is added to the personalized participation demand model s (t), dynamically reflects the students’ current participation needs, and the participation demand coefficient λ s (t) is expressed as follows: λ s (t)=γ(C final (t))·B s (t) Among them, λ s (t) represents the participation requirement standard of student s at time t, γ(C final (t)) represents the adjustment factor based on the type of classroom activities, B s (t) represents the student's historical participation baseline; The personalized participation demand model is expressed as follows: M s (t)=λ s (t)+δ s Among them, M s (t) represents the individualized participation demand model of student s at time t; δ s Represents a correction factor to account for individual performance deviations in special cases.

4. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The correction of the difference vector is based on the correction term δ s (t), which is used to dynamically adjust the acceptance of differences based on students’ historical performance, is expressed as follows: D s ′(t)=D s (t)+λ s ·δ s (t) Among them, D s (t) represents the difference vector, D s ′(t) represents the difference vector after adding the correction term; λ s represents the adjustment parameter, which is used to control the impact of the correction term and is dynamically adjusted due to different individual characteristics of students; s (t) represents the difference correction term, which includes dynamic adjustment factors in multiple dimensions; The anomaly detection formula for determining whether a student's behavior deviates from the normal range is expressed as follows: ||D s ′(t)||2>θ s (t) Among them, ||D s ′(t)||2 represents the second norm of the difference vector, which indicates the total deviation of the student from the expected value in all dimensions; θ s (t) represents the adaptive anomaly detection threshold, which is flexibly set according to the individual differences of students and the current classroom activities to prevent misjudgment caused by a unified standard. When the second norm of the difference vector exceeds the threshold, the student's participation behavior at this moment is abnormal.

5. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The total abnormality score is calculated as follows: Among them, S s (t) represents the total abnormal score of student s at time t, which is used to assess the abnormality of the student’s mental health status; w k (C final (t)) represents the kth dimension in the current classroom activity type C final (t), which changes dynamically with different classroom activities; D s ' k (t) represents the corrected difference in participation of student s in the kth dimension.

6. A multimodal student classroom mental health early warning method according to claim 5, characterized in that: The adaptive warning threshold is calculated as follows: in, Indicates the basic threshold preset based on historical data, which is used to judge different levels of warnings; s represents an adjustment coefficient, which is personalized according to the student's mental health profile and historical participation behavior; δ s (t) represents the fluctuation of current classroom activities and recent participation behaviors, reflecting the changes in students’ mental health in the short term and used to dynamically adjust the threshold; Add the anomaly score regularization term, expressed as follows: S s ′(t)=S s (t)+Ω s (t) Among them, S s ′(t) is the total anomaly score after regularization; is a regularization term based on the variance of historical scores, which represents the variance of abnormal scores of students in a short period of time. N is the observation window, and β is the regularization coefficient term Ω s (t). According to the total anomaly score S after regularization s ′(t), generates the corresponding mental health warning, which is expressed as follows: in, are the adaptive thresholds of student s under mild, moderate and severe warnings respectively.

7. The multimodal student classroom mental health early warning method according to claim 1 is characterized in that: The S5 specifically includes: S501, generating a feedback matrix according to the abnormal feedback value of each student in each dimension, wherein the feedback matrix reflects the abnormal situation of the student in each dimension, and adjusting the feedback weight in combination with the type of classroom activity; S502: Generate an intervention suggestion matrix based on the feedback matrix, wherein the intervention suggestion matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current warning level; S503: The generated feedback matrix and intervention suggestion matrix are presented to the teacher through the smart terminal device in the form of feedback and suggestions, and dynamic adjustments are made according to the teacher's feedback and suggestions.

8. A multimodal student classroom mental health early warning method according to claim 7, characterized in that: The abnormal feedback value of each student in each dimension is expressed as follows: in, represents the feedback value of the kth dimension, reflecting the severity of the abnormality in this dimension; the feedback matrix is 9. The multimodal student classroom mental health early warning method according to claim 1 is characterized in that: The S6 specifically includes: S601, integrate teacher feedback, student historical data and current classroom behavior to generate a personalized feedback learning matrix L s (t), used to dynamically adjust the personalized participation demand model M s (t); The personalized feedback learning matrix L s (t) is expressed as follows: L s (t)=α·F s (t)+β·H s (t)+γ·I s (t) Among them, L s (t) represents the feedback learning matrix of student s at time t, which integrates the current feedback and historical behavior data; α, β, γ represent weight parameters, which respectively represent the influence of current feedback, historical data and intervention suggestions on the learning matrix; F s (t) represents the feedback matrix; H s (t) represents the student's historical behavior matrix, representing long-term data; I s (t) represents the intervention suggestion matrix, which reflects the intervention measures taken by teachers and their effects; S602, using adaptive learning strategy and dynamic learning rate to personalize students' participation demand model M through feedback learning matrix s (t) Perform dynamic optimization; the adaptive learning strategy is expressed as follows: M s ′(t+1)=M s (t)+η(t)·L s (t) Among them, M s ′(t+1) represents the optimized personalized participation demand model, which is used for the next class monitoring; M s (t) represents the personalized participation demand model at the current moment; η(t) represents the dynamic learning rate at time step t, which is used to control the learning matrix L s (t) Impact on model updating; The dynamic learning rate is calculated as follows: Among them, Δ s (t) represents the abnormal change of student s at time t; λ represents the adjustment factor, which is used to control the change amplitude of the learning rate; μ s It represents the mean of student s’ long-term participation behavior, reflecting the average participation level of students; S602. Introduce a long-term fluctuation control mechanism to enable the personalized participation demand model to adapt to long-term changes and perform long-term optimization.

10. A multimodal student classroom mental health early warning system, the system comprising: A multimodal acquisition unit, used to detect the type of activities in the current classroom in real time through multiple sensors; A personality analysis unit, which is used to generate a dynamically adjusted personalized participation demand model based on the activity type and the personalized data of each student, and to generate individualized participation criteria; The abnormal analysis unit is used to monitor the participation behavior of students in the classroom in real time, and conduct difference analysis with the personalized participation demand model. It determines whether there is abnormal behavior based on the difference sharing, and marks it if there is an abnormality; A psychological warning setting unit is used to generate hierarchical psychological health warnings based on the difference analysis results, including mild, moderate and severe warnings; The early warning intervention unit is used to build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning; The system optimization unit is used to optimize the individualized participation demand model based on the teacher's feedback and the students' subsequent behavior, so that the model can adapt to the students' long-term performance and form a closed-loop optimization.

Images