A Multimodal-Based Method and System for Early Warning of Students' Mental Health in the Classroom

Abstract

The present invention proposes a multi-modal based method and system for mental health early warning in the student classroom. The method includes: detecting the activity type of the current classroom in real time through multiple sensors; generating individualized participation criteria; monitoring the participation behavior of students in the classroom in real time, and performing differential analysis with the personalized participation demand model to determine whether there is abnormal behavior, and if so, marking it; generating a hierarchical mental health early warning according to the differential analysis results; constructing an intelligent feedback model according to the type of mental health early warning and the activity type of the classroom, and feeding it back to the teacher for intervention according to the type of mental health early warning; optimizing the individualized participation demand model according to the teacher's feedback and the subsequent behavior of the students, so that the model adapts to the long-term performance of the students and forms a closed-loop optimization. The present invention not only improves the accuracy of mental health early warning, but also improves the classroom management efficiency through an effective intervention mechanism, and has a wide range of application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of mental health early warning, and particularly relates to a method and system for mental health early warning of students in the classroom based on multi-modalities. Background Art

[0002] With the development of society and the progress of educational concepts, the importance of students' mental health in the field of education has become increasingly prominent. Mental health not only affects students' learning efficiency but also has a profound impact on their long-term development. Therefore, how to monitor and early warn students' mental health problems in real time during the teaching process has become an important topic in educational research. At present, mental health monitoring systems have begun to be widely applied in the classroom environment. Common systems mainly rely on the monitoring of emotions, behaviors, and physiological signals. For example, by means of facial expression recognition, speech emotion analysis, body language monitoring, etc., to evaluate the emotional state of students, or by wearable devices to collect students' physiological data, such as heart rate, skin conductance, etc., to infer their mental stress and anxiety levels. These systems can, to a certain extent, identify abnormal emotional fluctuations and stress signals of students and help teachers and psychological counselors to conduct early intervention on students' mental health.

[0003] However, the existing mental health monitoring systems have some obvious deficiencies in practical applications. First of all, most systems adopt static and unified monitoring standards and lack effective consideration of individual differences among different students. For example, introverted students rarely speak in class and may be misjudged as having abnormal mental states by the system, while extroverted students may be ignored by the system when showing silence under certain circumstances. Secondly, the existing systems have poor adaptability to dynamic environmental changes in the classroom. Classroom activities are usually diverse, covering different links such as lectures, discussions, group assignments, and self-study. Each link has a different impact on students' mental health. A single monitoring strategy cannot accurately capture the mental changes of students in different activities. Traditional systems are prone to false alarms or missed alarms in these dynamic scenarios and cannot adjust the early warning standards in real time according to the changes in the classroom situation.

[0004] In addition, most of the existing systems lack an intelligent feedback mechanism for teachers. Teachers are difficult to obtain the mental health status of students in a timely manner in the classroom and do not receive timely intervention suggestions. Especially in large-class teaching, teachers often cannot notice the emotional and behavioral changes of all students, resulting in potential mental health problems being ignored. These technical defects not only limit the application effect of the system but also increase the burden on teachers in actual operation and reduce the accuracy and effectiveness of early warning. Summary of the Invention

[0005] The object of the present invention is to design a multi-modal based method and system for early warning of students' mental health in the classroom, a mental health monitoring system that combines students' individual differences, dynamically adapts to the classroom environment, and provides real-time intelligent feedback to teachers. This system not only needs to solve the problems of false alarms and missed alarms in traditional systems under unified standards, but also be able to flexibly adjust the early warning rules according to different classroom activities, and provide effective intervention guidance for teachers, so that teachers can timely pay attention to the mental health status of students without disturbing teaching. By overcoming these technical deficiencies, the new system will greatly improve the accuracy, real-time performance and intervention effect of mental health monitoring, and provide better support for the healthy development of students.

[0006] To achieve the above object, in the first aspect of the present invention, there is provided a multi-modal based method for early warning of students' mental health in the classroom, the method comprising the following steps:

[0007] S1. Real-time detect the activity type of the current classroom through multiple sensors, including:

[0008] S101. Collect multi-modal data of the current classroom through multiple sensors, perform multi-modal feature extraction and fusion to obtain an overall feature vector; the multi-modal features include video features, audio features and environmental features;

[0009] S102. Input the overall feature vector into a classification model to judge the activity type of the current classroom;

[0010] S2. According to the activity type, combine the personalized data of each student to generate a dynamically adjusted personalized participation demand model, and generate individualized participation criteria, including:

[0011] Define the historical average participation level of students under different activity types as the participation baseline of each student, and combine the current classroom activity type to construct a personalized participation demand model, the personalized participation demand model represents the expected participation performance of the current student in the current time period;

[0012] S3. Real-time monitor the participation behavior of students in the classroom, perform difference analysis with the personalized participation demand model, and judge whether there is abnormal behavior according to the difference sharing. If there is an abnormality, mark it, including:

[0013] S301. Form a multi-dimensional vector with the expected participation performance of each current student in the personalized participation demand model, and form a real-time participation behavior vector with the real-time collected participation behavior data;

[0014] S302. Calculate the difference vector between the real-time participation behavior vector and the multi-dimensional vector, which represents the difference between the actual performance and the expected performance of the student, and correct the difference vector, dynamically adjust the acceptance degree of the difference according to the historical performance of the student, and obtain the corrected difference vector;

[0015] S303. Analyze the corrected difference vector to determine whether the student's behavior deviates from the normal range. If the two-norm of the difference vector exceeds the threshold, the student's participation behavior at this moment is abnormal behavior;

[0016] S4. Generate a hierarchical mental health warning based on the result of the difference analysis, including mild, moderate, and severe warnings, including:

[0017] S401. Dynamically weight the corrected difference vector and calculate the total abnormal score of the student in combination with the current classroom activity type;

[0018] S402. Design an adaptive warning threshold according to historical data, the student's participation performance, and real-time adjustment of the classroom environment. At the same time, during the design process, design an abnormal score regularization term in the total abnormal score for the situation of false alarms caused by short-term behavior fluctuations of the student, ensuring that the warning mechanism is sensitive to long-term behavior abnormalities but tolerant of short-term fluctuations;

[0019] S403. Generate corresponding mental health warnings according to the total abnormal score with the regularization term added, including: no warning, mild warning, moderate warning, and severe warning, and feedback the final warning signal to the teacher side for display;

[0020] S5. Construct an intelligent feedback model according to the type of mental health warning and the classroom activity type, and feedback to the teacher for intervention according to the type of mental health warning;

[0021] S6. Optimize the individualized participation demand model according to the teacher's feedback and the student's subsequent behavior, so that the model adapts to the student's long-term performance and forms a closed-loop optimization.

[0022] Furthermore, the video feature is to extract the dynamic features of the student using the optical flow method; the audio feature is to reflect the voice interaction mode in the classroom by analyzing the voice features of the teacher and the student; the environmental feature is the background noise level in the classroom environment extracted by noise analysis; the classification model dynamically adjusts the decision boundary of the model through incremental learning, which is expressed as follows:

[0023]

[0024] where C i represents the i-th type of classroom activity, i = 1, 2,..., k; F(t) represents the classroom feature vector at the current time t; β i represents the feature weight vector corresponding to each activity type; P(C i |F(t)) represents the probability that the current classroom belongs to the type C i ; β jRepresents the feature weight vector corresponding to the current activity type.

[0025] Furthermore, in step S2, the designed participation demand coefficient λ s (t) is added to the personalized participation demand model to dynamically reflect the current participation demand of the student. The participation demand coefficient λ s (t) is expressed as follows:

[0026] λ s (t) = γ(C final (t)) · B s (t)

[0027] Among them, λ s (t) represents the participation demand standard of student s at time t, γ(C final (t)) represents the adjustment factor based on the classroom activity type, and B s (t) represents the historical participation baseline of the student;

[0028] The personalized participation demand model is expressed as follows:

[0029] M s (t) = λ s (t) + δ s

[0030] Among them, M s (t) represents the individualized participation demand model of student s at time t; δ s represents a correction factor used to consider the individual performance deviation in special cases.

[0031] Furthermore, the correction of the difference vector is based on the correction term δ s (t), which is used to dynamically adjust the acceptance degree of the difference according to the historical performance of the student, and is expressed as follows:

[0032] D s ′(t) = D s (t) + λ s ·δ s (t)

[0033] Among them, D s (t) represents the difference vector, and D s ′(t) represents the difference vector after adding the correction term; λ s represents the adjustment parameter used to control the influence size of the correction term, and is dynamically adjusted according to different individual characteristics of the student; δ s (t) represents the difference correction term, which includes dynamic adjustment factors in multiple dimensions;

[0034] The anomaly detection formula for judging whether the behavior of the student deviates from the normal range is expressed as follows:

[0035] ||D s ′(t)|| 2 >θ s (t)

[0036] Among them, ||D s ′(t)|| 2 represents the two-norm of the difference vector, indicating the total deviation of the student from the expected value in all dimensions; θ s (t) represents the adaptive anomaly detection threshold, which is flexibly set according to the individual differences of students and the differences in current classroom activities to prevent misjudgment caused by a unified standard; when the two-norm of the difference vector exceeds the threshold, the participation behavior of the student at this moment is an abnormal behavior.

[0037] Furthermore, the total anomaly score is calculated as follows:

[0038]

[0039] Among them, S s (t) represents the total anomaly score of student s at time t, used to evaluate the degree of abnormality in the student's mental health status; w k (C final (t)) represents the weight of the k-th dimension under the current classroom activity type C final (t), which changes dynamically with different classroom activities; D s ′ k (t) represents the corrected participation difference of student s in the k-th dimension.

[0040] Furthermore, the adaptive warning threshold is calculated as follows:

[0041]

[0042] Among them, represents the basic threshold preset according to historical data, used to judge different levels of warnings; λ s represents a regulation coefficient, which is adjusted individually according to the student's mental health file and historical participation behavior; δ s (t) represents the fluctuation term of the current classroom activity and recent participation behavior, reflecting the short-term mental health changes of the student, used to dynamically adjust the threshold;

[0043] Add the anomaly score regularization term, which is expressed as follows:

[0044] S s ′(t) = S s (t) + Ω s (t)

[0045] Among them, S sS′(t) is the total abnormal score after regularization; is a regularization term based on the variance of historical scores, representing the variance of the abnormal scores of students in the short term. N is the observation window, and β is the regularization coefficient term Ω s (t).

[0046] According to the total abnormal score S s ′(t) after regularization, the corresponding mental health early warning is generated, which is expressed as follows:

[0047]

[0048] where are the adaptive thresholds for students s under mild, moderate, and severe early warnings respectively.

[0049] Furthermore, the S5 specifically includes:

[0050] S501. Generate a feedback matrix based on the abnormal feedback values of each student in each dimension. The feedback matrix reflects the abnormal situation of students in each dimension and adjusts the feedback weights in combination with the classroom activity type;

[0051] S502. Generate an intervention suggestion matrix based on the feedback matrix. The intervention suggestion matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current early warning level;

[0052] S503. Form the generated feedback matrix and intervention suggestion matrix into feedback and suggestions and display them to the teacher through intelligent terminal devices, and make dynamic adjustments according to the teacher's feedback and suggestions.

[0053] Furthermore, the abnormal feedback value of each student in each dimension is expressed as follows:

[0054]

[0055] where represents the feedback value of the k-th dimension, reflecting the severity of the abnormality in this dimension; the feedback matrix is

[0056] Furthermore, the S6 specifically includes:

[0057] S601. Integrate the teacher's feedback, the student's historical data, and the current classroom behavior to generate a personalized feedback learning matrix L s (t), which is used to dynamically adjust the personalized participation demand model M s (t); the personalized feedback learning matrix L s (t) is expressed as follows:

[0058] L sL(t) = α·F(t) + β·H(t) + γ·I(t) s + β·H(t) s + γ·I(t) s (t)

[0059] Wherein, L s (t) represents the feedback learning matrix of student s at time t, integrating current feedback and historical behavior data; α, β, and γ represent weight parameters, indicating the impacts of current feedback, historical data, and intervention suggestions on the learning matrix respectively; F s (t) represents the feedback matrix; H s (t) represents the historical behavior matrix of the student, representing long-term data; I s (t) represents the intervention suggestion matrix, reflecting the intervention measures taken by the teacher and their effects;

[0060] S602. Dynamically optimize the personalized participation demand model M(t) of the student by using an adaptive learning strategy and a dynamic learning rate through the feedback learning matrix; the adaptive learning strategy is expressed as follows: s (t) is dynamically optimized; the adaptive learning strategy is expressed as follows:

[0061] M s '(t + 1) = M s (t) + η(t)·L s (t)

[0062] Wherein, M s '(t + 1) represents the optimized personalized participation demand model for the next classroom monitoring; M s (t) represents the personalized participation demand model at the current moment; η(t) represents the dynamic learning rate at time step t, used to control the impact of the learning matrix L s (t) on the model update;

[0063] The dynamic learning rate is calculated as follows:

[0064]

[0065] Wherein, Δ s (t) represents the abnormal change amount of student s at time t; λ represents a regulation factor, used to control the change amplitude of the learning rate; μ s represents the long-term participation behavior mean value of student s, reflecting the average participation level of the student;

[0066] S602. Introduce a long-term fluctuation control mechanism to make the personalized participation demand model adapt to long-term changes and perform long-term optimization.

[0067] In the second aspect of the present invention, a multi-modal-based student classroom mental health early warning system is provided, and the system includes:

[0068] A multi-modal acquisition unit for detecting the activity type of the current classroom in real time through multiple sensors;

[0069] A personality analysis unit for generating a dynamically adjusted personalized participation requirement model based on the activity type and combining the personalized data of each student, and generating individualized participation criteria;

[0070] An anomaly analysis unit for monitoring the participation behavior of students in the classroom in real time, performing a difference analysis with the personalized participation requirement model, and judging whether there is abnormal behavior based on the difference sharing. If there is an anomaly, it will be marked;

[0071] A psychological early warning setting unit for generating hierarchical mental health early warnings according to the results of the difference analysis, including mild, moderate and severe warnings;

[0072] An early warning intervention unit for constructing an intelligent feedback model based on the type of mental health early warning and the activity type of the classroom, and feeding it back to the teacher for intervention according to the type of mental health early warning;

[0073] A system optimization unit for optimizing the individualized participation requirement model according to the teacher's feedback and the subsequent behavior of the students, so that the model adapts to the long-term performance of the students and forms a closed-loop optimization.

[0074] The beneficial technical effects of the present invention are at least as follows:

[0075] (1) By establishing an individualized participation requirement model for each student, the present invention comprehensively considers their personality characteristics, historical participation data and mental health records, and dynamically adjusts the participation monitoring standard for each student. In different links of the classroom, the system makes early warning adjustments according to the unique behavior patterns of each student. For example, in the discussion session, extroverted students should have a higher participation rate, while introverted students can remain relatively silent. Through the personalized model, the system can perform refined management according to the specific needs of each student, avoiding false alarms and missed alarms under the unified standard, and greatly improving the accuracy of early warnings.

[0076] (2) By detecting the classroom activities in real time (such as teacher's explanation, interactive discussion, silent self-study, etc.), the present invention automatically adjusts the monitoring strategy of mental health early warnings. For example, in the explanation mode, the system focuses on monitoring the concentration of students' attention, while in the discussion mode, it focuses on monitoring the interaction frequency and the number of speeches of students. Each classroom activity corresponds to different mental health monitoring standards, and the system can flexibly adjust the early warning rules according to the changes in the classroom situation, ensuring the accuracy and real-time nature of early warnings.

[0077] (3) Through an intelligent feedback mechanism, the mental health early warning results of students are fed back to teachers in a timely manner in an appropriate way. For example, the system enables teachers to promptly understand which students have abnormal participation or obvious emotional fluctuations through a visual interface or appropriate reminder methods. At the same time, the system combines the individualized participation demand model of students to provide specific intervention suggestions for teachers. For example, if a certain student shows long-term low participation, the system will suggest that the teacher intervene by means of calling the student to answer questions or having individual after-class conversations. This feedback and intervention mechanism greatly enhances the teacher's classroom management ability, helps teachers pay attention to students' mental health problems in a timely manner without disturbing normal teaching, and take appropriate actions for intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The present invention will be further described with reference to the accompanying drawings. However, the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on the following drawings without creative efforts.

[0079] Figure 1 It is a flowchart of a method for early warning of students' mental health in the classroom based on multi-modal for an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0080] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0081] In one or more embodiments, as Figure 1 shown, a method for early warning of students' mental health in the classroom based on multi-modal of the present invention is disclosed. The method includes steps 1 to 6, including:

[0082] S1. Detect the activity type of the current classroom in real time through multiple sensors.

[0083] Specifically, in the detection of the classroom dynamic environment, first, the classroom audio and video data are collected in real time through multiple sensors. Set the video frame sequence V(t) and the audio signal A(t), where t is the time sequence. For each frame V(t), the present invention extracts multi-modal features:

[0084] Video features: Use the optical flow method to extract dynamic features F v (t) such as the limb movements and posture changes of students, and capture the overall activity of students in the classroom.

[0085] Audio features: By analyzing the voice features of teachers and students, the extracted audio signal features F a(t) includes volume, pitch, speech rate, etc., which can reflect the voice interaction pattern in the classroom.

[0086] Environmental characteristics: Through noise analysis, the background noise level N(t) in the classroom environment is extracted to judge the quietness of the classroom (for example, the noise level is lower in the self-study mode and higher in the discussion mode).

[0087] Furthermore, these multimodal data are combined into the overall feature vector F(t) of classroom activities through a formula:

[0088] F(t) = α 1 F v (t) + α 2 F a (t) + α 3 N(t)

[0089] where α 1 , α 2 , α 3 are weight parameters that measure the importance of video features, audio features, and noise features respectively. These parameters are adjusted according to different classroom activity types (for example, in the self-study mode, α 3 is higher because the quietness is more critical; in the discussion mode, α 2 accounts for the main weight because voice interaction is more frequent).

[0090] Furthermore, the above feature vector F(t) is input into the classification model, and the model will judge the current classroom activity type based on these features. The present invention proposes a decision-making mechanism based on dynamic feature fusion, which dynamically adjusts the decision boundary of the model through incremental learning. The core formula for classification is as follows:

[0091]

[0092] where C i represents the i-th classroom activity type (such as explanation, discussion, self-study, etc.), i = 1, 2,..., k. F(t) is the classroom feature vector at the current time t. β i is the feature weight vector corresponding to each activity type. P(C i |F(t)) is the probability that the current classroom belongs to type C i .

[0093] By continuously iterating, β i is updated to enable it to adapt to the feature changes of different classroom modes, ensure the accuracy of the classification model in long-term teaching, and be able to capture the dynamic changes of classroom activities.

[0094] Furthermore, to avoid misclassification caused by short-term fluctuations or noise, a sliding window smoothing strategy is proposed. Define the sliding window size as w and the time step as Δt, and perform a weighted average on the activity classification results within the past w time steps. The formula is as follows:

[0095]

[0096] Through this smoothing mechanism, the system can effectively remove temporary classification fluctuations, enhance the robustness of the system under short-term interference, and ensure accurate identification of the long-term trend in the classroom environment.

[0097] Furthermore, the smoothed result C final (t) represents the final classroom activity type at the current time point t. C final (t) represents the most likely activity scenario in the current classroom (such as lecture, discussion, self-study, etc.) by selecting the activity type with the highest probability from the classification probabilities within multiple time steps. The specific explanation is as follows:

[0098] C final (t) is the final output of the system, representing the classroom activity type recognized by the system at the current moment.

[0099] The C final (t) output by the system at each time step t will be used as the input for the next step to guide the adjustment of the student individual participation demand model and ensure the use of corresponding criteria under different types of classroom activities.

[0100] Therefore, C final (t) is the final classroom activity type label determined by the system at the current time point t, and this activity type will directly affect the dynamic adjustment of the subsequent student individual demand model.

[0101] S2. Generate a dynamically adjusted individual participation demand model based on the activity type and the personalized data of each student, and generate individual participation criteria.

[0102] In step 1, the output classroom activity type C final (t) (such as lecture, discussion, self-study) is used as the input for this step. According to different activity types, the system will initialize a corresponding individual participation demand model for each student s, and this model reflects the expected participation degree of the student in the current classroom activity.

[0103] Set the participation baseline B s (t) for each student, which is defined as in different activity types C final(t) represents the historical average participation level of the student. This baseline is constructed from the student's historical performance data and indicates the participation frequency in different classroom scenarios. For example, in a discussion scenario, the baseline of an extroverted student s may be relatively high, while in a self-study scenario, the baseline is relatively low.

[0104]

[0105] Among them, B s (t) is the participation baseline of student s at time t. n represents the number of samples in the historical data. P s (C final (t i )) is the participation of student s in the corresponding activity type C i (t final (t i ) at historical time t. This formula is used to generate a baseline participation model based on the student's historical performance.

[0106] Furthermore, the system dynamically adjusts the individualized participation requirements based on the student's participation baseline B s (t), in combination with the current classroom activity type C final (t). To flexibly reflect the student's current participation needs, a participation requirement coefficient λ s (t) is set to adjust the student's real-time participation standard.

[0107] λ s (t) = γ(C final (t)) · B s (t)

[0108] Among them, λ s (t) is the participation requirement standard of student s at time t. γ(C final (t)) is the adjustment factor based on the classroom activity type. Different classroom activities have different requirements for participation. For example, in the discussion mode, the γ value is relatively large, while in the self-study mode, the γ value is relatively small. B s (t) is the student's historical participation baseline. This dynamically adjusted model ensures that the system flexibly adjusts the participation requirements of each student according to the specific classroom activity type.

[0109] Furthermore, based on the participation requirement coefficient λ s (t) dynamically adjusted in step 2, an individualized participation requirement model of the student at the current time t is generated. This model reflects the expected behavior standard of the current student in the current classroom scenario.

[0110] M s (t) = λ s (t) + δ s

[0111] Among them, M s (t) represents the individualized participation demand model of student s at time t. δ s is a correction factor used to consider individual performance deviations in special cases. For example, factors such as a student's mood and recent performance may cause minor deviations in participation demand, and this value can be adjusted in real time through a machine learning model.

[0112] Finally, the individualized participation demand model M s (t) of each student generated by the system will be used in subsequent steps (such as real-time behavior monitoring and difference analysis) to ensure that the participation demand standard for each student is updated in real time according to their individual characteristics, historical participation, and classroom activity types, and can flexibly respond to demand changes in different scenarios.

[0113] This model will be used as the standard for subsequent behavior monitoring to determine whether a student's actual performance meets expectations and provide a basis for mental health early warning.

[0114] S3. Real-time monitor the participation behavior of students in the classroom, conduct difference analysis with the personalized participation demand model, and judge whether there is abnormal behavior based on the difference. If there is an abnormality, mark it.

[0115] Specifically, in the previous step (step 2), the generated individualized participation demand model M s (t) has been initialized according to the historical participation data, classroom activity types, and individual characteristics of each student. Now enter the stage of real-time participation behavior monitoring.

[0116] This model M s (t) represents the expected participation performance of student s at time t. It includes different dimensions, such as speaking frequency, eye contact, body movements, etc. These dimensions reflect the participation degree expected of students in the classroom. For each student s, the present invention defines these expected participation degrees as a multi-dimensional vector:

[0117]

[0118] Among them, represents the expected value of the kth participation dimension (such as speaking frequency, eye contact, etc.) of student s at time t, and d is the number of participation dimensions.

[0119] Next, the system collects the real-time participation behavior data of students through sensors and generates the real-time participation behavior vector A s (t). This vector contains the actual behavior data of each student in different participation dimensions. Formally:

[0120]

[0121] Among them, represents the actual performance value of student s in the k-th participation dimension at time t. These data come from a variety of sensors. For example, limb movements are captured by a camera, the number and duration of speeches are recorded by a microphone, and the attention concentration of students is analyzed by an eye movement tracking system.

[0122] Furthermore, after receiving the real-time participation behavior data A s (t), the system needs to compare it with the individualized participation demand model M s (t) to calculate the difference between the actual performance and the expected performance of the student. The difference vector D s (t) is calculated as follows:

[0123]

[0124] Among them, D s (t) is the participation difference vector of student s at time t, and each component represents the deviation between the actual performance and the expected value of the student in a certain participation dimension. Through this difference vector, the system can judge the degree of deviation between the actual performance of the student in each participation dimension and the model expectation.

[0125] Furthermore, in order to cope with the behavioral fluctuations of individual students and unpredictable factors in classroom activities, the system introduces a difference control term and a regularization term to enhance the robustness of the model. The present invention defines a new correction term δ s (t), whose function is to dynamically adjust the acceptance of differences according to the historical performance of the student.

[0126] D s ′(t) = D s (t) + λ s ·δ s (t)

[0127] Among them, D s ′(t) is the difference vector after adding the correction term. λ s is a regulation parameter used to control the influence size of the correction term, and is dynamically adjusted according to different individual characteristics of students. δ s (t) is the difference correction term, which contains dynamic adjustment factors in multiple dimensions, and considers factors such as the personality, emotional fluctuations, and long-term performance deviation of the student. This correction term is generated through the long-term performance curve and personality characteristics of the student, avoiding excessive influence of short-term fluctuations on the results. For example, if a certain student often has emotional fluctuations at certain times, the system will increase the tolerance for these differences according to his historical fluctuation situation and reduce false alarms.

[0128] Furthermore, once the present invention obtains the corrected difference vector D s′(t), the system will analyze this vector to determine whether the student's participation behavior deviates from the normal range. At this stage, the present invention introduces an adaptive participation anomaly threshold θ s (t), which is dynamically adjusted according to the student's historical performance, mental health file, and current classroom activities. The anomaly detection formula is as follows:

[0129] Anomaly condition: ||D s ′(t)|| 2 >θ s (t)

[0130] Where, ||D s ′(t)|| 2 is the two-norm of the difference vector, representing the total deviation of the student from the expected value in all dimensions. θ s (t) is the adaptive anomaly detection threshold, which is flexibly set according to individual student differences and different current classroom activities to prevent misjudgment caused by a unified standard. When the two-norm of the difference vector exceeds the threshold, the system will determine that the student's participation behavior at this moment is an abnormal behavior.

[0131] Furthermore, based on the results of the difference calculation and anomaly detection, the system will output the difference analysis conclusion for each student. Specifically, the system will mark which participation dimensions are abnormal, forming a report on behavioral anomalies. These reports will be used as the input for the next step (mental health early warning generation) to help the system better judge the mental state of students and perform early warning processing.

[0132] In summary, the system monitors the student's participation behavior in real time to generate the participation behavior vector A s (t). The real-time behavior data is used to calculate the difference with the individualized participation demand model M s (t) to obtain the difference vector D s (t). The innovative correction term δ s (t) and the adaptive threshold θ s (t) are introduced to enhance the system's adaptability to individual differences and behavioral fluctuations, ensuring accurate detection of abnormal behaviors. Finally, the difference analysis results are output, marking the abnormal participation behaviors, which serve as the basis for subsequent mental health early warning generation.

[0133] S4. Generate hierarchical mental health early warnings according to the difference analysis results, including mild, moderate, and severe warnings.

[0134] In step 3, the output difference vector D s ′(t) represents the corrected difference situation between the student s' participation behavior at time t and the expected model. This difference vector combines the student's individual participation performance, emotional fluctuations, and individualized participation needs, and adjusts the accuracy of anomaly detection through the correction term and the adaptive threshold.

[0135] In step 4, the goal of the present invention is to generate a mental health warning for students based on these difference data, ensuring that the system can identify potential mental problems of students and inform teachers in a timely manner. To enhance the innovation of the system and its adaptability to complex classroom scenarios, this step will be processed through the following stages to finally generate a multi-level mental health warning.

[0136] Furthermore, in a complex classroom environment, the behavior and mental state of students may be affected by multi-dimensional participation characteristics. The abnormalities in each participation dimension (such as speaking, body language, eye contact, etc.) have different impacts on the mental state of students. Therefore, before generating a warning, the system needs to perform weighted processing on the differences in different dimensions. Let D s ′(t) = [D s ′ 1 (t), D s ′ 2 (t),..., D s ′ d (t)] be the corrected difference vector of student s at time t. The system weights these difference values and combines the current classroom activity type C final (t) to obtain the total abnormality score S s (t) of the student.

[0137]

[0138] Among them, S s (t) is the total abnormality score of student s at time t, which is used to evaluate the degree of abnormality of the student's mental health state. w k (C final (t)) is the weight of the kth dimension under the current classroom activity type C final (t). This weight will change dynamically with different classroom activities. For example, in the lecture mode, the weights of body movements and eye contact may be larger, while in the discussion mode, the weight of the speaking frequency is higher. D s ′ k (t) is the corrected participation difference of student s in the kth dimension.

[0139] Furthermore, to enhance the accuracy of multi-dimensional abnormality scoring in complex situations, the present invention introduces a dynamic weighting mechanism, using the current classroom activity type as a weight adjustment factor. Through this mechanism, the system can flexibly adjust the weights of each dimension according to specific classroom activities (such as discussion, lecture, self-study) to more accurately reflect the current mental health state of students. For example, in a discussion mode, the weight of w 发言 will be larger, and the system focuses on monitoring whether students participate in the discussion; while in the self-study mode, the weight of w 目光接触The weight is even higher to ensure that students' attention is not distracted in the long term.

[0140] Furthermore, the performances of different students in class vary greatly, so a unified warning standard is not suitable for all students. To solve this problem, the system generates an adaptive warning threshold for each student. These thresholds are set not only based on historical data but also adjusted in real time according to the students' participation performance and classroom environment.

[0141] Set They are the adaptive thresholds for student s under mild, moderate, and severe warnings respectively. The calculation formula is as follows:

[0142]

[0143] Among them, is the basic threshold preset according to historical data, used to judge different levels of warnings. λ s is a regulation coefficient, which is adjusted individually according to the students' mental health files and historical participation behaviors. δ s (t) is the fluctuation term of the current classroom activities and recent participation behaviors, reflecting the short-term mental health changes of students and used to dynamically adjust the threshold.

[0144] This warning threshold ensures that the warning generation for each student is personalized, avoiding false positives and false negatives caused by a unified standard.

[0145] Furthermore, to avoid excessive false positives caused by short-term behavioral fluctuations of students (such as occasional distraction of attention), the system introduces an abnormal score regularization term Ω s (t). This regularization term can control the growth rate of the abnormal score, ensuring that the warning mechanism is sensitive to long-term behavioral abnormalities but has a certain tolerance for short-term fluctuations.

[0146] S s ′(t) = S s (t) + Ω s (t)

[0147] Among them, S s ′(t) is the total abnormal score after regularization. is a regularization term based on the variance of historical scores, representing the variance of the abnormal scores of students in the short term, N is the observation window, and β is the regularization coefficient.

[0148] When a student's behavior deviates from the expected value for a long time, Ω s (t) will increase, enhancing the cumulative effect of the abnormal score; while if a student has fluctuations in the short term but quickly recovers, Ω s (t) will decrease, thus avoiding false positives.

[0149] Further, based on the regularized total anomaly score S s ′(t), the system will generate corresponding mental health warnings. The warnings are divided into four levels: no warning, mild warning, moderate warning, and severe warning. The specific calculations are as follows:

[0150]

[0151] No warning: S s ′(t) is below the mild warning threshold, indicating that the student's participation behavior is normal or fluctuates within an acceptable range.

[0152] Mild warning: The student's total anomaly score is slightly higher, and there may be minor mental health problems. It is recommended that teachers pay mild attention or observe their subsequent performance.

[0153] Moderate warning: The student's total anomaly score is significantly higher than the mild threshold. The system will remind the teacher to pay attention to the student's mental health and recommends appropriate interventions in the classroom (such as asking questions, encouraging interaction, etc.).

[0154] Severe warning: The student's anomaly score has exceeded the severe warning threshold. The system will strongly recommend that the teacher have a private conversation with the student after class or seek the help of a professional psychological counselor.

[0155] Further, finally, the system will display the generated warning information through the teacher's terminal. The warning results include the following aspects:

[0156] Warning level: Displays which warning state the current student is in (mild, moderate, severe).

[0157] Anomaly dimension analysis: Details which participation dimensions are abnormal, such as speaking, body language, attention, etc., and gives specific difference data.

[0158] Suggested intervention measures: The system will provide personalized intervention suggestions for teachers based on the warning level and anomaly dimensions. For example, a mild warning may suggest that the teacher encourage student interaction by asking questions; a moderate warning recommends that the teacher communicate with the student after class.

[0159] Through this real-time feedback, teachers can quickly understand the mental health status of students, take appropriate actions in a timely manner, and prevent problems from worsening.

[0160] S5. Construct an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and feedback to the teacher for intervention according to the type of mental health warning.

[0161] In step 4, the system generated based on the student anomaly score S sMulti-level mental health early warning for ′(t), including mild, moderate, and severe early warnings. Each early warning result includes the abnormal level (mild, moderate, severe) and the differential analysis of specific participating dimensions (e.g., speech, eye contact, body language, etc.). This information serves as the input for this step to construct an intelligent feedback model. The core input variables are:

[0162] Early warning level: The level of mental health status determined by the system.

[0163] Dimension abnormal analysis: Reflects the abnormal degree of each participating dimension and represents the deviation of students in different dimensions.

[0164] The system needs to generate effective and actionable feedback based on these inputs and provide targeted intervention suggestions for teachers.

[0165] Furthermore, to provide personalized and accurate feedback, the system generates a feedback matrix F s (t) according to the abnormal dimension analysis results of each student. This matrix reflects the abnormal conditions of students in each dimension and adjusts the feedback weights in combination with the type of classroom activity. The formula for the feedback matrix is:

[0166]

[0167] Where, Where represents the feedback value of the k-th dimension, reflecting the severity of the abnormality in this dimension. w k (C final (t)) is the weight of the k-th dimension, indicating the importance of this dimension under the current type of classroom activity C final (t). For example, in the discussion mode, the weight w 发言 of speech will be relatively large; in the self-study mode, the weight w 目光接触 of eye contact is relatively high. This feedback matrix F s (t) is the core data of the system feedback, showing the specific abnormal performance of students to teachers. The feedback value of each dimension is dynamically adjusted according to the abnormal degree of participating behaviors.

[0168] Furthermore, the system not only generates the feedback matrix but also associates it with the type of classroom activity, enabling flexible adjustment of the feedback priorities in different scenarios. For example, in the discussion mode, the speech frequency is the dimension that is key monitored, while in the explanation mode, the attention of students (such as eye contact) is more important. Through this scenario-based weight adjustment, the system can provide targeted feedback under different teaching activities.

[0169] Furthermore, to further assist teachers in taking appropriate intervention measures, the system generates an intervention suggestion matrix I s(t). This matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current warning levels (mild, moderate, severe). The formula for the intervention recommendation matrix is:

[0170]

[0171] Among them, is the intervention recommendation for the k-th dimension, which combines the feedback value and the warning level. Its generation formula is:

[0172]

[0173] Among them, is an intervention generation function, which generates targeted intervention recommendations based on the feedback value the current warning level and the student's historical mental health file H s is the intervention recommendation generated for the k-th dimension, which may include question interaction, classroom nomination for speaking, after-class communication, or seeking psychological counseling, etc.

[0174] Furthermore, under different warning levels, the intensity of the intervention recommendations is different. Under mild warning, the system may recommend that teachers motivate students to participate through simple questions; while under moderate or severe warning, the system may recommend that teachers communicate with students one-on-one or contact the psychological counselor.

[0175] It can be understood that the intervention recommendations not only depend on the current feedback matrix, but also refer to the student's historical mental health file H s . For example, if a student has shown a similar behavior pattern in past courses, the system will recommend that teachers pay more attention to the student or recommend long-term counseling measures.

[0176] Furthermore, the generated feedback matrix F s (t) and the intervention recommendation matrix I s (t) will be displayed to the teacher through the intelligent terminal device so that the teacher can intuitively understand the current mental health status of the students and the intervention measures recommended by the system. To ensure that teachers can quickly understand the information, the system provides a variety of visualization tools:

[0177] Feedback dashboard: Displays the feedback matrix F s (t) of each student, and highlights the abnormal dimensions through visualization charts (such as heat maps, radar charts). For example, when a student shows abnormalities in certain dimensions (such as speaking frequency or attention concentration), the system will use color coding to mark these dimensions to help teachers quickly locate the problems.

[0178] The feedback value of each dimension ​It will be presented in a graphical way, enabling teachers to clearly see which dimensions require key attention. If the value is significantly high, the system will highlight this dimension and prompt the teacher to prioritize attention to the students' participation.

[0179] Intervention Suggestion Panel: The system simultaneously displays the generated intervention suggestion matrix I s (t), providing specific operation suggestions for each abnormal dimension. There will be corresponding operation guides for suggestions under different warning levels to ensure that teachers can quickly apply them in actual teaching. For example, under mild warning, the system may suggest that teachers motivate students to participate by asking questions; under severe warning, the system will suggest that teachers conduct individual communication after class or contact the school psychologist.

[0180] Furthermore, in addition to the intelligent feedback generated by the system, the system also allows teachers to dynamically adjust the feedback and suggestions. This mechanism enables teachers to optimize the suggestions in combination with the actual classroom situation and feedback them to the system, forming a closed loop of teacher-system interaction. Teachers can adjust the following:

[0181] Adjustment of Intervention Intensity: If teachers believe that the system's intervention suggestions are too radical or insufficient, they can adjust according to the actual classroom situation. For example, teachers can adjust moderate intervention to mild intervention, or upgrade mild intervention to moderate. Intervention intensity adjustment formula:

[0182]

[0183] where ψ is the teacher feedback adjustment function, which dynamically adjusts the intervention measures according to the teacher's feedback. Through this mechanism, the system not only provides standardized intervention suggestions but also learns and improves through teachers' feedback.

[0184] Confirmation and Revision of Feedback Content: If teachers believe that the system's abnormal warning for students does not conform to the actual situation, they can mark specific dimensions through the system and submit feedback. The system will adjust the subsequent monitoring strategy for this student according to the teacher's feedback and update its individualized participation demand model M s (t).

[0185] Furthermore, the teachers' feedback will be recorded by the system and used as an important basis for subsequent system optimization. Each adjustment and revision of the teachers' feedback to the system will affect the individualized participation demand model M s (t) of the students, enabling the system to more accurately judge the students' mental state in future monitoring and avoid false alarms or missed alarms.

[0186] Through this adaptive learning mechanism, the system can not only improve the mental health monitoring ability of the current classroom, but also gradually optimize the personalized demand model of each student during long-term operation, enhancing the accuracy and effectiveness of overall mental health management.

[0187] S6. Optimize the individualized participation demand model according to the teacher's feedback and the subsequent behavior of the student, enabling the model to adapt to the long-term performance of the student and form a closed-loop optimization.

[0188] In step 5, the system collects the teacher's intervention feedback and the real-time behavior performance of the student through the intelligent feedback mechanism. These data provide key inputs for the dynamic optimization of the individualized demand model. At this time, the inputs include:

[0189] Teacher feedback data: The result after the teacher's intervention on the student's participation behavior according to the feedback matrix and intervention suggestions.

[0190] Student historical data H s (t): The past participation behavior records of the student, including classroom participation, warning information, and intervention effects, etc.

[0191] These inputs will serve as the basis for optimizing the individualized demand model. The dynamic optimization of this model can not only better reflect the real-time performance of the student, but also make adaptive adjustments according to the long-term participation behavior of the student to improve the accuracy and personalization level of long-term monitoring.

[0192] Furthermore, in order to fully integrate the teacher's feedback, the student's historical data, and the current classroom behavior, the system generates a personalized feedback learning matrix L s (t), which will be used to dynamically adjust the individualized demand model M s (t).

[0193] The generation formula of the feedback learning matrix is as follows:

[0194] L s (t) = α·F s (t) + β·H s (t) + γ·I s (t)

[0195] Among them, L s (t) is the feedback learning matrix of student s at time t, integrating the current feedback and historical behavior data. α, β, Y are weight parameters, representing the influence of the current feedback, historical data, and intervention suggestions on the learning matrix respectively. These weights can be dynamically adjusted depending on the performance of different students. For example, for students with large long-term fluctuations in behavior, the value of β is larger to enhance the weight of historical data, while for students who are sensitive to feedback, α will increase accordingly. F s(t) is the feedback matrix, from Step 5, representing the teacher's feedback results for each participation dimension. H s (t) is the historical behavior matrix of the student, representing long-term data, such as participation fluctuations and mental health changes in the past few months, etc. I s (t) is the intervention recommendation matrix, reflecting the intervention measures taken by the teacher and their effects. Through this combination, L s (t) can effectively capture the participation anomalies of students in the current classroom, while referring to historical performance, to ensure that the model update takes into account both short-term fluctuations and long-term trends.

[0196] Furthermore, the system uses the feedback learning matrix L s (t) to dynamically optimize the individualized needs model M s (t) of the student. The optimization process adopts an adaptive learning strategy, enabling the model to flexibly adjust to adapt to the short-term behavior changes and long-term participation trends of the student. The optimization formula for the individualized needs model of student s at time t + 1 is defined as:

[0197] M s ′(t + 1) = M s (t) + η(t)·L s (t)

[0198] Where, M s ′(t + 1) is the optimized individualized needs model, used for the next classroom monitoring. M s (t) is the individualized needs model at the current moment. η(t) is the adaptive learning rate at time step t, used to control the impact of the learning matrix L s (t) on the model update.

[0199] Furthermore, the system introduces a dynamic learning rate η(t) for adaptively adjusting the step size of model optimization. To make the model more sensitive to short-term fluctuations of students while maintaining adaptability to long-term trends, the system calculates the dynamic learning rate through the following formula:

[0200]

[0201] Where, Δ s (t) is the anomaly change amount of student s at time t (such as a rapid change in participation). λ is a regulation factor used to control the change amplitude of the learning rate. μ s is the long-term participation behavior mean of student s, reflecting the average participation level of the student.

[0202] The design of this dynamic learning rate ensures that when the abnormal behavior of the student increases sharply (for example, the engagement drops significantly), η(t) increases, enabling the model to quickly adapt to the changes; while when the change amplitude is small, η(t) is small, maintaining the stable update of the model.

[0203] Furthermore, to ensure that the optimization of the individualized demand model not only depends on short-term feedback but also can adapt to long-term changes, the system introduces a long-term fluctuation control mechanism, which restricts the overfitting and short-term fluctuation interference of the model through a regularization term.

[0204] The present invention introduces a regularization term Ω s (t) to balance the update amplitude of the model and control the excessive influence of short-term fluctuations on the model:

[0205] M s ′(t + 1) = M s (t) + η(t)·L s (t) - λ r ·Ω s (t)

[0206] where Ω s (t) is a regularization term based on the long-term participation fluctuations of the student, controlling the excessive amplitude of the model update. λ r is the regularization strength coefficient, determining the influence size of the regularization term. If the student's participation performance is stable in the long term, λ r is small; if the student has large fluctuations in the long term, λ r is large, to suppress overfitting of short-term behaviors. The regularization term Ω s (t) calculates considering the participation fluctuations of the student in multiple past time periods, to ensure that the model can balance short-term behaviors and long-term trends. Its specific calculation is as follows:

[0207]

[0208] where N is the size of the historical window, indicating that the system refers to the participation data in the past N classes. is the mean value of the student's individualized demand model within this window period, reflecting the long-term participation performance. This design can effectively suppress the excessive model adjustment caused by short-term fluctuations, ensuring that the model can absorb the stable data in the long-term trend when updated.

[0209] Furthermore, after the system completes the optimization of the model, the generated new individualized demand model M s ′(t + 1) is used for the classroom monitoring in the next cycle and enters the closed-loop feedback loop. The long-term goal of the system is that through each optimization, the model can more and more accurately reflect the participation needs of the student, thereby improving the accuracy of monitoring and the intervention effect.

[0210] The final output is the optimized model M s ′(t + 1), and the system will continue to monitor the difference between the model and the actual behavior in subsequent classes to form a complete closed-loop optimization mechanism.

[0211] In one or more embodiments, the present invention discloses a multi-modal student classroom mental health early warning system, which includes:

[0212] A multi-modal acquisition unit for real-time detecting the activity type of the current classroom through multiple sensors;

[0213] A personality analysis unit for generating a dynamically adjusted personalized participation requirement model according to the activity type and combining the personalized data of each student, and generating individualized participation criteria;

[0214] An anomaly analysis unit for real-time monitoring the participation behavior of students in the classroom, performing difference analysis with the personalized participation requirement model, and judging whether there is abnormal behavior according to the difference sharing. If there is an anomaly, it will be marked;

[0215] A mental health early warning setting unit for generating hierarchical mental health early warnings according to the difference analysis results, including mild, moderate and severe warnings;

[0216] An early warning intervention unit for constructing an intelligent feedback model according to the type of mental health early warning and the activity type of the classroom, and feeding it back to the teacher for intervention according to the type of mental health early warning;

[0217] A system optimization unit for optimizing the individualized participation requirement model according to the teacher's feedback and the subsequent behavior of students, so that the model adapts to the long-term performance of students and forms a closed-loop optimization.

[0218] For some preferred embodiments of the present invention, of course, the scope of the rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A multimodal student classroom mental health early warning method, characterized in that: The method comprises the following steps: S1. Use multiple sensors to detect the current classroom activity type in real time, including: S101, collecting multimodal data of the current classroom through multiple sensors, and extracting and fusing multimodal features to obtain an overall feature vector; the multimodal features include video features, audio features and environmental features; S102, inputting the overall feature vector into a classification model to determine the activity type of the current class; S2. Generate a dynamically adjusted personalized participation demand model based on the activity type and each student’s personalized data, and generate individualized participation criteria, including: Define the historical average participation level of students under different activity types as the participation baseline of each student, and build a personalized participation demand model in combination with the current classroom activity type, wherein the personalized participation demand model represents the expected participation performance of the current student in the current time period; S3. Monitor students’ participation behaviors in class in real time and analyze the differences with the personalized participation demand model. Determine whether there are abnormal behaviors based on the differences and mark them if there are any abnormal behaviors, including: S301, forming a multidimensional vector with the expected participation performance of each student in the personalized participation demand model, and forming a real-time participation behavior vector with the participation behavior data collected in real time; S302, calculating the difference vector between the real-time participation behavior vector and the multidimensional vector, which is expressed as the difference between the student's actual performance and the expected performance, and correcting the difference vector, dynamically adjusting the acceptance of the difference according to the student's historical performance, and obtaining a corrected difference vector; S303, analyzing the corrected difference vector to determine whether the student's behavior deviates from the normal range. If the second norm of the difference vector exceeds a threshold, the student's participation behavior at this moment is abnormal behavior; S4. Generate stratified mental health warnings based on the difference analysis results, including mild, moderate and severe warnings, including: S401, dynamically weighting the corrected difference vector and calculating the total abnormality score of the student in combination with the activity type of the current class; S402. Design an adaptive warning threshold based on historical data, students’ participation performance and real-time adjustment of the classroom environment. At the same time, during the design process, design an abnormality score regularization term in the total abnormality score to target the situation where students’ short-term behavioral fluctuations lead to false alarms, so as to ensure that the warning mechanism is sensitive to long-term behavioral abnormalities but has tolerance for short-term fluctuations. S403, generating corresponding mental health warnings according to the total abnormality score with the regularization term added, including: no warning, mild warning, moderate warning and severe warning, and feeding back the final warning signal to the teacher for display; S5. Build an intelligent feedback model based on the type of mental health warning and the type of classroom activities, and provide feedback to teachers for intervention based on the type of mental health warning; S6. Optimize the personalized participation demand model based on teacher feedback and students’ subsequent behavior, so that the model can adapt to students’ long-term performance and form a closed-loop optimization.

2. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The video feature is the dynamic feature of students extracted by optical flow method; the audio feature is the voice interaction mode in the classroom by analyzing the voice features of teachers and students; the environmental feature is the background noise level in the classroom environment extracted by noise analysis; the classification model dynamically adjusts the decision boundary of the model through incremental learning, which is expressed as follows: ； in, Indicates Types of classroom activities ; Indicates the current time The classroom feature vector of Represents the feature weight vector corresponding to each activity type; Indicates that the current class belongs to the type The probability of Represents the feature weight vector corresponding to the current activity type.

3. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: In S2, the designed participation demand coefficient is added to the personalized participation demand model , dynamically reflects the students' current participation needs, the participation demand coefficient It is expressed as follows: ； in, Indicates time Lower school students The participation requirements standard, represents the adjustment factor based on the type of classroom activity, represents the student’s historical participation baseline; The personalized participation demand model is expressed as follows: ； in, Indicates students In time Personalized participation demand model; represents a correction factor to account for individual performance deviations in special cases.

4. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The difference vector is modified to dynamically adjust the acceptance of the difference according to the student's historical performance, as shown below: ； in, represents the difference vector, represents the difference vector after adding the correction term; It represents the adjustment parameter, which is used to control the effect size of the correction term and is dynamically adjusted due to different individual characteristics of students; Represents the difference correction term, which includes dynamic adjustment factors in multiple dimensions; The anomaly detection formula for determining whether a student's behavior deviates from the normal range is expressed as follows: ； in, represents the two-norm of the difference vector, which represents the total deviation of the student from the expected value in all dimensions; Represents the adaptive anomaly detection threshold, which is flexibly set according to the individual differences of students and the current classroom activities to prevent misjudgment caused by a unified standard; when the second norm of the difference vector exceeds the threshold, the student's participation behavior at this moment is abnormal.

5. According to the multimodal student classroom mental health early warning method of claim 1, it is characterized in that: The total abnormality score is calculated as follows: ； in, Indicates students In time The total abnormality score is used to assess the abnormality of students’ mental health status; Indicates Dimensions in current classroom activity types The weights under, which change dynamically with different classroom activities; Indicates students In the Corrected participation differences by dimension.

6. A multimodal student classroom mental health early warning method according to claim 5, characterized in that: The adaptive warning threshold is calculated as follows: ； in, It indicates the basic threshold preset based on historical data, which is used to judge different levels of warnings; It represents an adjustment coefficient, which is personalized according to the student's mental health profile and historical participation behavior; The fluctuation term representing current classroom activities and recent participation behaviors reflects the changes in students’ mental health in the short term and is used to dynamically adjust the threshold. Add the anomaly score regularization term, expressed as follows: ； in, is the total anomaly score after regularization; is a regularization term based on the variance of historical scores, which represents the variance of abnormal scores of students in the short term. It is the observation window. is the regularization coefficient ; According to the total anomaly score after regularization , generate the corresponding mental health warning, as follows: ； in, For students Adaptive thresholds for mild, moderate, and severe warnings.

7. The multimodal student classroom mental health early warning method according to claim 1 is characterized in that: The S5 specifically includes: S501, generating a feedback matrix according to the abnormal feedback value of each student in each dimension, wherein the feedback matrix reflects the abnormal situation of the student in each dimension, and adjusting the feedback weight in combination with the type of classroom activity; S502: Generate an intervention suggestion matrix based on the feedback matrix, wherein the intervention suggestion matrix provides specific intervention actions for each abnormal dimension and quantifies the intervention intensity in combination with the current warning level; S503: The generated feedback matrix and intervention suggestion matrix are presented to the teacher through the smart terminal device in the form of feedback and suggestions, and dynamic adjustments are made according to the teacher's feedback and suggestions.

8. A multimodal student classroom mental health early warning method according to claim 7, characterized in that: The abnormal feedback value of each student in each dimension is expressed as follows: ； in, Indicates Dimensions in current classroom activity types The weight under Representative The feedback value of the dimension reflects the severity of the abnormality in this dimension; the feedback matrix is .

9. The multimodal student classroom mental health early warning method according to claim 1 is characterized in that: The S6 specifically includes: S601, Integrate teacher feedback, student historical data and current classroom behavior to generate a personalized feedback learning matrix , used to dynamically adjust the personalized participation demand model ; The personalized feedback learning matrix It is expressed as follows: ； in, Indicates students In time The feedback learning matrix integrates current feedback and historical behavior data; represents the weight parameters, which respectively represent the influence of current feedback, historical data and intervention suggestions on the learning matrix; represents the feedback matrix; The matrix represents the students' historical behavior, representing long-term data; It represents the intervention suggestion matrix, which reflects the intervention measures taken by teachers and their effects; S602. Modeling students' personalized participation needs using adaptive learning strategies and dynamic learning rates through feedback learning matrices Perform dynamic optimization; the adaptive learning strategy is expressed as follows: ； in, It represents the optimized personalized participation demand model for the next class monitoring; Represents the personalized participation demand model at the current moment; Represents the time step The dynamic learning rate under is used to control the learning matrix Impact on model updates; The dynamic learning rate is calculated as follows: ； in, Indicates students In time The amount of abnormal change; Represents the adjustment factor, which is used to control the variation of the learning rate; Indicates students The mean of long-term participation behavior reflects the average level of students’ participation; S602. Introduce a long-term fluctuation control mechanism to enable the personalized participation demand model to adapt to long-term changes and perform long-term optimization.

10. A system for executing a multimodal student classroom mental health early warning method as claimed in claim 1, the system comprising: The multimodal acquisition unit is used to detect the activity type of the current classroom in real time through multiple sensors, and performs the following operations: Collect multimodal data of the current class through multiple sensors, extract and fuse multimodal features to obtain an overall feature vector; the multimodal features include video features, audio features and environmental features; input the overall feature vector into a classification model to determine the activity type of the current class; The personality analysis unit is used to generate a dynamically adjusted personalized participation demand model based on the activity type and the personalized data of each student, and to generate individualized participation standards. The execution includes: Define the historical average participation level of students under different activity types as the participation baseline of each student, and build a personalized participation demand model in combination with the current classroom activity type, wherein the personalized participation demand model represents the expected participation performance of the current student in the current time period; The abnormal analysis unit is used to monitor the participation behavior of students in the classroom in real time, and perform difference analysis with the personalized participation demand model. It determines whether there is abnormal behavior based on the difference sharing, and marks it if there is an abnormality. The execution includes: The expected participation performance of each student in the personalized participation demand model is composed into a multidimensional vector, and the participation behavior data collected in real time is composed into a real-time participation behavior vector; the difference vector between the real-time participation behavior vector and the multidimensional vector is calculated, which is expressed as the difference between the actual performance and the expected performance of the student, and the difference vector is corrected, and the acceptance of the difference is dynamically adjusted according to the historical performance of the student to obtain the corrected difference vector; Analyze the corrected difference vector to determine whether the student's behavior deviates from the normal range. If the second norm of the difference vector exceeds the threshold, the student's participation behavior at this moment is abnormal. The psychological warning setting unit is used to generate hierarchical psychological health warnings based on the difference analysis results, including mild, moderate and severe warnings. The execution includes: Dynamically weight the corrected difference vector, and calculate the total abnormality score of the student in combination with the current class activity type; design an adaptive warning threshold based on historical data, student participation performance and real-time adjustment of the classroom environment. At the same time, in the design process, design an abnormality score regularization term in the total abnormality score to address the situation where students' short-term behavioral fluctuations lead to false alarms, ensuring that the warning mechanism is sensitive to long-term behavioral abnormalities but has tolerance for short-term fluctuations; generate corresponding mental health warnings based on the total abnormality score with the addition of regularization terms, including: no warning, mild warning, moderate warning and severe warning, and feed back the final warning signal to the teacher for display; the warning intervention unit is used to build an intelligent feedback model based on the type of mental health warning and the type of classroom activity, and feed back to the teacher for intervention based on the type of mental health warning; The system optimization unit is used to optimize the personalized participation demand model based on the teacher's feedback and the students' subsequent behavior, so that the model can adapt to the students' long-term performance and form a closed-loop optimization.

Images