Student psychological health education monitoring method

Abstract

The invention relates to the technical field of data processing, in particular to a student psychological health education monitoring method, which comprises the following steps: extracting historical video frames based on historical activity videos of students collected in a target scene, and determining historical expression states, action states and social contact states of each student in the historical video frames to establish a dynamic baseline model; determining a normal behavior mode and a normal emotional response range of each student in each target scene according to the dynamic baseline model; acquiring real-time activity videos of students in the target scene to extract real-time video frames, and determining real-time expression states, real-time action states and real-time social contact states of the students according to the real-time video frames; and according to the real-time expression state, the real-time action state and the real-time social state of each student and the corresponding normal behavior mode and the normal emotional response range, determining an abnormal target and carrying out mental health early warning. The psychological health monitoring accuracy of students is improved.

Description

Technical Field

[0001] The invention relates to the technical field of data processing, and in particular to a method for monitoring students' mental health education. Background Art

[0002] Student mental health monitoring is a mental health monitoring work carried out for all students and is an important basis for carrying out school mental health education. Students may encounter various psychological problems in the process of growing up, such as learning pressure, interpersonal relationship problems, family changes, etc., which may cause anxiety, depression, inferiority and other psychological problems. Through mental health education monitoring, students' psychological abnormalities can be detected in time so that appropriate intervention measures can be taken to avoid further deterioration of the problem.

[0003] Chinese patent publication number: CN110729049B, discloses a mental health warning method, including: obtaining audio-visual data of multiple students listening to classes in different subjects within a preset first time period; wherein the audio-visual data of classroom listening to classes includes emotional information and body movements; according to the emotional information of each student, obtaining a first curve of each student's emotion changing over time; according to each student's body movements, obtaining a second curve of each student's body movements changing over time; according to each student's first curve and / or second curve, judging whether the student has psychological abnormality; when there is psychological abnormality, generating a first warning prompt information; sending the first warning prompt information to a terminal.

[0004] It can be seen that although the above technical solution can achieve early warning of students' abnormal psychology, promote students' healthy development and improve teaching effectiveness, there are still the following problems: using unified standards to measure students' mental health, not considering individual differences, and unable to flexibly monitor dynamically according to students' status. Summary of the invention

[0005] To this end, the present invention provides a student mental health education monitoring method to overcome the problems in the prior art that the mental health education monitoring method does not consider the interference of individual differences and cannot flexibly perform dynamic monitoring according to the student status.

[0006] To achieve the above object, the present invention provides a method for monitoring student mental health education, comprising:

[0007] Step S1, extracting historical video frames based on the student historical activity videos collected in the target scene, and determining the historical expression state, historical action state and historical social state of each student in the historical video frames;

[0008] Step S2, establishing a dynamic baseline model according to the historical expression state, historical action state and historical social state, and determining the normal behavior pattern and normal emotional response range of each student in each target scenario according to the dynamic baseline model;

[0009] Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state and real-time social state of each student according to the real-time video frames;

[0010] Step S4, determining abnormal targets and issuing mental health warnings based on the real-time expression status, real-time action status, real-time social status of each student and their corresponding normal behavior patterns and normal emotional response ranges.

[0011] Furthermore, in the step S1, the expression degree of the key facial parts of each student in the historical video frame is determined respectively, and the historical expression state of each student in each historical video frame is determined according to the expression degree;

[0012] Among them, the key facial parts include eyes, mouth and eyebrows, and the expression degree is the degree of eye opening and closing, the distance between eyebrows, the degree of eyebrow raising, and the degree of mouth raising.

[0013] Furthermore, in the step S1, the body contours of each student in the historical video frames are determined respectively, the skeletal key points of each student are determined according to the body contours, and the historical action states of each student are determined according to the skeletal key points.

[0014] Furthermore, in the step S1, the interaction distance between each student and others in each of the historical video frames is determined, and the social status of each student is determined based on the interaction distance.

[0015] Furthermore, in step S2, the single-frame state combination of each student is determined according to the historical expression state, historical action state and historical social state of each student in each historical video frame to construct a historical state set, and the state distribution probability of each student is determined based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model.

[0016] Furthermore, in step S2, the behavior sequence of each student in the target scene is determined according to the appearance order of the single-frame state combination of each student in the dynamic baseline model and the corresponding target scene, and the normal behavior pattern of each student is determined according to the behavior sequence.

[0017] Furthermore, in step S2, the historical expression state and historical action state of each student in the normal behavior mode are determined, and the normal emotional response range of each student is determined according to the corresponding relationship between the historical expression state and the historical action state.

[0018] Furthermore, in step S4, a single-frame real-time state combination of each student is determined according to the real-time expression state, real-time action state and real-time social state of each student to determine the real-time behavior sequence, the behavior matching degree is determined according to the real-time behavior sequence and the normal behavior pattern in the corresponding target scene, the emotion matching degree is determined according to the real-time expression state, real-time action state and the normal emotional response range, and the abnormal target is determined according to the behavior matching degree and the emotion matching degree.

[0019] Furthermore, in the step S4, a comprehensive matching degree is determined according to the behavior matching degree and the emotion matching degree, and an abnormal target is determined according to the comprehensive matching degree and the preset matching degree.

[0020] Furthermore, the abnormal targets monitored within a preset time period are analyzed to determine the abnormal distribution range and the abnormal duration, and the update timing of the dynamic baseline model is determined according to the abnormal distribution range and the abnormal duration.

[0021] Compared with the prior art, the beneficial effect of the present invention is that the present invention comprehensively captures students' behavior and emotional information, including historical expression status, historical social status, and historical behavior status, by analyzing historical video data collected in the target scene, thereby constructing a targeted individualized dynamic baseline model to improve the accuracy of mental health monitoring. At the same time, combined with real-time monitoring and comparison, it can timely detect abnormal changes in students' behavior and emotions, which helps prevent the occurrence of serious psychological problems and ensure the psychological health development of students.

[0022] Furthermore, by analyzing the expression levels of key facial parts of students in a large number of historical video frames, the present invention can determine the patterns of changes in students' facial expressions in different scenarios, thereby providing an important basis for analyzing students' psychological state, thereby achieving effective monitoring of students' mental health and helping to further ensure students' mental health development.

[0023] Furthermore, the present invention can directly obtain students' social behavior data in natural scenes by determining the interaction distance between each student and others in each historical video frame, and obtain more objective and real social status information based on the interaction distance. It is helpful to establish a normal social distance mode for each student through long-term monitoring of students' interaction distance, so as to effectively monitor and warn students' mental health, and help further ensure students' mental health development.

[0024] Furthermore, the present invention analyzes all historical video frames, collects single-frame state combinations at each moment in the target scene to form historical state sets of each student, determines the state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model, and further determines whether the student behavior is normal, thereby effectively monitoring and warning the students' mental health, which helps to further ensure the students' mental health development.

[0025] Furthermore, by determining the behavior sequence and normal behavior pattern, the present invention can accurately capture the personalized behavior characteristics of each student, avoid ignoring individual differences, and use a simple assessment method to judge abnormal behavior of all students. Based on the monitoring of behavior sequence and normal behavior pattern, the present invention can more accurately judge the mental health of students, which helps to further ensure the mental health development of students.

[0026] Furthermore, the present invention determines the single-frame real-time state combination and real-time behavior sequence by comprehensively considering the real-time expressions, actions, and social states of students, and further calculates the behavior matching degree and emotion matching degree to determine the abnormal target, breaking the limitation of single-dimensional evaluation, and can monitor students from multiple angles in a three-dimensional manner, accurately capturing the subtle changes in students' states, and greatly improving the accuracy of monitoring students' mental health and behavior states. This further improves the accuracy of students' mental health warnings and helps to protect students' mental health.

[0027] Furthermore, the present invention determines the comprehensive matching degree according to the behavior matching degree and the emotion matching degree, and determines the abnormal target according to the comprehensive matching degree and the preset matching degree. This helps to improve the accuracy and reliability of mental health monitoring. It further facilitates the timely issuance of mental health warnings, realizes the attention to abnormal targets, and helps to protect the mental health of students.

[0028] Furthermore, the present invention analyzes the abnormal targets monitored within a preset time period, determines the abnormal distribution range and abnormal duration, and determines the update timing of the dynamic baseline model accordingly, thereby avoiding misjudgment or missed judgment due to the lag of the dynamic baseline model, improving the reliability of mental health early warning, and avoiding waste of resources caused by excessively frequent updates while reasonably determining the update timing to ensure model accuracy, thereby achieving efficient use of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A step diagram of the student mental health education monitoring method of the present invention;

[0030] Figure 2 A diagram of the steps for constructing a dynamic baseline model according to an embodiment of the present invention;

[0031] Figure 3 A diagram showing the steps of determining an abnormal target according to an embodiment of the present invention;

[0032] Figure 4 A diagram of the steps for determining a dynamic baseline model update timing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0035] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0036] See also Figure 1 As shown, it is a step diagram of a student mental health education monitoring method according to an embodiment of the present invention. Specifically, the present invention provides a student mental health education monitoring method, comprising:

[0037] Step S1, extracting historical video frames based on the student historical activity videos collected in the target scene, and determining the historical expression state, historical action state and historical social state of each student in the historical video frames;

[0038] Step S2, establishing a dynamic baseline model according to the historical expression state, historical action state and historical social state, and determining the normal behavior pattern and normal emotional response range of each student in each target scenario according to the dynamic baseline model;

[0039] Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state and real-time social state of each student according to the real-time video frames;

[0040] Step S4, determining abnormal targets and issuing mental health warnings based on the real-time expression status, real-time action status, real-time social status of each student and their corresponding normal behavior patterns and normal emotional response ranges.

[0041] It is understandable that collecting historical activity videos in target scenes, such as classrooms, playgrounds and other daily activity places, can obtain students' behavior patterns in a natural state. At the same time, students' mental state will be manifested in many aspects such as expressions, actions and social interactions. Therefore, collecting historical video frames and analyzing historical expression states, historical action states and historical social states can obtain students' behavioral information from multiple angles, which is convenient for a more comprehensive and accurate understanding of students' mental conditions.

[0042] It is understandable that each student has a unique personality, habits and way of expression. By establishing a dynamic baseline model and determining normal behavior patterns and normal emotional response ranges based on students' own historical data, we can fully respect individual differences, ensure the timeliness and accuracy of evaluation standards, and achieve timely and effective early warning of students' mental health.

[0043] In a specific embodiment, the target scene includes scenes such as classrooms, cafeterias, libraries, and playgrounds. The student historical activity video is composed of a series of continuous image frames. The frame rate represents the number of frames displayed per second, so the historical video frames are uniformly extracted based on the frame rate. For example, for a video with a frame rate of 30 frames per second, a historical video frame is extracted every 30 frames. In implementation, the target scene can be determined according to actual conditions, which is not specifically limited here and will not be repeated.

[0044] The present invention comprehensively captures students' behavior and emotional information, including historical expression status, historical social status, and historical behavior status, by analyzing historical video data collected in target scenarios, thereby constructing a targeted individualized dynamic baseline model to improve the accuracy of mental health monitoring. At the same time, combined with real-time monitoring and comparison, it can timely detect abnormal changes in students' behavior and emotions, which helps prevent the occurrence of serious psychological problems and ensure the mental health development of students.

[0045] Specifically, in the step S1, the expression degree of the key facial parts of each student in the historical video frame is determined respectively, and the historical expression state of each student in each historical video frame is determined according to the expression degree;

[0046] Among them, the key facial parts include eyes, mouth and eyebrows, and the expression degree is the degree of eye opening and closing, the distance between eyebrows, the degree of eyebrow raising, and the degree of mouth raising.

[0047] It is understandable that people's facial expressions are important external manifestations of emotions and psychological states, and eyes, mouths and eyebrows play a key role in expressing emotions, so these three parts are selected as key parts to analyze students' expression states. Changes in the degree of eye opening and closing can reflect emotions such as surprise, excitement, fatigue, and thinking. Changes in the distance between eyebrows can reflect the degree of tension or relaxation of emotions. The degree of eyebrow upturn can also help distinguish different emotional tendencies. The greater the degree of mouth upturn, the happier and happier it is. By comprehensively analyzing the degree of each expression of key facial parts such as eyes, mouth, and eyebrows, the expression state of each student in the historical video frame can be determined. For example, if the eyes are wide open, the eyebrows are upturned, and the mouth is obviously upturned, it means that the student is in a happy and excited state; if the eyes are slightly narrowed, the eyebrows are tightly furrowed, and the mouth is downturned, it may mean that the student is unhappy, troubled or angry.

[0048] In a specific embodiment, image processing technology and deep learning technology can be used to determine the degree of expression of each student's key facial parts. Facial key point detection models such as Dlib, MTCNN, etc. can be pre-trained. These models can accurately detect multiple key points of the face, including key points of the eyes, eyebrows, mouth, etc. The degree of eye opening and closing can be measured by calculating the distance between the key points of the upper and lower eyelids; the distance between the eyebrows can be obtained by the distance between the two eyebrow key points; the degree of eyebrow lifting can be determined by the position change of the eyebrow peak key point relative to other eyebrow key points; the degree of mouth lifting can be determined by the vertical position change of the mouth corner key point. In implementation, the degree of expression of each student's key facial parts can also be determined by image recognition or other deep learning model building methods, which are not specifically limited here and will not be repeated.

[0049] By analyzing the expression levels of key facial parts of students in a large number of historical video frames, the present invention can determine the changing patterns of students' facial expressions in different scenarios, thereby providing an important basis for analyzing students' psychological state, thereby achieving effective monitoring of students' mental health and helping to further ensure students' mental health development.

[0050] Specifically, in step S1, the body contours of each student in the historical video frame are determined respectively, the skeletal key points of each student are determined based on the body contours, and the historical action states of each student are determined based on the skeletal key points.

[0051] It is understandable that the skeletal key points can reflect the relative positions and movement trends of various parts of the student's body. By analyzing the position changes, angle changes and relative relationships between these skeletal key points, the student's historical movement status can be inferred. For example, when the relative positions of the skeletal key points of the elbow and the shoulder and wrist skeletal key points undergo specific changes, and the arm bones form a certain angle, it may mean that the student is raising his hand; if the skeletal key points of the hip, knee and ankle show specific movement trajectories and angle changes, it may mean that the student is walking, running or jumping. Through the dynamic analysis of skeletal key points in continuous video frames, it is possible to accurately determine the specific actions of students at different times, and then understand the behavior patterns and activities of students in the target scene.

[0052] In a specific embodiment, the body contour of each student in the historical video frame is determined by using color space difference, edge detection algorithm, etc. If the background in the student's historical activity video is significantly different from the student's body color, a color threshold is set to extract the body contour. Alternatively, an edge detection algorithm is used to determine the student's body contour. Determine the key points of the skeleton on the basis of obtaining the body contour. The scope of the key points of the skeleton includes shoulders, elbows, wrists, hips, knees, ankles, etc. A human posture estimation method based on deep learning can be used to input a historical video frame image containing the student's body contour. The model will determine the position of the key points of the skeleton of the student in the historical video frame according to the learned human body structure features. In implementation, a traditional feature-based method can also be used to locate the key points of the skeleton according to the shape, proportion and other features of the body contour, combined with some prior knowledge, which is not specifically limited here and will not be repeated.

[0053] Specifically, in the step S1, the interaction distance between each student and others in each of the historical video frames is determined, and the social status of each student is determined based on the interaction distance.

[0054] In a specific embodiment, different interaction distances reflect different social relationships and interactive properties. The interaction distance is the spatial distance between each student and other students in the historical video frame. The interaction distance can be determined by measuring the pixel distance between the key points of the student's body (such as the head, the center of the torso, etc.), and then converting it into the actual physical distance according to the actual scene and proportional relationship of the video shooting. When the distance between students is relatively close within 0.5m, it means intimate distance, and they may be interacting intimately, cooperating or playing; the distance is between 0.5m and 1.5m, which is a personal distance, suitable for general interactive scenes such as dialogue and discussion. When the distance is 1.5m, it is a public distance, which means that there are relatively few interactions, the social relationship is relatively distant, or it is in a non-interactive state. For example, on the playground, some students are active in different positions, and the distance between them will be far. In implementation, the interaction distance is determined according to the actual situation, which is not specifically limited here and will not be repeated.

[0055] In a specific embodiment, the interaction frequency between each student and others may also be determined based on the historical video frames, and the social status of each student may be determined based on the interaction distance and the interaction frequency.

[0056] It can be understood that the interaction frequency refers to the number of times a student interacts with other students within a certain time frame. When analyzing historical video frames, the interaction frequency can be determined by observing the changes in students' behavior in consecutive video frames. A higher interaction frequency usually means that students are more actively involved in social activities, have a higher degree of social activity, and have a stronger willingness and ability to socialize. On the contrary, if students are relatively introverted, passive, or have certain obstacles in social interaction, they will have a lower interaction frequency and will not actively communicate and interact with others.

[0057] In a specific embodiment, combining the interaction distance and the interaction frequency can more comprehensively and accurately determine the social status of the student.

[0058] In a specific embodiment, the interaction distance within 0.5m is divided into intimate distance, 0.5m to 1.5m is personal distance, and more than 1.5 meters is public distance; the interaction frequency interval is divided according to the distribution of the number of interactions within 5min to 10min. For example, the number of interactions within 10min is greater than 10 times, which is set as high frequency, the number of interactions within 10min is 5 to 10 times, which is medium frequency, and the number of interactions within 10min is less than 5 times, which is low frequency. Different interaction distance intervals and interaction frequency intervals are combined to construct a social status classification system to determine the historical social status of students in historical video frames. For example, being at an intimate distance and having a high interaction frequency indicates that the student has a very close relationship with others and has a positive historical social status. Being at a medium frequency in a personal distance indicates that ordinary classmates or friends have normal communication and interaction, and the relationship is relatively harmonious. The public distance is low frequency, which means that the student has less communication with others.

[0059] The present invention determines the interaction distance between each student and others in each historical video frame, and determines the social status of each student based on the interaction distance, so as to directly obtain the social behavior data of students in natural scenes and obtain more objective and real social status information. It is helpful to establish a normal social distance mode for each student through long-term monitoring of the interaction distance of students, so as to effectively monitor and warn the mental health of students, and help to further ensure the mental health development of students.

[0060] See also Figure 2 As shown, it is a step diagram of building a dynamic baseline model in an embodiment of the present invention; specifically, in step S2, it includes:

[0061] Step S21, determining a single-frame state combination of each student according to the historical expression state, historical action state and historical social state of each student in each historical video frame;

[0062] Step S22, constructing a corresponding historical state set according to the single-frame state combination of each student;

[0063] Step S23, determining the state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model.

[0064] It is understandable that each historical video frame records the student's state information at a certain moment, including historical expression state, historical action state, and historical social state. These different dimensions of state information together constitute the student's state in the historical video frame, that is, the single-frame state combination. For example, in a certain frame, the historical expression state is smiling, the historical action state is raising hands, and the historical social state is having a heated discussion with the classmates next to them, which constitutes the single-frame state combination of the frame. By calculating the probabilities of all single-frame state combinations, we can understand the probability of various state combinations of students.

[0065] It can be understood that the dynamic baseline model is a quantitative representation of the normal behavior patterns and emotional response range of students. Using the obtained state distribution probability, it can be determined whether the single-frame state combination is a high-probability combination or a low-probability combination. This probability information constitutes the benchmark for judging whether the student's behavior is normal. For example, if a student frequently appears in a single-frame state combination with a very low probability in the historical state set within a certain period of time, it means that the student's behavior is abnormal.

[0066] In a specific embodiment, based on the historical state set, the number of occurrences of each single-frame state combination is counted and then divided by the total number of frames of the student's historical activity video, the occurrence probability of each single-frame state combination, that is, the state distribution probability, can be obtained. For example, in 1000 frames of historical video, the number of times a student appears in the single-frame state combination of "smiling-raising hands-having heated discussions with classmates" is 200, and the state distribution probability corresponding to the single-frame state combination is 0.2.

[0067] The present invention analyzes all historical video frames, collects single-frame state combinations at various moments in the target scene to form historical state sets for each student, determines the state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model, and further determines whether the student's behavior is normal, thereby effectively monitoring and warning the students' mental health, which helps to further ensure the students' mental health development.

[0068] Specifically, in step S2, the behavior sequence of each student in the target scene is determined according to the appearance order of the single-frame state combination of each student in the dynamic baseline model and the corresponding target scene, and the normal behavior pattern of each student is determined according to the behavior sequence.

[0069] It is understandable that different scenes will trigger different behaviors of students. For example, in classroom scenes and playground scenes, students' behavior patterns will be very different. Therefore, the order of single-frame state combinations is sorted out according to the specific target scenes corresponding to the video.

[0070] It is understandable that the order of several single-frame state combinations in the historical video frames constitutes a student's behavior fragment in the target scene, that is, a behavior sequence, and the normal behavior pattern of each student can be determined based on the behavior sequence. For example, in the target scene of campus recess, a student first "stands-smiles-at a distance", then "walks-pats classmates on the shoulder to talk-gets closer", and then "stops-listens-nods". These single-frame state combinations that appear in order constitute a behavior fragment.

[0071] It is understandable that by analyzing a large number of behavioral sequences, we can identify repetitive and regular behavioral patterns and determine them as the normal behavioral patterns of the student in the target scenario. This normal behavioral pattern represents the typical behavior of the student in a specific scenario and is an important basis for judging whether his behavior is normal. If in subsequent monitoring, the student's behavioral sequence deviates significantly from the determined normal behavioral pattern, it may indicate that the student's behavior is abnormal and further attention needs to be paid to his mental health.

[0072] By determining the behavior sequence and normal behavior pattern, the present invention can accurately capture the personalized behavior characteristics of each student, avoid ignoring individual differences, and use a simple assessment method to judge abnormal behavior of all students. Based on the monitoring of behavior sequence and normal behavior pattern, the present invention can more accurately judge the mental health of students, which helps to further ensure the mental health development of students.

[0073] Specifically, in step S2, the historical expression state and historical action state of each student in the normal behavior mode are determined, and the normal emotional response range of each student is determined according to the corresponding relationship between the historical expression state and the historical action state.

[0074] It is understandable that students' facial expressions and action states are often closely related to their emotional states, and different facial expressions and action combinations correspond to specific emotions. After determining the normal behavior pattern, the historical facial expressions and action states presented by students in this behavior pattern are extracted from the historical video frames to facilitate the determination of the normal emotional response range of each student.

[0075] In a specific embodiment, when a student smiles, relaxes, and actively interacts with classmates, it indicates that he is in a pleasant and positive emotional state; and when a student frowns, keeps silent with his head down, and curls up, it indicates that he is in an anxious and depressed emotional state.

[0076] See also Figure 3 As shown, it is a step diagram of determining an abnormal target in an embodiment of the present invention; specifically, in the step S4, it includes:

[0077] Step S41, determining a single-frame real-time state combination of each student according to the real-time expression state, real-time action state and real-time social state of each student to determine a real-time behavior sequence;

[0078] Step S42, determining a behavior matching degree according to the real-time behavior sequence and a normal behavior pattern in a corresponding target scene;

[0079] Step S43, determining the emotion matching degree according to the real-time expression state, the real-time action state and the normal emotional reaction range;

[0080] Step S44, determining abnormal targets according to the behavior matching degree and the emotion matching degree.

[0081] It can be understood that, similar to the analysis of historical states, in the real-time monitoring process, real-time video frames are extracted from the collected real-time activity videos of students to determine the real-time expression state, real-time action state and real-time social state of each student in each frame. These three types of state information constitute the single-frame real-time state combination of the student in the frame. The single-frame real-time state combination in the continuous real-time video frames is arranged in chronological order to form the real-time behavior sequence of the student. This sequence reflects the dynamic change process of the student's behavior in the current target scene. The behavior matching degree is determined by comparing the student's real-time behavior sequence with the normal behavior pattern in the corresponding target scene previously determined based on historical data. If most of the single-frame real-time state combinations in the real-time behavior sequence are consistent with the state combinations in the normal behavior pattern, and the order of appearance of the state combinations is also similar, it means that the behavior matching degree is high; on the contrary, if there is a large difference between the real-time behavior sequence and the normal behavior pattern, such as the appearance of state combinations that rarely or never appear in the normal pattern, the behavior matching degree is low.

[0082] It is understandable that the student's real-time expression and real-time action state indicate his or her current emotional tendency, and the emotional tendency is compared with the previously determined normal emotional response range to determine the emotional matching degree. If the student's real-time emotional tendency falls within the normal emotional response range, it means that the emotional matching degree is high; if it exceeds the normal range, such as under normal circumstances, the student should be calm in this scene, but the real-time state is extremely angry or extremely sad, then the emotional matching degree is low.

[0083] It is understandable that comprehensive consideration of behavioral matching and emotional matching to determine abnormal targets will facilitate the system to issue mental health warnings in a timely manner, reminding relevant personnel (such as teachers, psychological counselors, etc.) to pay attention to the student's mental health status so that timely intervention measures can be taken.

[0084] In a specific embodiment, a variety of similarity calculation methods can be used to measure the degree of behavioral matching between real-time behavior sequences and normal behavior patterns. Common methods include edit distance (Levenshtein distance), dynamic time warping (DTW) algorithm, etc. The extracted facial expressions and action features are combined into a real-time emotion vector, and the student's current emotional state is represented in the form of a vector. The emotional matching degree is determined by calculating the distance between the real-time emotion vector and the normal emotion vector space. Common distance measurement methods include Euclidean distance, Manhattan distance, etc. In implementation, the behavioral matching degree and emotional matching degree can be determined according to actual conditions, which are not specifically limited here and will not be repeated.

[0085] The present invention determines the single-frame real-time state combination and real-time behavior sequence by comprehensively considering the students' real-time expressions, actions, and social status, and further calculates the behavior matching degree and emotion matching degree to determine the abnormal target, breaking the limitation of single-dimensional evaluation, and can monitor students from multiple angles in a three-dimensional manner, accurately capturing the subtle changes in students' status, and greatly improving the accuracy of monitoring students' mental health and behavior status. This further improves the accuracy of students' mental health warning and helps to protect students' mental health.

[0086] Specifically, in step S4, a comprehensive matching degree is determined according to the behavior matching degree and the emotion matching degree, and an abnormal target is determined according to the comprehensive matching degree and the preset matching degree.

[0087] Specifically, the abnormal target is determined according to the comparison result between the comprehensive matching degree and the preset matching degree, wherein if the comprehensive matching degree is less than the preset matching degree, the student is an abnormal target.

[0088] It is understandable that the behavioral match reflects the degree of fit between students' real-time behavior and normal behavior patterns, and the emotional match reflects the fit between real-time emotions and the normal emotional response range. Combining the behavioral match and emotional match can obtain a comprehensive match, which facilitates a more comprehensive and accurate comprehensive evaluation.

[0089] In a specific embodiment, a weighted summation method can be used to calculate the comprehensive matching degree. Certain weights are assigned to the behavioral matching degree and the emotional matching degree respectively. The emotional matching degree corresponds to the emotional weight value range of 0.5 to 0.7, and the behavioral matching degree corresponds to the behavioral weight value range of 0.3 to 0.5. Preferably, the emotional matching degree corresponds to the emotional weight value range of 0.6, and the behavioral matching degree corresponds to the behavioral weight value range of 0.4. The sum of the weights corresponding to the emotional matching degree and the behavioral matching degree is 1. In implementation, the comprehensive matching degree can be determined according to actual conditions, which is not specifically limited here and will not be repeated.

[0090] The present invention determines the comprehensive matching degree according to the behavior matching degree and the emotion matching degree, and determines the abnormal target according to the comprehensive matching degree and the preset matching degree. It is helpful to improve the accuracy and reliability of mental health monitoring. It is further convenient to issue mental health warnings in a timely manner, realize the attention to abnormal targets, and help to protect the mental health of students.

[0091] See also Figure 4 As shown, it is a step diagram for determining the update timing of the dynamic baseline model in an embodiment of the present invention. Specifically, it also includes: analyzing the abnormal targets monitored within a preset time period to determine the abnormal distribution range and the abnormal duration, and determining the update timing of the dynamic baseline model according to the abnormal distribution range and the abnormal duration.

[0092] It is understandable that after discovering an abnormal target, a trend analysis of its abnormal situation within a preset time period can be performed to determine the abnormal distribution range and abnormal duration, and then determine whether the abnormal behavior is due to environmental factors. For example, when the exam week is approaching, some classes or some students show similar abnormal trends, showing more anxious behavior patterns than usual, and this situation persists and has a certain degree of universality. In this case, these abnormal situations can be regarded as new normal behavior ranges that may appear in specific situations, and the relevant data can be included in the historical data set to update the dynamic baseline model to better adapt to the behavioral and emotional changes of students during special periods.

[0093] In a specific embodiment, the value range of the preset time period is 14 days to 30 days, and preferably, the value of the preset time period is 20 days. The abnormal distribution range can be measured by the proportion of the number of students with abnormalities to the total number of monitored students. If the abnormal distribution range is between 15% and 30%, and the abnormal duration is between 1 and 2 weeks, the dynamic baseline model is updated. The expression state, behavioral state and social state corresponding to the abnormal target within the preset time period are brought into the dynamic baseline model for updating. In implementation, the value range and preferred value of the preset time period and the update timing of the dynamic baseline model can be determined according to actual conditions, and are not specifically limited here and will not be repeated.

[0094] The present invention analyzes abnormal targets monitored within a preset time period, determines the abnormal distribution range and abnormal duration, and determines the update timing of the dynamic baseline model accordingly, thereby avoiding misjudgment or missed judgment due to the lag of the dynamic baseline model, improving the reliability of mental health early warning, and avoiding resource waste caused by excessively frequent updates while reasonably determining the update timing to ensure model accuracy, thereby achieving efficient use of resources.

[0095] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for monitoring student mental health education, characterized in that: include: Step S1, extracting historical video frames based on the student historical activity videos collected in the target scene, and determining the historical expression state, historical action state and historical social state of each student in the historical video frames; Step S2, establishing a dynamic baseline model according to the historical expression state, historical action state and historical social state, and determining the normal behavior pattern and normal emotional response range of each student in each target scenario according to the dynamic baseline model; Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state and real-time social state of each student according to the real-time video frames; Step S4, determining abnormal targets and issuing mental health warnings based on the real-time expression status, real-time action status, real-time social status of each student and their corresponding normal behavior patterns and normal emotional response ranges.

2. The student mental health education monitoring method according to claim 1 is characterized in that: In the step S1, the expression degree of the key facial parts of each student in the historical video frame is determined respectively, and the historical expression state of each student in each historical video frame is determined according to the expression degree; Among them, the key facial parts include eyes, mouth and eyebrows, and the expression degree is the degree of eye opening and closing, the distance between eyebrows, the degree of eyebrow raising, and the degree of mouth raising.

3. The student mental health education monitoring method according to claim 2 is characterized in that: In the step S1, the body contours of each student in the historical video frame are determined respectively, the skeleton key points of each student are determined according to the body contours, and the historical action states of each student are determined according to the skeleton key points.

4. The student mental health education monitoring method according to claim 3 is characterized in that: In the step S1, the interaction distance between each student and others in each of the historical video frames is determined, and the social status of each student is determined based on the interaction distance.

5. The student mental health education monitoring method according to claim 4 is characterized in that: In step S2, the single-frame state combination of each student is determined according to the historical expression state, historical action state and historical social state of each student in each historical video frame to construct a historical state set, and the state distribution probability of each student is determined based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model.

6. The student mental health education monitoring method according to claim 5 is characterized in that: In step S2, the behavior sequence of each student in the target scene is determined according to the appearance order of the single-frame state combination of each student in the dynamic baseline model and the corresponding target scene, and the normal behavior pattern of each student is determined according to the behavior sequence.

7. The student mental health education monitoring method according to claim 6 is characterized in that: In step S2, the historical expression state and historical action state of each student in the normal behavior mode are determined, and the normal emotional response range of each student is determined according to the corresponding relationship between the historical expression state and the historical action state.

8. The student mental health education monitoring method according to claim 7 is characterized in that: In step S4, a single-frame real-time state combination of each student is determined according to the real-time expression state, real-time action state and real-time social state of each student to determine a real-time behavior sequence, a behavior matching degree is determined according to the real-time behavior sequence and a normal behavior pattern in a corresponding target scene, an emotion matching degree is determined according to the real-time expression state, real-time action state and a normal emotional response range, and an abnormal target is determined according to the behavior matching degree and the emotion matching degree.

9. The student mental health education monitoring method according to claim 8, characterized in that: In the step S4, a comprehensive matching degree is determined according to the behavior matching degree and the emotion matching degree, and an abnormal target is determined according to the comprehensive matching degree and the preset matching degree.

10. The student mental health education monitoring method according to claim 1, characterized in that: Also includes: The abnormal targets monitored within a preset time period are analyzed to determine the abnormal distribution range and the abnormal duration, and the update timing of the dynamic baseline model is determined according to the abnormal distribution range and the abnormal duration.

Images