A student mental health education monitoring method

Abstract

The present application relates to the technical field of data processing, and more particularly to a student mental health education monitoring method, comprising: extracting historical video frames from a student historical activity video collected under a target scene, determining the historical expression state, action state and social state of each student in the historical video frames to establish a dynamic baseline model, and determining the normal behavior mode and normal emotional response range of each student under each target scene according to the dynamic baseline model; collecting student real-time activity video in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state and real-time social state of each student according to the real-time video frames; determining an abnormal target and performing a mental health warning according to the real-time expression state, real-time action state and real-time social state of each student, and the corresponding normal behavior mode and normal emotional response range of each student. The present application improves the accuracy of student mental health monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a student mental health education monitoring method. BACKGROUND

[0002] Student mental health monitoring is a mental health monitoring work for all students, and is an important basis for carrying out school mental health education work. Students may encounter various psychological problems in the growth process, such as learning pressure, interpersonal relationship problems, family changes, etc., which may cause anxiety, depression, self-esteem and other psychological problems. Through mental health education monitoring, the psychological abnormalities of students can be detected in time, so that appropriate intervention measures can be taken to avoid further deterioration of the problem.

[0003] Chinese patent publication No. CN110729049B discloses a mental health early warning method, which comprises: acquiring a plurality of students' classroom listening audio and video data in different subjects within a preset first time period; wherein the classroom listening audio and video data includes emotional information and body movements; obtaining a first curve of the emotional change of each student over time according to the emotional information of each student; obtaining a second curve of the body movement change of each student over time according to the body movement of each student; judging whether the student has psychological abnormalities according to the first curve and / or the second curve of each student; generating a first warning prompt information when there is a psychological abnormality; and sending the first warning prompt information to a terminal.

[0004] It can be seen that the above technical solution can realize the early warning of student abnormal psychology, achieve the purpose of promoting the healthy development of students and improving the teaching effect, but still has the following problems: using a unified standard to measure student mental health, without considering individual difference interference, and unable to dynamically monitor according to the state of students. SUMMARY

[0005] Therefore, the present application provides a student mental health education monitoring method to overcome the problem that the prior art mental health education monitoring method does not consider individual difference interference and cannot dynamically monitor according to the state of students.

[0006] To achieve the above purpose, the present application provides a student mental health education monitoring method, comprising:

[0007] Step S1, extracting historical video frames from the historical activity video of students collected under the target scene, and determining the historical expression state, historical action state and historical social state of each student in the historical video frames;

[0008] Step S2: establishing a dynamic baseline model based on the historical expression state, historical action state, and historical social state, and determining the normal behavior pattern and normal emotional response range of each student in each target scenario based on the dynamic baseline model;

[0009] Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state, and real-time social state of each student based on the real-time video frames;

[0010] Step S4, determining abnormal targets and issuing mental health warnings based on the real-time expression status, real-time action status, real-time social status of each student and their corresponding normal behavior patterns and normal emotional response ranges.

[0011] Furthermore, in the step S1, the expression level of the key facial parts of each student in the historical video frames is determined respectively, and the historical expression state of each student in each historical video frame is determined according to the expression level;

[0012] Among them, the key facial parts include eyes, mouth and eyebrows, and the expression degree is the degree of eye opening and closing, the distance between eyebrows, the degree of eyebrow raising, and the degree of mouth raising.

[0013] Furthermore, in step S1, the body contours of each student in the historical video frames are determined respectively, the skeletal key points of each student are determined based on the body contours, and the historical action states of each student are determined based on the skeletal key points.

[0014] Furthermore, in the step S1, the interaction distance between each student and others in each of the historical video frames is determined, and the social status of each student is determined based on the interaction distance.

[0015] Furthermore, in step S2, the single-frame state combination of each student is determined according to the historical expression state, historical action state and historical social state of each student in each historical video frame to construct a historical state set, and the state distribution probability of each student is determined based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model.

[0016] Furthermore, in step S2, the behavior sequence of each student in the target scene is determined according to the appearance order of the single-frame state combination of each student in the dynamic baseline model and the corresponding target scene, and the normal behavior pattern of each student is determined according to the behavior sequence.

[0017] Furthermore, in step S2, the historical expression state and historical action state of each student under the normal behavior mode are determined, and the normal emotional response range of each student is determined based on the corresponding relationship between the historical expression state and the historical action state.

[0018] Further, in the step S4, the single-frame real-time state combination of each student is determined according to the real-time expression state, the real-time action state and the real-time social state of each student to determine a real-time behavior sequence, the behavior matching degree is determined according to the real-time behavior sequence and the normal behavior mode under the corresponding target scene, the emotion matching degree is determined according to the real-time expression state, the real-time action state and the normal emotional response range, and the abnormal target is determined according to the behavior matching degree and the emotion matching degree.

[0019] Further, in the step S4, the single-frame real-time state combination of each student is determined according to the real-time expression state, the real-time action state and the real-time social state of each student to determine a real-time behavior sequence, the behavior matching degree is determined according to the real-time behavior sequence and the normal behavior mode under the corresponding target scene, the emotion matching degree is determined according to the real-time expression state, the real-time action state and the normal emotional response range, and the abnormal target is determined according to the behavior matching degree and the emotion matching degree.

[0020] Further, the abnormal distribution range and the abnormal duration are determined by analyzing the abnormal targets monitored in the preset time period, and the updating time of the dynamic baseline model is determined according to the abnormal distribution range and the abnormal duration.

[0021] Compared with the prior art, the beneficial effects of the present application are that the present application comprehensively captures the behavior and emotion information of students, including historical expression state, historical social state and historical behavior state, by analyzing the historical video data collected under the target scene, thereby constructing a targeted individualized dynamic baseline model, improving the accuracy of mental health monitoring. At the same time, combined with real-time monitoring and comparison, the abnormal changes of student behavior and emotion can be found in time, which helps to prevent the occurrence of serious psychological problems and protect the psychological health development of students.

[0022] Further, the present application can determine the expression change rule of students under different scenes by analyzing the performance degree of the key parts of the face of students in a large number of historical video frames, thereby providing an important basis for analyzing the psychological state of students, realizing effective monitoring of the mental health of students, and further helping to protect the psychological health development of students.

[0023] Further, the present application can directly obtain the social behavior data of students in a natural scene by determining the interaction distance of each student with others in each historical video frame and determining the social state of each student based on the interaction distance, thereby obtaining more objective and real social state information. By long-term monitoring of the interaction distance of students, the normal social distance mode of each student can be established, thereby effectively monitoring and warning the mental health of students, and further helping to protect the psychological health development of students.

[0024] Further, the present application collects single-frame state combinations at each moment under the target scene to form a respective historical state set of the student by analyzing all historical video frames, determines the state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model, and further judges the normal benchmark of the student behavior, thereby effectively monitoring and warning the student mental health, and helping to further protect the mental health development of the student.

[0025] Further, the present application can accurately capture the individualized behavior characteristics of each student by determining the behavior sequence and the normal behavior mode, avoid ignoring individual differences, and judge the abnormal behavior of all students using a simple evaluation method. The present application can more accurately judge the mental health of the student based on the monitoring of the behavior sequence and the normal behavior mode, and help to further protect the mental health development of the student.

[0026] Further, the present application determines the single-frame real-time state combination and the real-time behavior sequence by comprehensively considering the real-time expression, action and social state of the student, and further calculates the behavior matching degree and the emotion matching degree to determine the abnormal target, breaks the limitation of single-dimensional evaluation, can stereoscopic monitor the student from multiple angles, accurately captures the subtle changes of the student state, and greatly improves the accuracy of the monitoring of the mental health and behavior state of the student. Thus, the accuracy of the mental health warning of the student is further improved, and the mental health of the student is helped to be protected.

[0027] Further, the present application determines the comprehensive matching degree according to the behavior matching degree and the emotion matching degree, and determines the abnormal target according to the comprehensive matching degree and the preset matching degree. It helps to improve the accuracy and reliability of the mental health monitoring. Further, the mental health warning is conveniently and timely issued, the attention to the abnormal target is realized, and the mental health of the student is helped to be protected.

[0028] Further, the present application analyzes the abnormal target monitored in the preset time period, determines the abnormal distribution range and the abnormal duration, and determines the updating time of the dynamic baseline model accordingly, avoids the misjudgment or omission caused by the lag of the dynamic baseline model, improves the reliability of the mental health warning, reasonably determines the updating time to ensure the accuracy of the model, avoids the resource waste caused by excessive frequent updating, and realizes the efficient use of resources. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The step diagram of the student mental health education monitoring method of the present application;

[0030] Figure 2 The step diagram of constructing the dynamic baseline model in the embodiment of the present application;

[0031] Figure 3 The step diagram of determining the abnormal target in the embodiment of the present application;

[0032] Figure 4 The step diagram for determining the dynamic baseline model updating timing for the embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objects and advantages of the present application clearer, the present application will be further described below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0034] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood by those skilled in the art that the embodiments are only used to explain the technical principles of the present application and are not used to limit the protection scope of the present application.

[0035] It should be noted that, in the description of the present application, the terms of direction or position relationship such as "upper", "lower", "left", "right", "inner", "outer" and the like are based on the direction or position relationship shown in the drawings, which is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0036] Please refer to Figure 1 As shown in the figure, it is a step diagram of the student mental health education monitoring method of the embodiment of the present application. Specifically, the present application provides a student mental health education monitoring method, which comprises:

[0037] Step S1, extracting historical video frames of students based on historical activity videos of students collected under a target scene, determining historical expression state, historical action state and historical social state of each student in the historical video frames;

[0038] Step S2, establishing a dynamic baseline model according to the historical expression state, the historical action state and the historical social state, and determining normal behavior patterns and normal emotional response ranges of each student under each target scene according to the dynamic baseline model;

[0039] Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining real-time expression state, real-time action state and real-time social state of each student according to the real-time video frames;

[0040] Step S4, determining abnormal targets and performing mental health warning according to the real-time expression state, the real-time action state, the real-time social state of each student and the corresponding normal behavior patterns and normal emotional response ranges.

[0041] It can be understood that the target scene collects historical activity video, for example, in a classroom, a playground and the like, and behavior patterns of students in a natural state can be obtained, and the psychological state of the students is expressed through multiple aspects such as expressions, actions and social interactions, so that historical video frames are collected and historical expression states, historical action states and historical social states are analyzed to obtain behavior information of the students from multiple aspects, and the psychological state of the students is conveniently and comprehensively and accurately understood.

[0042] It can be understood that each student has a unique personality, habit and expression mode. By establishing a dynamic baseline model, a normal behavior mode and a normal emotional response range are determined according to historical data of the students, individual differences are fully respected, the timeliness and accuracy of the evaluation standard are ensured, and timely and effective early warning of the psychological health of the students is realized.

[0043] In a specific embodiment, the target scene includes a classroom, a dining hall, a library, a playground and the like, the historical activity video of the students is composed of a series of continuous image frames, a frame rate represents a number of frames displayed per second, and therefore uniform extraction of the historical video frames is performed based on the frame rate. For example, for a video with a frame rate of 30 frames / second, one historical video frame is extracted every 30 frames. In implementation, the target scene can be determined according to actual conditions, and is not specifically limited here, and will not be described again.

[0044] The application comprehensively captures behavior and emotional information of students, including historical expression states, historical social states and historical behavior states, by analyzing historical video data collected in a target scene, to build a targeted individualized dynamic baseline model, and improve the accuracy of psychological health monitoring. In combination with real-time monitoring and comparison, abnormal changes in behavior and emotions of the students can be found in time, which helps to prevent the occurrence of serious psychological problems and ensures the psychological health development of the students.

[0045] Specifically, in the step S1, the performance degree of the facial key part of each student in the historical video frame is determined, and the historical expression state of each student in each historical video frame is determined according to the performance degree.

[0046] The facial key part includes eyes, a mouth and eyebrows, and the performance degree is an opening and closing degree of the eyes, an inter-brow distance, an upward degree of the eyebrows and an upward degree of the mouth.

[0047] It can be understood that the facial expression of a person is an important external manifestation of emotion and psychological state, and the eyes, mouth and eyebrows play a key role in expressing emotion, so these three parts are selected as key parts to analyze the expression state of students. The change of the opening degree of the eyes can reflect emotions such as surprise, excitement, fatigue and thinking. The change of the distance between the eyebrows can reflect the degree of tension or relaxation of emotion. The degree of eyebrow raising can also help to distinguish different emotional tendencies. The greater the degree of mouth raising, the happier and happier. By comprehensively analyzing the performance degree of the eyes, mouth and eyebrows, the expression state of each student in the historical video frame can be determined. For example, if the eyes are wide open, the eyebrows are raised and the mouth is obviously raised, it means that the student is in a happy and excited state; if the eyes are slightly squinted, the forehead is tightly locked and the mouth is slightly turned down, it may indicate that the student is in an unhappy, annoyed or angry state.

[0048] In a specific embodiment, the performance degree of the facial key parts of each student can be determined by using image processing technology and deep learning technology. A facial key point detection model such as Dlib, MTCNN, etc. can be pre-trained, which can accurately detect multiple key points of the face, including the key points of the eyes, eyebrows and mouth. The opening degree of the eyes can be measured by calculating the distance between the upper and lower eyelid key points; the distance between the two eyebrow key points can be obtained to get the distance between the eyebrows; the degree of eyebrow raising can be determined according to the position change of the eyebrow peak key point relative to other eyebrow key points; and the degree of mouth raising can be determined by the vertical position change of the mouth corner key point. In the implementation, the performance degree of the facial key parts of each student can also be determined by image recognition or other deep learning model construction methods, which are not limited here and will not be repeated.

[0049] By analyzing the performance degree of the facial key parts of students in a large number of historical video frames, the expression change rule of students in different scenes can be determined, which provides an important basis for analyzing the psychological state of students, so as to realize the effective monitoring of the psychological health of students and help to further protect the psychological health development of students.

[0050] Specifically, in the step S1, the body contour of each student in the historical video frame is determined respectively, the skeletal key points of each student are determined according to the body contour, and the historical action state of each student is determined according to the skeletal key points.

[0051] It can be understood that the skeleton key points can reflect the relative positions and motion trends of various parts of the student's body. By analyzing the position changes, angle changes and relative relationships between these skeleton key points, the historical action state of the student can be inferred. For example, when the relative positions of the elbow skeleton key point and the shoulder and wrist skeleton key points change in a specific manner, and the arm skeleton forms a certain angle, it can indicate that the student is making a hand-raising action. If the hip, knee and ankle skeleton key points exhibit specific motion trajectories and angle changes, it can indicate that the student is walking, running or jumping. By dynamically analyzing the skeleton key points in consecutive video frames, the specific actions of the student at different times can be accurately determined, and the behavior patterns and activity conditions of the student in the target scene can be understood.

[0052] In a specific embodiment, the body contours of each student in the historical video frames are determined using color space differences, edge detection algorithms, etc. If the background in the student historical activity video has a significant difference in color from the student's body, a color threshold is set, and the body contour is extracted. Alternatively, an edge detection algorithm is used to determine the body contour of the student. The skeleton key points are determined based on the obtained body contour. The range of the skeleton key points includes the shoulder, elbow, wrist, hip, knee and ankle, etc. A human pose estimation method based on deep learning can be used, the historical video frame image containing the student's body contour is input, and the model will determine the position of the student's skeleton key points in the historical video frame according to the learned human structure features. In implementation, a traditional feature-based method can also be used to locate the skeleton key points according to the shape, proportion and other features of the body contour, combined with some prior knowledge, which is not specifically limited here and will not be described again.

[0053] Specifically, in the step S1, the interaction distance of each student from others in each historical video frame is determined, and the social state of each student is determined based on the interaction distance.

[0054] In one specific embodiment, different interaction distances reflect different social relationships and interaction natures. The interaction distance is the spatial distance between each student and other students in the historical video frames. The interaction distance can be determined by measuring the pixel distance between the key points of the students' bodies (such as the head, the center of the torso, etc.), and then converting the actual physical distance according to the actual scene and scale of the video shooting. When the distance between students is close, within 0.5 m, it indicates an intimate distance, which may be an intimate interaction, cooperation or play; when the distance is between 0.5 m and 1.5 m, it is a personal distance, which is suitable for general conversation, discussion and other interaction scenarios. When the distance is 1.5 m, it is a public distance, which means that the interaction is relatively less and the social relationship is relatively distant, or in a non-interactive state, such as on the playground, some students are active in different positions, and the distance between them will be farther. In the implementation, the interaction distance is determined according to the actual situation, which is not limited here and will not be repeated.

[0055] In one specific embodiment, the interaction frequency of each student with others can also be determined according to the historical video frames, and the social state of each student can be determined according to the interaction distance and the interaction frequency.

[0056] It can be understood that the interaction frequency refers to the number of interactions between a student and other students within a certain time range. When analyzing the historical video frames, the interaction frequency can be determined by observing the behavior changes of the students in the continuous video frames. A higher interaction frequency usually indicates that the student is more actively involved in social activities, has a higher social activity, has a stronger social willingness and social ability, and vice versa. The student is relatively introverted, passive, or has certain obstacles in social interaction, and the interaction frequency is low, and the student does not actively communicate and interact with others.

[0057] In one specific embodiment, the combination of the interaction distance and the interaction frequency can more comprehensively and accurately determine the social state of the student.

[0058] In one specific embodiment, the interaction distance within 0.5 m is divided into an intimate distance, 0.5 m to 1.5 m is personal distance, and more than 1.5 meters is public distance; according to the distribution of the number of interactions within 5 min to 10 min, the interaction frequency interval is divided. For example, if the number of interactions within 10 min is more than 10 times, it is set as high frequency, the number of interactions within 10 min is 5 to 10 times as medium frequency, and the number of interactions within 10 min is less than 5 times as low frequency. Different interaction distance intervals and interaction frequency intervals are combined to construct a social state classification system to determine the historical social state of the student in the historical video frame. For example, being in an intimate distance and having a high interaction frequency indicates that the student has a very close relationship with others, and the historical social state is positive. Being in a personal distance and having a medium frequency indicates that the student has normal communication and interaction with ordinary classmates or friends, and the relationship is relatively harmonious. Being in a public distance and having a low frequency indicates that the student has less communication with others.

[0059] The application can directly obtain the social behavior data of the students in the natural scene by determining the interaction distance of each student with others in each historical video frame, determining the social state of each student based on the interaction distance, and obtaining more objective and real social state information. It is helpful to establish the normal social distance mode of each student by long-term monitoring of the interaction distance of the students, thereby effectively monitoring and warning the mental health of the students, and further helping to protect the mental health development of the students.

[0060] Please refer to Figure 2 The step S2 includes the following steps.

[0061] Step S21, determining a single-frame state combination of each student according to a historical expression state, a historical action state and a historical social state of each student in each historical video frame;

[0062] Step S22, constructing a corresponding historical state set according to the single-frame state combination of each student;

[0063] Step S23, determining a state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model.

[0064] It can be understood that each historical video frame records the state information of the student at a certain moment, including the historical expression state, the historical action state and the historical social state. These different dimension state information together constitute the state of the student in the historical video frame, that is, the single-frame state combination. For example, in a certain frame, the historical expression state is smiling, the historical action state is raising hands, and the historical social state is discussing with the classmate next to him, which constitutes the single-frame state combination of the frame. By calculating the probability of all single-frame state combinations, the possibility of the occurrence of various state combinations of the student can be understood.

[0065] It can be understood that the dynamic baseline model is a quantitative representation of the normal behavior pattern and emotional response range of the student. Using the obtained state distribution probability, it can be determined that a single-frame state combination is a high-probability combination or a low-probability combination. These probability information constitutes a baseline for judging whether the behavior of the student is normal. For example, if a student frequently appears a single-frame state combination with a very low probability in the historical state set within a certain period of time, it means that the student's behavior is abnormal.

[0066] In a specific embodiment, based on the historical state set, the number of occurrences of each single-frame state combination is counted, and then divided by the total number of frames of the student's historical activity video to obtain the occurrence probability of each single-frame state combination, i.e. the state distribution probability. For example, in 1000 frames of historical video, the student appears a single-frame state combination of “smiling- raising hands-having a heated discussion with classmates” 200 times, and the state distribution probability corresponding to the single-frame state combination is 0.2.

[0067] The present application collects the single-frame state combinations of each student at each time in the target scene by analyzing all historical video frames to form the historical state set of each student, determines the state distribution probability of each student based on the historical state set and the single-frame state combination to construct the corresponding dynamic baseline model, and further judges the normal baseline of the student behavior, thereby effectively monitoring and warning the mental health of the student, and helping to further protect the mental health development of the student.

[0068] Specifically, in the step S2, the behavior sequence of each student in the target scene is determined according to the order of occurrence of the single-frame state combination of each student in the dynamic baseline model and the corresponding target scene, and the normal behavior pattern of each student is determined according to the behavior sequence.

[0069] It can be understood that different scenes will trigger different behavior of the student. For example, in the classroom scene and the playground scene, the behavior pattern of the student will be very different. Therefore, the order of the single-frame state combination is sorted according to the specific target scene corresponding to the video.

[0070] It can be understood that the order of a plurality of single-frame state combinations in the historical video frames constitutes a behavior segment of the student in the target scene, i.e. a behavior sequence, and the normal behavior pattern of each student can be determined according to the behavior sequence. For example, in the target scene of the schoolyard break, a student first appears “standing-smiling-distancing”, then appears “walking-patting the shoulder of a classmate to talk-distancing”, and then appears “stopping-listening-nodding”, and these single-frame state combinations appearing in the order constitute a behavior segment.

[0071] It can be understood that by analyzing a large number of behavior sequences, a repetitive and regular behavior pattern can be determined as the normal behavior pattern of the student in the target scene. The normal behavior pattern represents the typical behavior of the student in a specific scene and is an important basis for judging whether the behavior is normal. If the behavior sequence of the student deviates greatly from the determined normal behavior pattern in subsequent monitoring, it may imply that the student's behavior is abnormal, and further attention should be paid to the mental health of the student.

[0072] The present application can accurately capture the individualized behavior characteristics of each student by determining the behavior sequence and the normal behavior pattern, avoid ignoring individual differences, and judge abnormal behavior using a simple evaluation method for all students. The monitoring based on the behavior sequence and the normal behavior pattern can more accurately judge the mental health of the students and help further protect the mental health development of the students.

[0073] Specifically, in the step S2, the historical expression state and the historical action state of each student in the normal behavior pattern are determined, and the normal emotional response range of each student is determined according to the correspondence between the historical expression state and the historical action state.

[0074] It can be understood that the expression state and the action state of the student are often closely related to their emotional state, and different expression and action combinations correspond to specific emotions. After determining the normal behavior pattern, the historical expression state and the historical action state of the student presented in the historical video frame under this behavior pattern are extracted to facilitate the determination of the normal emotional response range of each student.

[0075] In a specific embodiment, when the student has a smile, a relaxed body, and actively interacts with classmates, it indicates a happy and positive emotional state; when the student has a frown, a low head, a silent body, and a curled body, it indicates an anxious and depressed emotional state.

[0076] Referring to Figure 3 As shown in FIG. 4, which is a step diagram for determining an abnormal target according to an embodiment of the present application; specifically, in the step S4, it includes:

[0077] Step S41, determining a single-frame real-time state combination of each student according to the real-time expression state, the real-time action state, and the real-time social state of each student to determine a real-time behavior sequence;

[0078] Step S42, determining a behavior matching degree according to the real-time behavior sequence and the normal behavior pattern in the corresponding target scene;

[0079] Step S43, determining an emotional matching degree according to the real-time expression state, the real-time action state, and the normal emotional response range;

[0080] Step S44, determining the abnormal target according to the behavior matching degree and the emotion matching degree.

[0081] It can be understood that, similar to the analysis of the historical state, in the real-time monitoring process, the real-time video frames of the collected student real-time activities are extracted, the real-time expression state, the real-time action state and the real-time social state of each student in each frame are determined. The three kinds of state information constitute the single-frame real-time state combination of the student in the frame. The single-frame real-time state combinations in the continuous real-time video frames are arranged in time sequence, and the real-time behavior sequence of the student is formed. This sequence reflects the dynamic change process of the student's behavior in the current target scene. The behavior matching degree is determined by comparing the real-time behavior sequence of the student with the normal behavior mode in the corresponding target scene determined according to the historical data. If most of the single-frame real-time state combinations in the real-time behavior sequence are consistent with the state combinations in the normal behavior mode, and the order of the appearance of the state combinations is also similar, it means that the behavior matching degree is high; on the contrary, if the real-time behavior sequence and the normal behavior mode have great differences, such as the appearance of state combinations that rarely or never appear in the normal mode, the behavior matching degree is low.

[0082] It can be understood that the real-time expression state and the real-time action state of the student indicate the current emotional tendency, and the emotional tendency is compared with the previously determined normal emotional response range to determine the emotional matching degree. If the real-time emotional tendency of the student falls within the normal emotional response range, it means that the emotional matching degree is high; if it exceeds the normal range, such as the student should show calmness in the scene under normal circumstances, but the real-time state is extreme anger or extreme sadness, then the emotional matching degree is low.

[0083] It can be understood that the behavior matching degree and the emotion matching degree are comprehensively considered to determine the abnormal target, which facilitates the system to timely issue a mental health warning and reminds relevant personnel (such as teachers, psychological counselors, etc.) to pay attention to the mental health status of the student, so as to take timely intervention measures.

[0084] In a specific embodiment, a variety of similarity calculation methods can be used to measure the behavior matching degree between the real-time behavior sequence and the normal behavior mode, common methods include Levenshtein distance, dynamic time warping (DTW) algorithm, etc. The extracted expression and action features are combined into a real-time emotion vector, and the current emotional state of the student is represented in the form of a vector. The emotional matching degree is determined by calculating the distance between the real-time emotion vector and the normal emotion vector space, and common distance measurement methods include Euclidean distance, Manhattan distance, etc. In the implementation, the behavior matching degree and the emotion matching degree can be determined according to the actual situation, which is not limited here, and will not be described again.

[0085] The application determines a single-frame real-time state combination and a real-time behavior sequence by comprehensively considering the real-time expression, action and social state of the student, and further calculates a behavior matching degree and an emotion matching degree to determine an abnormal target, breaking the limitation of single-dimensional evaluation, enabling stereoscopic monitoring of the student from multiple angles, accurately capturing the subtle changes of the student state, and greatly improving the accuracy of monitoring the mental health and behavior state of the student.

[0086] Specifically, in the step S4, a comprehensive matching degree is determined according to the behavior matching degree and the emotion matching degree, and an abnormal target is determined according to the comprehensive matching degree and a preset matching degree.

[0087] Specifically, the abnormal target is determined according to the comparison result of the comprehensive matching degree and the preset matching degree, wherein if the comprehensive matching degree is less than the preset matching degree, the student is the abnormal target.

[0088] It can be understood that the behavior matching degree reflects the degree of fit between the real-time behavior of the student and the normal behavior mode, and the emotion matching degree reflects the degree of fit between the real-time emotion and the normal emotional response range. The behavior matching degree and the emotion matching degree are combined to obtain the comprehensive matching degree, which is convenient for more comprehensive and accurate comprehensive evaluation.

[0089] In a specific embodiment, a weighted summation method can be used to calculate the comprehensive matching degree. The behavior matching degree and the emotion matching degree are respectively given a certain weight, the emotion matching degree corresponds to an emotion weight value range of 0.5-0.7, the behavior matching degree corresponds to a behavior weight value range of 0.3-0.5, preferably, the emotion matching degree corresponds to an emotion weight value range of 0.6, the behavior matching degree corresponds to a behavior weight value range of 0.4, and the sum of the weights corresponding to the emotion matching degree and the behavior matching degree is 1. In the implementation, the comprehensive matching degree can be determined according to the actual situation, which is not specifically limited here and will not be described again.

[0090] The application determines a comprehensive matching degree according to a behavior matching degree and an emotion matching degree, and determines an abnormal target according to the comprehensive matching degree and a preset matching degree. This is helpful to improve the accuracy and reliability of mental health monitoring, further facilitate timely mental health warning, realize attention to the abnormal target, and help to protect the mental health of students.

[0091] Please refer to Figure 4 As shown in the figure, it is a step diagram for determining the updating time of the dynamic baseline model according to an embodiment of the application, and specifically further comprises: analyzing the abnormal target monitored in a preset time period to determine an abnormal distribution range and an abnormal duration, and determining the updating time of the dynamic baseline model according to the abnormal distribution range and the abnormal duration.

[0092] It can be understood that after the abnormal target is found, trend analysis can be performed on the abnormal situation of the abnormal target in a preset time period, so as to determine the abnormal distribution range and the abnormal duration, and then determine whether the abnormal behavior is due to environmental factors. For example, when the examination week is approaching, some classes or some students show similar abnormal trends, and all show more anxious behavior patterns than usual, and this situation lasts and has a certain universality, so these abnormal situations can be regarded as new normal behavior range that may appear in a specific situation. The related data is included in the historical data set, and the dynamic baseline model is updated to better adapt to the behavior and emotion changes of students in special period.

[0093] In a specific embodiment, the preset time period ranges from 14 days to 30 days, and preferably, the preset time period is 20 days. The proportion of the number of students appearing abnormal to the total number of monitored students can be used to measure the abnormal distribution range. If the abnormal distribution range is between 15% and 30%, and the abnormal duration is 1-2 weeks, the dynamic baseline model is updated. The expression state, behavior state and social state corresponding to the abnormal target in the preset time period are brought into the dynamic baseline model for updating. In the implementation, the value range and the preferred value of the preset time period, and the updating time of the dynamic baseline model can be determined according to the actual situation, which is not limited here and will not be described again.

[0094] The present application analyzes the abnormal target monitored in the preset time period, determines the abnormal distribution range and the abnormal duration, and determines the updating time of the dynamic baseline model accordingly, avoids misjudgment or omission caused by the lag of the dynamic baseline model, improves the reliability of the mental health warning, and avoids the resource waste caused by excessive frequent updating while ensuring the accuracy of the model by reasonably determining the updating time, and realizes the efficient use of resources.

[0095] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without deviating from the principles of the present application, those skilled in the art can make equivalent changes or replacements to related technical features, and the technical solutions after the changes or replacements will fall within the protection scope of the present application.

Claims

1. A student mental health education monitoring method, characterized in that: include: Step S1, extracting historical video frames based on student historical activity videos collected in a target scene, and determining the historical expression state, historical action state, and historical social state of each student in the historical video frames, including respectively determining the expression degree of each student's facial key parts in the historical video frames, and determining the historical expression state of each student in each historical video frame according to the expression degree; respectively determining the body contour of each student in the historical video frames, determining the skeletal key points of each student according to the body contour, and determining the historical action state of each student according to the skeletal key points; determining the interaction distance between each student and others in each historical video frame, and determining the social state of each student based on the interaction distance; Step S2: determining the single-frame state combination of each student based on the historical expression state, historical action state, and historical social state to construct a historical state set; determining the state distribution probability of each student based on the historical state set and the single-frame state combination to construct a corresponding dynamic baseline model; determining the behavior sequence of each student in the target scenario based on the appearance order of the single-frame state combination of each student and the corresponding target scenario in the dynamic baseline model to determine the normal behavior pattern and normal emotional response range of each student in each target scenario; Step S3, collecting real-time activity videos of students in the target scene to extract real-time video frames, and determining the real-time expression state, real-time action state, and real-time social state of each student based on the real-time video frames; Step S4, determining abnormal targets and performing mental health warnings based on the real-time expression status, real-time action status, real-time social status, and corresponding normal behavior patterns and normal emotional response ranges of each student, wherein the single-frame real-time state combination of each student is determined based on the real-time expression status, real-time action status, and real-time social status of each student to determine the real-time behavior sequence, determining the behavior matching degree based on the real-time behavior sequence and the normal behavior pattern in the corresponding target scene, determining the emotion matching degree based on the real-time expression status, real-time action status, and normal emotional response range, and determining abnormal targets based on the behavior matching degree and the emotion matching degree, wherein the behavior matching degree is determined by comparing the student's real-time behavior sequence with the normal behavior pattern in the corresponding target scene determined based on historical data.

2. The student mental health education monitoring method according to claim 1, characterized in that: In the step S1, The key facial parts include eyes, mouth and eyebrows, and the expression degree is the degree of eye opening and closing, the distance between eyebrows, the degree of eyebrow raising, and the degree of mouth raising.

3. The student mental health education monitoring method according to claim 2, characterized in that: In step S2, the historical expression state and historical action state of each student under the normal behavior mode are determined, and the normal emotional response range of each student is determined based on the corresponding relationship between the historical expression state and the historical action state.

4. The student mental health education monitoring method according to claim 3, characterized in that: In step S4, a comprehensive matching degree is determined based on the behavior matching degree and the emotion matching degree, and an abnormal target is determined based on the comprehensive matching degree and the preset matching degree.

5. The student mental health education monitoring method according to claim 1, characterized in that: Also includes: The abnormal targets monitored within a preset time period are analyzed to determine the abnormal distribution range and abnormal duration, and the update timing of the dynamic baseline model is determined according to the abnormal distribution range and abnormal duration.

Images