Student depression state identification method based on visual behavior analysis

Abstract

The invention relates to the technical field of medicine, in particular to a student depression state recognition method based on visual behavior analysis, which can timely discover the possible depression tendency of students through early warning and provide early intervention opportunities for schools and parents through an abnormal warning mechanism, thereby being helpful for preventing problem deterioration and improving the student depression state recognition efficiency. Through continuous monitoring and analysis of daily behavior data of students, the psychological state of each student can be more accurately known, psychological health resources can be more reasonably allocated and high-risk students can be preferentially concerned based on a data analysis result, and when student video data is collected and processed, a strict privacy protection principle is followed, so that the student video data collection and processing efficiency is improved. The safety and respect of personal information of students are ensured, and the sense of trust of parents and society is enhanced, so that the problem that patients miss treatment opportunities due to the fact that early-stage patients have contradictive psychology to psychologists and do not want to face the doctors is solved.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a method for identifying a student's depressive state based on visual behavior analysis. Background Art

[0003] Computer vision is the science and technology of expressing objects by analyzing images obtained by cameras. Object detection and object tracking are currently one of the research hotspots in this field. Emotion is a state that integrates a person's feelings, thoughts and behaviors. It includes a person's psychological reaction to external or self-stimuli, as well as the physiological reaction that accompanies this psychological reaction. The content of emotion recognition research includes facial expressions, voice and physiological signal recognition, and the above content is used to judge the user's emotional state.

[0004] At present, traditional methods of identifying depressive states, such as clinical assessment and self-assessment of scales, are highly subjective and contain many uncertain factors. In addition, a large number of early patients are resistant to psychologists and believe that they have no symptoms of depression, and are therefore unwilling to face doctors, which causes patients to miss the opportunity for treatment and their condition worsens. Summary of the invention

[0006] The purpose of the present invention is to provide a method for identifying the depressive state of students based on visual behavior analysis, aiming to solve the problem that patients in the early stage have a resistance to psychologists and are unwilling to face doctors, causing patients to miss the opportunity for treatment.

[0007] To achieve the above object, the present invention provides a method for identifying a student's depression state based on visual behavior analysis, comprising the following steps: Collect multiple videos of students’ daily life; Extracting characters from a plurality of daily videos to obtain a plurality of extracted data; Performing posture and expression extraction on the plurality of extracted data to obtain extracted postures and extracted expressions; Matching the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data, and recording the matching times of the plurality of matching data; Calculate an average value based on the number of matches of the plurality of matching data; The average value is compared with the threshold value, and if it exceeds the threshold value, an abnormal alarm is issued; if it does not exceed the threshold value, it is marked and stored.

[0008] Among them, in "collecting multiple daily videos of students", it includes: Use video recording equipment to collect video data of students’ daily activities; The collected video data is stored and named and classified.

[0009] Among them, in “using video recording equipment to collect video data of students’ daily activities”, the content of the video data includes learning scenes, rest scenes and social scenes.

[0010] Among them, “extracting characters from a plurality of daily videos to obtain a plurality of extracted data” includes: Perform frame extraction on the video to decompose the video into individual picture frames; Performing human detection on each of the picture frames using a human detection algorithm based on deep learning to identify a human body region in the image; A character tracking algorithm is used to track the character region of the same character across frames to obtain extracted data.

[0011] Among them, in “extracting posture and expression from the plurality of extracted data to obtain extracted posture and extracted expression”, it includes: Use the posture estimation algorithm to analyze and extract the character image to obtain the posture information of the human body; The facial expression recognition algorithm is used to analyze and extract the facial area of ​​the character to obtain the character's expression.

[0012] Among them, in “matching the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data, and recording the matching times of the plurality of matching data”, it includes: Build a database containing normal and abnormal postures and expressions; Using a similarity measurement method, the extracted posture and the extracted expression are compared with data in a database to obtain matching data; Record the number of matches between each extracted data and the data in the database.

[0013] Among them, in “comparing the average value with the threshold value, if it exceeds, issuing an abnormal alarm, if it does not exceed, marking and storing”, it includes: Set judgment thresholds; The average value is compared with a threshold value. If the average value exceeds the threshold value, it is considered that the student may have abnormal depressive behavior, an abnormal alarm is issued, and relevant personnel are notified to take timely intervention measures; if the average value does not exceed the threshold value, the relevant data is marked and stored in a database for subsequent analysis.

[0014] The present invention discloses a method for identifying students' depressive state based on visual behavior analysis, comprising the following steps: collecting multiple daily videos of students; extracting characters from the multiple daily videos to obtain multiple extracted data; extracting postures and expressions from the multiple extracted data to obtain extracted postures and extracted expressions; matching the extracted postures and the extracted expressions based on data in a database to obtain multiple matching data, and recording the matching times of the multiple matching data; calculating an average value based on the matching times of the multiple matching data; comparing the average value with a threshold value, and issuing an abnormal alarm if the value exceeds the threshold value, and marking and storing the values ​​if the value does not exceed the threshold value. The present invention can promptly detect possible depressive tendencies in students through early warning, and provide schools and parents with opportunities for early intervention through abnormal alarm mechanisms, which helps prevent the problem from worsening. Through continuous monitoring and analysis of students' daily behavior data, it can more accurately understand the psychological state of each student and provide data support for personalized psychological counseling. Based on the results of data analysis, mental health resources can be allocated more reasonably, with priority given to high-risk students, and the efficiency and pertinence of mental health services can be improved. When collecting and processing student video data, strict privacy protection principles are followed to ensure the safety and respect of students' personal information, enhance the trust of parents and society, and thus solve the problem of patients' resistance to psychologists in the early stage and their unwillingness to face doctors, which causes patients to miss the opportunity for treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 The present invention provides a flow chart of a method for identifying a student's depressive state based on visual behavior analysis.

[0018] Figure 2 It is a flowchart for collecting multiple daily videos of students.

[0019] Figure 3 The flowchart is a process of extracting characters from a plurality of daily videos to obtain a plurality of extracted data.

[0020] Figure 4 The flowchart is to extract posture and expression from the plurality of extracted data to obtain extracted posture and extracted expression.

[0021] Figure 5It is a flowchart of matching the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data and record the matching times of the plurality of matching data.

[0022] Figure 6 The flowchart is a process of comparing the average value with a threshold value, and issuing an abnormal alarm if the average value exceeds the threshold value, and marking and storing the abnormal alarm if the average value does not exceed the threshold value. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0025] See also Figures 1 to 6 The present invention provides a method for identifying a student's depression state based on visual behavior analysis, comprising the following steps: S1 collects multiple daily videos of students; S11 used video recording equipment to collect video data of students’ daily activities; The content of the video data includes learning scenes, rest scenes and social scenes.

[0026] Specifically, install hidden video recording equipment that is not easy to attract students' attention in public areas such as classrooms, canteens, and dormitories to ensure that students' daily activities are automatically recorded without infringing on their privacy. The video content must comprehensively cover learning scenes (classes, self-study), rest scenes (lunch breaks, dinners), and social scenes (exchanges between classes, club activities) to capture students' multi-faceted behavioral performance.

[0027] S12 stores the collected video data, and names and classifies the video data.

[0028] Specifically, the recorded video data is classified according to time, location, person and other dimensions, and stored using a unified naming rule to facilitate subsequent data retrieval and processing. At the same time, the security and privacy of video data are ensured, and access rights are restricted to prevent unauthorized personnel from viewing.

[0029] S2 extracts characters from the plurality of daily videos to obtain a plurality of extracted data; S21 extracts frames from the video to decompose the video into individual picture frames; Specifically, video processing software is used to decompose the video file into independent picture frames at time intervals (one frame per second) to facilitate subsequent frame-by-frame analysis.

[0030] S22 uses a deep learning-based person detection algorithm to perform person detection on each of the picture frames to identify a human body area in the image; Specifically, deep learning models such as YOLO5 and SSD are used to quickly and accurately detect people in picture frames. These models can identify the human body area in the image and output the detection results containing the human body bounding box.

[0031] S23 uses a person tracking algorithm to track the person region of the same person across frames to obtain extracted data.

[0032] Specifically, SORT, DeepSORT and other character tracking algorithms are used in combination with the character detection results of the previous frame to continuously track the same person in subsequent frames to ensure the continuity and accuracy of the character information.

[0033] S3 performs posture and expression extraction on the plurality of extracted data to obtain extracted postures and extracted expressions; S31 uses a posture estimation algorithm to analyze and extract the human image to obtain human posture information; Specifically, posture estimation algorithms such as OpenPose and PoseNet are used to detect key points of character images, identify the joint positions of the human body, and infer the character's posture information, such as standing, sitting, walking, etc.

[0034] S32 uses a facial expression recognition algorithm to analyze and extract the facial area of ​​the person to obtain the person's expression.

[0035] Specifically, a facial expression recognition algorithm based on a convolutional neural network (CNN) is used to extract and classify features of a person's facial area, and identify the person's facial expressions, such as happiness, sadness, anger, and daze.

[0036] S4 matches the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data, and records the matching times of the plurality of matching data; S41 builds a database containing normal and abnormal postures and expressions; Specifically, based on the annotations of psychological experts and existing research, a database containing normal and abnormal (depressive expressions) postures and expressions is constructed as a benchmark for subsequent matching.

[0037] S42 uses a similarity measurement method to compare the extracted posture and the extracted expression with data in a database to obtain matching data; Specifically, similarity measurement methods such as Euclidean distance and cosine similarity are used to compare the extracted postures and expressions with the data in the database one by one, and the most similar entries are found to obtain matching data.

[0038] S43 records the number of matches between each extracted data and the data in the database.

[0039] Specifically, for each extracted data, the number of matches with each entry in the database is recorded, especially the number of matches with abnormal postures and expressions, to provide data support for subsequent analysis.

[0040] S5 calculates an average value based on the number of matches of the plurality of matching data; Specifically, for each student's video data, the average number of matches between all extracted data and abnormal data is calculated to quantify how often the student exhibits abnormal postures and expressions.

[0041] S6 compares the average value with the threshold value, and issues an abnormal alarm if the average value exceeds the threshold value; otherwise, the average value is marked and stored.

[0042] S61 sets a judgment threshold; Specifically, based on previous research and the opinions of psychological experts, reasonable thresholds were set to determine whether students’ behavior was abnormal.

[0043] S62 compares the average value with a threshold value. If the average value exceeds the threshold value, it is considered that the student may have abnormal depressive behavior, an abnormal alarm is issued, and relevant personnel are notified to take timely intervention measures. If the average value does not exceed the threshold value, the relevant data is marked and stored in a database for subsequent analysis.

[0044] Specifically, if a student is judged to be abnormal, the system should immediately issue an abnormal alarm and automatically or manually notify the class teacher, psychological counselor and other relevant personnel. Relevant personnel can take timely intervention measures, such as communicating with students and providing psychological counseling services, to avoid patients missing the treatment opportunity due to resistance.

[0045] The method for identifying student depression status based on visual behavior analysis provided by the present invention has the following beneficial effects: 1. The method for identifying student depression based on visual behavior analysis provided by the present invention effectively solves the problem of patients' resistance to psychologists in the early stage, and improves the efficiency and effect of mental health services through automated data analysis and timely intervention mechanism; Second, the method for identifying student depression based on visual behavior analysis provided by the present invention can timely detect the possible depression tendency of students, and provide schools and parents with opportunities for early intervention through abnormal alarm mechanism, which helps to prevent the problem from getting worse; 3. The method for identifying student depression based on visual behavior analysis provided by the present invention can more accurately understand the psychological state of each student through continuous monitoring and analysis of students' daily behavior data, and provide data support for personalized psychological counseling; 4. A method for identifying student depression based on visual behavior analysis provided by the present invention can allocate mental health resources more reasonably based on the results of data analysis, give priority to high-risk students, and improve the efficiency and pertinence of mental health services; 5. The method for identifying student depression status based on visual behavior analysis provided by the present invention follows strict privacy protection principles when collecting and processing student video data, ensures the security and respect of student personal information, and enhances the trust of parents and society.

[0046] What is disclosed above is only a preferred embodiment of the method for identifying student depression status based on visual behavior analysis of the present invention. Of course, it cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for identifying students' depression status based on visual behavior analysis, characterized in that: The following steps are involved: Collect multiple videos of students’ daily life; Extracting characters from a plurality of daily videos to obtain a plurality of extracted data; Performing posture and expression extraction on the plurality of extracted data to obtain extracted postures and extracted expressions; Matching the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data, and recording the matching times of the plurality of matching data; Calculate an average value based on the number of matches of the plurality of matching data; The average value is compared with the threshold value, and if it exceeds the threshold value, an abnormal alarm is issued; if it does not exceed the threshold value, it is marked and stored.

2. The method for identifying student depression status based on visual behavior analysis according to claim 1, characterized in that: In "Collecting multiple students' daily videos", we include: Use video recording equipment to collect video data of students’ daily activities; The collected video data is stored and named and classified.

3. The method for identifying student depression status based on visual behavior analysis according to claim 2, characterized in that: In "Collecting students' daily activity video data using video recording equipment", the content of the video data includes learning scenes, rest scenes and social scenes.

4. The method for identifying student depression status based on visual behavior analysis according to claim 2, characterized in that: In “extracting characters from a plurality of daily videos to obtain a plurality of extracted data”, it includes: Perform frame extraction on the video to decompose the video into individual picture frames; Performing human detection on each of the picture frames using a deep learning-based human detection algorithm to identify human body areas in the image; A character tracking algorithm is used to track the character region of the same character across frames to obtain extracted data.

5. The method for identifying student depression status based on visual behavior analysis according to claim 4, characterized in that: In “extracting posture and expression from the plurality of extracted data to obtain extracted posture and extracted expression”, it includes: Use the posture estimation algorithm to analyze and extract the character image to obtain the posture information of the human body; Use facial expression recognition algorithm to analyze and extract the facial area of ​​the character to obtain the character's expression.

6. The method for identifying student depression status based on visual behavior analysis according to claim 5, characterized in that: In “matching the extracted gesture and the extracted expression based on the data in the database to obtain a plurality of matching data, and recording the number of matches of the plurality of matching data”, it includes: Build a database containing normal and abnormal postures and expressions; Using a similarity measurement method, the extracted posture and the extracted expression are compared with data in a database to obtain matching data; Record the number of matches between each extracted data and the data in the database.

7. The method for identifying student depression status based on visual behavior analysis according to claim 6, characterized in that: In "comparing the average value with the threshold value, if it exceeds, issuing an abnormal alarm, if it does not exceed, marking and storing", it includes: Set judgment thresholds; The average value is compared with a threshold value. If the average value exceeds the threshold value, it is considered that the student may have abnormal depressive behavior, an abnormal alarm is issued, and relevant personnel are notified to take timely intervention measures; if the average value does not exceed the threshold value, the relevant data is marked and stored in a database for subsequent analysis.

Images