Emotion recognition system and method based on AI recognition

Abstract

The invention relates to the technical field of mental health monitoring, in particular to an emotion recognition system and method based on AI recognition. Comprising a cloud management platform, an information input module, a camera acquisition module, a preprocessing module, a data analysis module, a psychological test module, an early warning push module, a psychological health reservation module, a teacher login module and a student login module, face and neck muscle micro-vibration data of students in the campus are captured by using the camera acquisition module, the data analysis module converts the face and neck muscle micro-vibration data of the students into three-dimensional motion trails, the three-dimensional motion trails are compared and analyzed with an emotion model database, and the early warning push module performs risk assessment according to emotion results of the students. And early warning information is pushed to related teachers and school leaders in real time, so that the emotional states of the students can be analyzed and detected in real time, and powerful technical support is provided for mental health management of the students.

Description

Technical Field

[0001] The present invention relates to the technical field of mental health monitoring, and in particular to an emotion recognition system and method based on AI recognition. Background Art

[0003] Emotions are an important part of human spiritual life and have a significant impact on people's behavior and decision-making. Accurately identifying and understanding the emotional state of others is of great significance to fields such as psychological counseling, treatment, social interaction, and human-computer interaction. However, people's emotional states are often affected by multiple factors, including physiological, psychological, social, and cultural factors. Therefore, emotion recognition is a complex and challenging task.

[0004] At present, with the increasing complexity of the educational environment and the increasingly prominent mental health problems of students, schools will regularly offer mental health lectures and mental health tests to guide students' mental health.

[0005] However, existing schools lack real-time monitoring and early warning of students' emotional states, and it is difficult to meet the needs of rapid, comprehensive and accurate student mental health testing, and they cannot provide timely mental health support and intervention to students. Summary of the invention

[0007] The purpose of the present invention is to provide an emotion recognition system and method based on AI recognition, which aims to solve the technical problems that existing schools lack real-time monitoring and early warning of students' emotional states, are difficult to meet the needs of fast, comprehensive and accurate testing of students' mental health, and cannot provide timely mental health support and intervention to students.

[0008] To achieve the above-mentioned purpose, the present invention adopts an emotion recognition system based on AI recognition, including a cloud management platform, an information entry module, a video acquisition module, a preprocessing module, a data analysis module, a psychological testing module, an early warning push module, a mental health appointment module, a teacher login module and a student login module, wherein the information entry module, the data analysis module, the psychological testing module, the mental health appointment module and the early warning push module are all connected to the cloud management platform, the preprocessing module is connected to the data analysis module, the video acquisition module is connected to the preprocessing module, and the teacher login module and the student login module are both connected to the cloud management platform; The cloud management platform is used to manage students’ mental health status, monitor, analyze and predict students’ mental health status in real time; The information entry module is used to establish mental health files for all students in the school; The camera acquisition module is used to capture facial and neck muscle micro-vibration data of middle school students on campus; The preprocessing module is used to preprocess the captured video data; The data analysis module is used to convert the micro-vibration data of the students' facial and neck muscles into three-dimensional motion trajectories, and compare and analyze them with the emotion model database; The psychological testing module is used to test the mental health status of students, so that teachers and parents can intuitively understand the mental health status of the monitored subjects; The warning push module is used to conduct risk assessment based on the students' emotional results and push warning information to relevant teachers and school leaders in real time; The mental health appointment module is used to provide students with a convenient mental health service appointment channel to help them obtain professional psychological support and intervention in a timely manner; The teacher login module is used for teachers to log in to the cloud management platform; The student login module is used for students to log in to the cloud management platform.

[0009] Wherein, the preprocessing module includes a data cleaning unit, a data denoising unit and a data color enhancement unit, and the data cleaning unit, the data denoising unit and the data color enhancement unit are all connected to the data acquisition module; The data cleaning unit is used to remove redundant and irrelevant information in the video data; The data denoising unit is used to remove noise from video data; The data color enhancement unit is used to adjust the color balance, contrast and brightness of the video data.

[0010] Wherein, the data analysis module includes a pedestrian recognition unit, a face recognition unit, a morphological analysis unit and a model comparison unit, and the pedestrian recognition unit, the face recognition unit, the morphological analysis unit and the model comparison unit are all connected to the preprocessing module; The pedestrian recognition unit is used to detect and recognize pedestrians in the picture in real time; The face recognition unit is used to obtain facial expression changes of pedestrians; The morphological analysis unit is used to analyze the posture and movement of pedestrians, compare with normal behavior patterns, and identify possible abnormal behavior; The model comparison unit is used to put the data into the model for emotion recognition and predict the complex emotional state of pedestrians.

[0011] Wherein, the psychological test module includes an evaluation tool selection unit, a test analysis unit, a report generation unit and a report feedback unit, and the evaluation tool selection unit, the test analysis unit, the report generation unit and the report feedback unit are all connected to the cloud management platform; The assessment tool selection unit is used to select appropriate psychological assessment scales and questionnaires according to the assessment purpose and student group characteristics; The test analysis unit is used to perform emotional and psychological analysis based on the student's assessment results; The report generating unit is used to generate a student psychology report according to the analysis results; The report feedback unit is used to feed back the student psychology report to the student, parents and teachers.

[0012] The warning push module includes a risk assessment unit, a warning information generation unit and a push execution unit, and the risk assessment unit, the warning information generation unit and the push execution unit are all connected to the cloud management platform; The risk assessment unit is used to assess the student's mental health risk and set a warning threshold based on the emotion recognition results and behavior data analysis; The warning information generation unit is used to determine whether the risk assessment result exceeds a preset threshold and automatically generate warning information; The push execution unit is used to push the warning information to teachers and parents in real time.

[0013] Wherein, the mental health reservation module includes a service display unit and a reservation application unit, and both the service display unit and the reservation application unit are connected to the cloud management platform; The service display unit is used to list the mental health services provided to students; The reservation application unit is used for students to make reservations and select corresponding health service types in the mental health service list.

[0014] The present invention also provides an emotion recognition method based on AI recognition, comprising the following steps: Clean and preprocess the collected data; Using computer vision techniques, we extract emotion-related features from pre-processed facial images, as well as body posture, gesture, and action features from videos; Extract emotion-related acoustic features through speech analysis technology to determine the speaker's emotional state; Select appropriate deep learning models based on task requirements and data characteristics; Use the labeled dataset to train the model; Deploy the trained model to actual application scenarios; Capturing the user's facial expression through the camera acquisition module and the user's voice through the microphone; The extracted features are input into the model for emotion recognition, and the recognition results are fed back immediately.

[0015] The present invention discloses an emotion recognition system and method based on AI recognition. The cloud management platform manages the mental health status of students, monitors, analyzes and predicts the mental health status of students in real time. The information entry module establishes a mental health file for all students in the school, uses the video acquisition module to capture the micro-vibration data of the face and neck muscles of the students on campus, the preprocessing module preprocesses the captured video data, and the data analysis module converts the micro-vibration data of the face and neck muscles of the students into a three-dimensional motion trajectory, and compares and analyzes it with the emotion model database. Students test their mental health status through the psychological testing module, which is convenient for teachers and parents to intuitively understand the mental health status of the monitored objects. The early warning push module conducts risk assessment according to the results of the students' emotions, and pushes early warning information to relevant teachers and school leaders in real time. The mental health reservation module provides students with a convenient mental health service reservation channel to help them obtain professional psychological support and intervention in time. Teachers log in to the cloud management platform through the teacher login module, and students log in to the cloud management platform through the student login module. Through the above method, it is realized that the emotional state of students can be analyzed and detected in real time, providing strong technical support for students' mental health management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the principle of the emotion recognition system based on AI recognition of the present invention.

[0019] Figure 2 It is a flowchart of the steps of the emotion recognition method based on AI recognition of the present invention.

[0020] 101- cloud management platform, 102- information entry module, 103- camera acquisition module, 104- data cleaning unit, 105- data denoising unit, 106- data color enhancement unit, 107- pedestrian recognition unit, 108- face recognition unit, 109- morphological analysis unit, 110- model comparison unit, 111- assessment tool selection unit, 112- test analysis unit, 113- report generation unit, 114- report feedback unit, 115- risk assessment unit, 116- warning information generation unit, 117- push execution unit, 118- service display unit, 119- appointment application unit, 120- teacher login module, 121- student login module, 122- course tutoring unit, 123- emotional relaxation unit, 124- stress training unit. DETAILED DESCRIPTION

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0023] See also Figure 1 and Figure 2 The present invention provides an emotion recognition system based on AI recognition, including a cloud management platform 101, an information entry module 102, a video acquisition module 103, a preprocessing module, a data analysis module, a psychological test module, an early warning push module, a mental health appointment module, a teacher login module 120 and a student login module 121, wherein the information entry module 102, the data analysis module, the psychological test module, the mental health appointment module and the early warning push module are all connected to the cloud management platform 101, the preprocessing module is connected to the data analysis module, the video acquisition module 103 is connected to the preprocessing module, and the teacher login module 120 and the student login module 121 are both connected to the cloud management platform 101; The cloud management platform 101 is used to manage the mental health status of students, and monitor, analyze and predict the mental health status of students in real time; The information entry module 102 is used to establish a mental health file for all students in the school; The camera acquisition module 103 is used to capture facial and neck muscle micro-vibration data of campus students; The preprocessing module is used to preprocess the captured video data; The data analysis module is used to convert the micro-vibration data of the students' facial and neck muscles into three-dimensional motion trajectories, and compare and analyze them with the emotion model database; The psychological testing module is used to test the mental health status of students, so that teachers and parents can intuitively understand the mental health status of the monitored subjects; The warning push module is used to conduct risk assessment based on the students' emotional results and push warning information to relevant teachers and school leaders in real time; The mental health appointment module is used to provide students with a convenient mental health service appointment channel to help them obtain professional psychological support and intervention in a timely manner; The teacher login module 120 is used for teachers to log in to the cloud management platform 101; The student login module 121 is used for students to log in to the cloud management platform 101 .

[0024] In this embodiment, the cloud management platform 101 manages the mental health status of students, monitors, analyzes and predicts the mental health status of students in real time, the information entry module 102 establishes a mental health file for all students in the school, uses the camera acquisition module 103 to capture the facial and neck muscle micro-vibration data of students on campus, the preprocessing module preprocesses the captured video data, the data analysis module converts the micro-vibration data of the students' facial and neck muscles into three-dimensional motion trajectories, and compares and analyzes them with the emotional model database, and students test their mental health status through the psychological testing module, which is convenient for teachers and parents to intuitively understand the mental health status of the monitored objects. The early warning push module conducts risk assessment based on the students' emotional results and pushes early warning information to relevant teachers and school leaders in real time. The mental health appointment module provides students with a convenient mental health service appointment channel to help them obtain professional psychological support and intervention in a timely manner. Teachers log in to the cloud management platform 101 through the teacher login module 120, and students log in to the cloud management platform 101 through the student login module 121.

[0025] Further, the pre-processing module includes a data cleaning unit 104, a data denoising unit 105 and a data color enhancement unit 106, and the data cleaning unit 104, the data denoising unit 105 and the data color enhancement unit 106 are all connected to the data acquisition module; The data cleaning unit 104 is used to remove redundant and irrelevant information in the video data; The data denoising unit 105 is used to remove noise from the video data; The data color enhancement unit 106 is used to adjust the color balance, contrast and brightness of the video data.

[0026] In this embodiment, the data cleaning unit 104 is used to remove redundant and irrelevant information in the video data to reduce the amount of data and improve the efficiency of subsequent processing. The data denoising unit 105 is used to remove noise in the video data to improve the clarity and signal-to-noise ratio of the video. The data color enhancement unit 106 is used to adjust the color balance, contrast and brightness of the video data to ensure the color accuracy of the video.

[0027] Further, the data analysis module includes a pedestrian recognition unit 107, a face recognition unit 108, a morphological analysis unit 109 and a model comparison unit 110, and the pedestrian recognition unit 107, the face recognition unit 108, the morphological analysis unit 109 and the model comparison unit 110 are all connected to the preprocessing module; The pedestrian recognition unit 107 is used to detect and recognize pedestrians in the picture in real time; The face recognition unit 108 is used to obtain facial expression changes of pedestrians; The morphological analysis unit 109 is used to analyze the posture and movement of pedestrians, compare with normal behavior patterns, and identify possible abnormal behaviors; The model comparison unit 110 is used to put the data into the model for emotion recognition and predict the complex emotional state of pedestrians.

[0028] In this embodiment, the pedestrian recognition unit 107 is used to detect and identify pedestrians in the picture in real time, the face recognition unit 108 is used to obtain changes in facial expressions of pedestrians, the morphological analysis unit 109 is used to analyze the posture and movements of pedestrians, compare with normal behavior patterns, and identify possible abnormal behaviors. The model comparison unit 110 puts the data into the model for emotion recognition to predict the complex emotional state of pedestrians.

[0029] Further, the psychological test module includes an evaluation tool selection unit 111, a test analysis unit 112, a report generation unit 113 and a report feedback unit 114, and the evaluation tool selection unit 111, the test analysis unit 112, the report generation unit 113 and the report feedback unit 114 are all connected to the cloud management platform 101; The evaluation tool selection unit 111 is used to select appropriate psychological evaluation scales and questionnaires according to the evaluation purpose and characteristics of the student group; The test analysis unit 112 is used to perform emotional and psychological analysis based on the student's assessment results; The report generating unit 113 is used to generate a student psychology report according to the analysis results; The report feedback unit 114 is used to feed back the student psychological report to the student, parents and teachers.

[0030] In this embodiment, the test analysis unit 112 performs emotional and psychological analysis based on the student's assessment results, the report generation unit 113 generates a student psychological report based on the analysis results, and the student psychological report is fed back to the student, parents and teachers through the report feedback unit 114.

[0031] Further, the warning push module includes a risk assessment unit 115, a warning information generation unit 116 and a push execution unit 117, and the risk assessment unit 115, the warning information generation unit 116 and the push execution unit 117 are all connected to the cloud management platform 101; The risk assessment unit 115 is used to assess the student's mental health risk and set a warning threshold based on the emotion recognition results and behavior data analysis; The warning information generating unit 116 is used to determine whether the risk assessment result exceeds a preset threshold and automatically generate warning information; The push execution unit 117 is used to push the warning information to teachers and parents in real time.

[0032] In this embodiment, the risk assessment unit 115 assesses the students' mental health risks based on emotion recognition results and behavioral data analysis, and sets a warning threshold. The warning information generation unit 116 determines whether the risk assessment results exceed the preset threshold and automatically generates warning information. The push execution unit 117 pushes the warning information to teachers and parents in real time.

[0033] Further, the mental health reservation module includes a service display unit 118 and a reservation application unit 119, and both the service display unit 118 and the reservation application unit 119 are connected to the cloud management platform 101; The service display unit 118 is used to provide a list of mental health services for students; The reservation application unit 119 is used for students to make reservations and select corresponding health service types in the mental health service list.

[0034] In this embodiment, in the cloud management platform 101, the service display unit 118 provides a list of mental health services for students. Students make reservations and select corresponding health service types in the mental health service list and submit them through the reservation application unit 119.

[0035] Furthermore, the emotion recognition system based on AI recognition also includes an autonomous training module, which includes a course coaching unit 122, an emotion relaxation unit 123, and a stress training unit 124, and the course coaching unit 122, the emotion relaxation unit 123, and the stress training unit 124 are all connected to the cloud management platform 101; The course tutoring unit 122 is used to provide students with psychology courses; The emotional relaxation unit 123 is used to provide students with emotional relaxation tools; The stress training unit 124 is used to test the student's current psychological stress.

[0036] In this embodiment, the course tutoring unit 122 provides students with psychology courses, the emotional relaxation unit 123 provides students with emotional relaxation tools, such as videos, music, games and meditation guidance, etc., and students test their current psychological stress through the stress training unit 124.

[0037] The present invention also provides an emotion recognition method based on AI recognition, comprising the following steps: S1: Use public emotion databases and user-uploaded materials to collect facial images, videos, and audio data of students in different emotional states; S2: Clean and preprocess the collected data; S3: Using computer vision techniques, we extract emotion-related features from preprocessed facial images, as well as body posture, gesture, and action features from videos; S4: Extract emotion-related acoustic features through speech analysis technology to determine the speaker's emotional state; S5: Select appropriate deep learning models based on task requirements and data characteristics; S6: Use the labeled dataset to train the model; S7: Deploy the trained model to actual application scenarios; S8: Capturing the user's facial expression through the camera acquisition module 103 and the user's voice through the microphone; S9: Extract features and input them into the model for emotion recognition, and immediately feedback the recognition results.

[0038] In this embodiment, firstly, a public emotion database and materials uploaded by users are used to collect facial images, videos and audio data of students in different emotional states, and the collected data are cleaned and preprocessed. Computer vision technology is used to extract emotion-related features from the preprocessed facial images, and body posture, gesture and movement features are extracted from the videos. Then, acoustic features related to emotions are extracted through speech analysis technology to judge the emotional state of the speaker. A suitable deep learning model is selected according to task requirements and data characteristics, and then the model is trained using annotated data sets. The trained model is deployed to actual application scenarios. Finally, the user's facial expressions are captured by the camera acquisition module 103 and the user's voice is captured by the microphone. The extracted features are input into the model for emotion recognition, and the recognition results are immediately fed back.

[0039] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An emotion recognition system based on AI recognition, characterized in that: It includes a cloud management platform, an information entry module, a video acquisition module, a preprocessing module, a data analysis module, a psychological testing module, an early warning push module, a mental health appointment module, a teacher login module and a student login module, wherein the information entry module, the data analysis module, the psychological testing module, the mental health appointment module and the early warning push module are all connected to the cloud management platform, the preprocessing module is connected to the data analysis module, the video acquisition module is connected to the preprocessing module, and the teacher login module and the student login module are all connected to the cloud management platform; The cloud management platform is used to manage students’ mental health status, monitor, analyze and predict students’ mental health status in real time; The information entry module is used to establish mental health files for all students in the school; The camera acquisition module is used to capture facial and neck muscle micro-vibration data of middle school students on campus; The preprocessing module is used to preprocess the captured video data; The data analysis module is used to convert the micro-vibration data of the students' facial and neck muscles into three-dimensional motion trajectories, and compare and analyze them with the emotion model database; The psychological testing module is used to test the mental health status of students, so that teachers and parents can intuitively understand the mental health status of the monitored subjects; The warning push module is used to conduct risk assessment based on the students' emotional results and push warning information to relevant teachers and school leaders in real time; The mental health appointment module is used to provide students with a convenient mental health service appointment channel to help them obtain professional psychological support and intervention in a timely manner; The teacher login module is used for teachers to log in to the cloud management platform; The student login module is used for students to log in to the cloud management platform.

2. The emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The preprocessing module includes a data cleaning unit, a data denoising unit and a data color enhancement unit, and the data cleaning unit, the data denoising unit and the data color enhancement unit are all connected to the data acquisition module; The data cleaning unit is used to remove redundant and irrelevant information in the video data; The data denoising unit is used to remove noise from video data; The data color enhancement unit is used to adjust the color balance, contrast and brightness of the video data.

3. The emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The data analysis module includes a pedestrian recognition unit, a face recognition unit, a morphological analysis unit and a model comparison unit, and the pedestrian recognition unit, the face recognition unit, the morphological analysis unit and the model comparison unit are all connected to the preprocessing module; The pedestrian recognition unit is used to detect and recognize pedestrians in the picture in real time; The face recognition unit is used to obtain facial expression changes of pedestrians; The morphological analysis unit is used to analyze the posture and movement of pedestrians, compare with normal behavior patterns, and identify possible abnormal behavior; The model comparison unit is used to put the data into the model for emotion recognition and predict the complex emotional state of pedestrians.

4. The emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The psychological test module includes an evaluation tool selection unit, a test analysis unit, a report generation unit and a report feedback unit, and the evaluation tool selection unit, the test analysis unit, the report generation unit and the report feedback unit are all connected to the cloud management platform; The assessment tool selection unit is used to select appropriate psychological assessment scales and questionnaires according to the assessment purpose and student group characteristics; The test analysis unit is used to perform emotional and psychological analysis based on the student's assessment results; The report generating unit is used to generate a student psychology report according to the analysis results; The report feedback unit is used to feed back the student psychology report to the student, parents and teachers.

5. The emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The early warning push module includes a risk assessment unit, an early warning information generation unit and a push execution unit, and the risk assessment unit, the early warning information generation unit and the push execution unit are all connected to the cloud management platform; The risk assessment unit is used to assess the student's mental health risk and set a warning threshold based on the emotion recognition results and behavior data analysis; The warning information generation unit is used to determine whether the risk assessment result exceeds a preset threshold and automatically generate warning information; The push execution unit is used to push the warning information to teachers and parents in real time.

6. The emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The mental health reservation module includes a service display unit and a reservation application unit, and both the service display unit and the reservation application unit are connected to the cloud management platform; The service display unit is used to list the mental health services provided to students; The reservation application unit is used for students to make reservations and select corresponding health service types in the mental health service list.

7. An emotion recognition method based on AI recognition, applied to the emotion recognition system based on AI recognition as claimed in claim 1, characterized in that: The steps include: Use public emotion databases and user-uploaded materials to collect facial images, videos, and audio data of students in different emotional states; Clean and preprocess the collected data; Using computer vision techniques, we extract emotion-related features from pre-processed facial images, as well as body posture, gesture, and action features from videos; Extract emotion-related acoustic features through speech analysis technology to determine the speaker's emotional state; Select appropriate deep learning models based on task requirements and data characteristics; Use the labeled dataset to train the model; Deploy the trained model to actual application scenarios; Capturing the user's facial expression through the camera acquisition module and the user's voice through the microphone; The extracted features are input into the model for emotion recognition, and the recognition results are fed back immediately.

Images