A student emotion analysis method and system based on face recognition

Abstract

The present invention provides a student emotion analysis method and system based on face recognition, wherein the method comprises: based on an image acquisition device set in a target area, performing face acquisition on students in the target area; performing face recognition based on the face acquisition image, determining the facial emotion and behavior state of the student; determining the similarity between the facial emotion and the behavior state; and determining the emotion result of the student based on the facial emotion, behavior state and similarity. By performing image acquisition and face recognition on students, determining the facial emotion and behavior state, and then obtaining the emotion result of the students, it is convenient to timely discover and solve the psychological problems of students, and play a certain protective role for students.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a student emotion analysis method and system based on face recognition. Background Art

[0002] The prevalence of various mood disorders and anxiety disorders in my country has been increasing year by year, and mental and psychological diseases have become one of the important factors restricting the increase in life expectancy. Mental and psychological diseases are characterized by slowness, long-term and difficulty in detection. For individuals who do not have relevant professional knowledge, it is difficult to discover mental and psychological disease problems in daily work and life.

[0003] Because facial expressions and micro-expressions play an important role in the non-verbal communication process of interpersonal communication, they can sometimes better reflect people's emotions and feelings than language, and thus reflect people's psychological and mental health status.

[0004] Therefore, the present invention proposes a student emotion analysis method and system based on face recognition. Summary of the invention

[0005] The present invention provides a student emotion analysis method and system based on face recognition, which is used to determine the facial emotions and behavioral states of students through image collection and face recognition, and then obtain the students' emotional results, so as to facilitate timely discovery and timely resolution of students' existing psychological problems, and play a certain protective role for students.

[0006] The present invention provides a student emotion analysis method based on face recognition, comprising:

[0007] Step 1: Based on an image acquisition device set in the target area, face acquisition is performed on students in the target area;

[0008] Step 2: Performing face recognition based on the face collection image to determine the student's facial emotion and behavior state;

[0009] Step 3: Determine the similarity between the facial emotion and the behavioral state;

[0010] Step 4: Based on the facial emotions, behavioral states and similarities, the emotional results of the students are determined.

[0011] Preferably, step 1: performing face acquisition of students in the target area based on an image acquisition device arranged in the target area, comprises:

[0012] Determine the effective shooting range of each image acquisition device in the target area;

[0013] Based on the human body sensing sensors arranged in the target area, human body sensing is performed on the students in the target area, and based on the position layout of the human body sensing sensors and the image acquisition device, the human body sensing results are classified into ranges;

[0014] According to the range classification results, the image acquisition device corresponding to the effective shooting range is triggered to collect the faces of the students;

[0015] At the same time, capturing the movement trajectory of the student based on the target area to determine whether there is a cross-border connection point;

[0016] If not, continuously capture the student according to the triggered image capture device to obtain a first image set;

[0017] If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

[0018] Preferably, step 2: performing face recognition based on the face collection image to determine the student's facial emotion and behavior state includes:

[0019] Recognize the face collection image based on a face recognition model to obtain the identity of the student;

[0020] Based on the identity identifier, retrieve daily emotional behaviors matching the student from a daily database, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors;

[0021] Based on the emotion recognition model, the face collection image is recognized, and based on the daily emotion habit factor, the recognition result is first corrected to obtain the facial emotion of the student;

[0022] Based on the behavior recognition model, the face collection image is recognized, and based on the daily behavior habit factor, the recognition result is corrected for the second time to obtain the behavior status of the student.

[0023] Preferably, step 3: determining the similarity between the facial emotion and the behavioral state comprises:

[0024] Acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state;

[0025] Establishing a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude;

[0026] Based on all the relationship vectors, a relationship matrix is ​​constructed, and at the same time, a first sub-matrix based on the emotion attributes and the facial expression amplitude, and a second sub-matrix based on the state attributes and the behavior amplitude is constructed;

[0027] Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature;

[0028]

[0029] Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1 Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3;

[0030] When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

[0031] Preferably, step 4: determining the student's emotional result based on the facial emotion, behavioral state and similarity, comprises:

[0032] Obtain the emotion judgment results of the collected face images from professional instructors;

[0033] When the similarity is greater than or equal to a preset degree, the facial emotion, behavior state and emotion judgment result are combined to obtain the emotion result of the student;

[0034] When the similarity is less than a preset degree, extracting the first psychological characteristic of the student from the emotion judgment result;

[0035] Performing a psychological analysis on the facial emotion to obtain a second psychological characteristic, and at the same time, performing a psychological analysis on the behavioral state to obtain a third psychological characteristic;

[0036] Determining a first matching degree between the first psychological characteristic and the second psychological characteristic and a second matching degree between the first psychological characteristic and the third psychological characteristic;

[0037] Selecting a third matching degree with a greater matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree;

[0038] According to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree, a first weight is assigned to the emotion analysis content corresponding to the third matching degree, and a second weight is assigned to the emotion analysis content corresponding to the fourth matching degree;

[0039] Based on the facial emotions, behavioral states, configuration results and emotion judgment results, the emotion results of the students are obtained.

[0040] The present invention provides a student emotion analysis system based on face recognition, comprising:

[0041] A face acquisition module, used for acquiring faces of students in the target area based on an image acquisition device arranged in the target area;

[0042] A face recognition module, used to perform face recognition based on the face collection image to determine the facial emotion and behavior status of the student;

[0043] A similarity determination module, used to determine the similarity between the facial emotion and the behavioral state;

[0044] The emotion determination module is used to determine the emotion result of the student based on the facial emotion, behavior state and similarity.

[0045] Preferably, the face acquisition module includes:

[0046] A range determination unit, used to determine the effective shooting range of each image acquisition device in the target area;

[0047] A range classification unit, used to perform human body sensing on students in the target area based on a human body sensing sensor arranged in the target area, and to perform range classification on the human body sensing result based on the position layout of the human body sensing sensor and the image acquisition device;

[0048] A device triggering unit is used to trigger the image acquisition device corresponding to the effective shooting range to collect the face of the students according to the range classification result;

[0049] A cross-border judgment unit, used to capture the movement trajectory of the student based on the target area and judge whether there is a cross-border connection point;

[0050] If not, continuously capture the student according to the triggered image capture device to obtain a first image set;

[0051] If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

[0052] Preferably, the face recognition module includes:

[0053] An identity recognition unit, used to recognize the face collection image based on a face recognition model to obtain the identity of the student;

[0054] A behavior analysis unit is used to retrieve daily emotional behaviors matching the student from a daily database based on the identity identifier, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors;

[0055] A first correction unit is used to recognize the face collection image based on an emotion recognition model, and to perform a first correction on the recognition result based on a daily emotion habit factor to obtain the facial emotion of the student;

[0056] The second correction unit is used to identify the face collection image based on the behavior recognition model, and to perform a second correction on the recognition result based on the daily behavior habit factor to obtain the behavior state of the student.

[0057] Preferably, the similarity determination module includes:

[0058] An acquisition unit, used to acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state;

[0059] A vector building unit, used for building a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude;

[0060] A matrix construction unit, used to construct a relationship matrix based on all relationship vectors, and at the same time, construct a first sub-matrix based on the emotion attribute and the facial expression amplitude, and a second sub-matrix based on the state attribute and the behavior amplitude;

[0061] Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature;

[0062]

[0063] Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3;

[0064] When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

[0065] Preferably, the emotion determination module includes:

[0066] A human judgment result acquisition unit is used to obtain the emotion judgment results of the professional instructor on the collected face images;

[0067] A first emotion result acquisition unit, configured to combine the facial emotion, behavior state and emotion judgment result to acquire the emotion result of the student when the similarity is greater than or equal to a preset degree;

[0068] A feature extraction unit, configured to extract the first psychological feature of the student from the emotion judgment result when the similarity is less than a preset degree;

[0069] A psychological analysis unit, configured to perform a psychological analysis on the facial emotion to obtain a second psychological characteristic, and simultaneously perform a psychological analysis on the behavioral state to obtain a third psychological characteristic;

[0070] a feature matching unit, configured to determine a first matching degree between the first psychological feature and the second psychological feature and a second matching degree between the first psychological feature and the third psychological feature;

[0071] A matching degree comparison unit, configured to select a third matching degree with a larger matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree;

[0072] a weight configuration unit, configured to configure a first weight to the emotion analysis content corresponding to the third matching degree, and configure a second weight to the emotion analysis content corresponding to the fourth matching degree, according to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree;

[0073] The second emotion result acquisition unit is used to acquire the emotion result of the student based on the facial emotion, behavior state, configuration result and emotion judgment result.

[0074] Compared with the prior art, the beneficial effects of this application are:

[0075] By collecting images and performing face recognition on students, we can determine their facial emotions and behavioral states, and then obtain their emotional results, which makes it easier to discover and solve their psychological problems in a timely manner, and plays a certain protective role for them.

[0076] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0077] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0079] Figure 1 This is a flow chart of a student emotion analysis method based on face recognition in an embodiment of the present invention;

[0080] Figure 2 This is a structural diagram of a student emotion analysis system based on face recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0082] The present invention provides a student emotion analysis method based on face recognition, such as Figure 1 As shown, including:

[0083] Step 1: Based on an image acquisition device set in the target area, face acquisition is performed on students in the target area;

[0084] Step 2: Performing face recognition based on the face collection image to determine the student's facial emotion and behavior state;

[0085] Step 3: Determine the similarity between the facial emotion and the behavioral state;

[0086] Step 4: Based on the facial emotions, behavioral states and similarities, the emotional results of the students are determined.

[0087] In this embodiment, the image acquisition device can be implemented as a camera.

[0088] In this embodiment, for example, the target area is a classroom, and there are effective area ranges 1, 2, and 3 in the classroom, and an image acquisition device is set in each effective area to facilitate image acquisition of students in the corresponding range.

[0089] In this embodiment, facial emotions refer to students' facial expressions, such as sadness, happiness, grief and anger, etc.

[0090] In this embodiment, the behavior state refers to the student's action behavior, such as the swing amplitude of the arms, the swing amplitude of the steps, etc.

[0091] In this embodiment, the similarity is used to comprehensively determine the student's emotional results from two aspects. The higher the similarity, the greater the reference value.

[0092] The beneficial effect of the above technical solution is: by collecting images and performing face recognition on students, the facial emotions and behavioral states are determined, and then the emotional results of the students are obtained, which is convenient for timely discovery and timely resolution of the psychological problems of students, and plays a certain protective role for the students.

[0093] The present invention provides a student emotion analysis method based on face recognition, step 1: based on an image acquisition device set in a target area, performing face acquisition on students in the target area, comprising:

[0094] Determine the effective shooting range of each image acquisition device in the target area;

[0095] Based on the human body sensing sensors arranged in the target area, human body sensing is performed on the students in the target area, and based on the position layout of the human body sensing sensors and the image acquisition device, the human body sensing results are classified into ranges;

[0096] According to the range classification results, the image acquisition device corresponding to the effective shooting range is triggered to collect the faces of the students;

[0097] At the same time, capturing the movement trajectory of the student based on the target area to determine whether there is a cross-border connection point;

[0098] If not, continuously capture the student according to the triggered image capture device to obtain a first image set;

[0099] If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

[0100] In this embodiment, the human body sensor is set at the entrance of the target area, and after sensing, it triggers the equipment in the corresponding range to start working and collecting data.

[0101] In this embodiment, the cross-border connection point refers to the boundary line from the effective area range 1 to the effective area range 2, and each point on the line can be a cross-border connection point.

[0102] The beneficial effect of the above technical solution is: through human body sensing, the device that needs to be triggered is determined, and then the faces of students within the range are collected, which provides a basis for subsequent face recognition and provides a certain basis for timely discovery and timely resolution of students' psychological problems.

[0103] The present invention provides a student emotion analysis method based on face recognition, step 2: performing face recognition based on a face collection image to determine the student's facial emotion and behavior state, including:

[0104] Recognize the face collection image based on a face recognition model to obtain the identity of the student;

[0105] Based on the identity identifier, retrieve daily emotional behaviors matching the student from a daily database, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors;

[0106] Based on the emotion recognition model, the face collection image is recognized, and based on the daily emotion habit factor, the recognition result is first corrected to obtain the facial emotion of the student;

[0107] Based on the behavior recognition model, the face collection image is recognized, and based on the daily behavior habit factor, the recognition result is corrected for the second time to obtain the behavior status of the student.

[0108] In this embodiment, the face recognition model, the emotion recognition model, and the behavior recognition model are all pre-trained, and the corresponding training samples are: face images, face emotions, and action behaviors, respectively.

[0109] In this embodiment, the daily database includes identity identification and daily behaviors related to the identity identification, mainly for the purpose of making targeted corrections to the student, improving the accuracy of acquiring facial emotions and behavioral states, and better matching the student himself.

[0110] In this embodiment, daily emotional behavior is mainly used to determine the student's behavioral habits and facial expression habits, so as to correct the results after recognition.

[0111] In this embodiment, the daily emotional habit factor refers to the students' daily micro-expressions, etc., and the daily behavioral habit factor refers to the students' daily behavioral postures, etc. Different students express the same emotion differently, which is reflected in the daily behavioral habit factor and the daily emotional habit factor.

[0112] The beneficial effects of the above technical solution are: through preliminary recognition, the identity is determined, and then daily emotional behavior is extracted, and through analysis, habit factors based on emotions and emotions are obtained, and then the subsequent recognition results are corrected to ensure the authenticity of the student's facial emotions and behavioral states, providing a certain accurate foundation for timely discovery and timely resolution of students' existing psychological problems.

[0113] The present invention provides a student emotion analysis method based on face recognition, step 3: determining the similarity between the facial emotion and the behavior state, comprising:

[0114] Acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state;

[0115] Establishing a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude;

[0116] Based on all the relationship vectors, a relationship matrix is ​​constructed, and at the same time, a first sub-matrix based on the emotion attributes and the facial expression amplitude, and a second sub-matrix based on the state attributes and the behavior amplitude is constructed;

[0117] Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature;

[0118]

[0119] Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1 Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3;

[0120] When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

[0121] In this embodiment, the emotional attributes are related to sadness, happiness, etc., and the facial expression amplitude refers to the degree of expression of the student's facial muscles; the behavioral state is related to behavioral posture, etc., and the behavioral amplitude refers to the swing amplitude of different parts of the student.

[0122] In this embodiment, the relationship vector is determined based on the comprehensive combination of facial emotions and behavioral states, and the determined relationship matrix refers to the relationship vectors at different moments.

[0123] In this embodiment, the first sub-matrix is ​​constructed based on vectors corresponding to facial emotions at different moments, and the second sub-matrix is ​​constructed based on vectors corresponding to behavioral states at different moments.

[0124] In this embodiment, for example, the relationship feature, the first feature and the second feature can be obtained by analyzing the corresponding matrix based on a matrix analysis model, and the matrix analysis model is pre-trained.

[0125] In this embodiment, the value of the preset degree is generally 0.8.

[0126] The beneficial effect of the above technical solution is: by constructing a relationship matrix based on facial emotions and behavioral states, and separately constructing matrices of facial emotions and behavioral states, and then performing feature similarity comparison to calculate the similarity, the reliability of emotion determination is verified, thereby effectively improving the accuracy of discovering students' psychological problems and facilitating timely resolution.

[0127] The present invention provides a student emotion analysis method based on face recognition, step 4: based on the facial emotion, behavior state and similarity, determining the student's emotion result, including:

[0128] Obtain the emotion judgment results of the collected face images from professional instructors;

[0129] When the similarity is greater than or equal to a preset degree, the facial emotion, behavior state and emotion judgment result are combined to obtain the emotion result of the student;

[0130] When the similarity is less than a preset degree, extracting the first psychological characteristic of the student from the emotion judgment result;

[0131] Performing a psychological analysis on the facial emotion to obtain a second psychological characteristic, and at the same time, performing a psychological analysis on the behavioral state to obtain a third psychological characteristic;

[0132] Determining a first matching degree between the first psychological characteristic and the second psychological characteristic and a second matching degree between the first psychological characteristic and the third psychological characteristic;

[0133] Selecting a third matching degree with a greater matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree;

[0134] According to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree, a first weight is assigned to the emotion analysis content corresponding to the third matching degree, and a second weight is assigned to the emotion analysis content corresponding to the fourth matching degree;

[0135] Based on the facial emotions, behavioral states, configuration results and emotion judgment results, the emotion results of the students are obtained.

[0136] In this embodiment, the emotion judgment result is the result of human judgment.

[0137] In this embodiment, the psychological feature extraction model is used to extract the results in different situations to obtain psychological features, and the first psychological feature is extracted based on the emotion judgment result, and is trained based on different emotion results (facial emotions, behavioral states) and corresponding psychological features.

[0138] In this embodiment, the matching degree is calculated to determine the consistency between the machine judgment result and the human judgment result.

[0139] In this embodiment, the first psychological characteristic is 1, 2, 3, 4, the second psychological characteristic is 1, 2, and the second psychological characteristic is 3, 5. At this time, the corresponding first matching degree is 1, and the second matching degree is 0.5. At this time, the emotion analysis content corresponding to the third matching degree is facial emotion, and the emotion analysis content corresponding to the fourth matching degree is behavioral state.

[0140] For example, the standard matching degree is 0.8. At this time, the corresponding first difference is 0.2, the second difference is -0.3, and the third difference is 0.5. For example, at this time, the configured first weight is 0.7 and the second weight is 0.3.

[0141] The beneficial effect of the above technical solution is: by extracting features from three aspects and calculating the matching degree between different features and various differences with the standard matching degree, different weights are configured to provide an effective basis for determining emotional results and provide a certain basis for timely discovery and timely resolution of students' existing psychological problems.

[0142] The present invention provides a student emotion analysis system based on face recognition, such as Figure 2 As shown, including:

[0143] A face acquisition module, used for acquiring faces of students in the target area based on an image acquisition device arranged in the target area;

[0144] A face recognition module, used to perform face recognition based on the face collection image to determine the facial emotion and behavior status of the student;

[0145] A similarity determination module, used to determine the similarity between the facial emotion and the behavioral state;

[0146] The emotion determination module is used to determine the emotion result of the student based on the facial emotion, behavior state and similarity.

[0147] The beneficial effect of the above technical solution is: by collecting images and performing face recognition on students, the facial emotions and behavioral states are determined, and then the emotional results of the students are obtained, which is convenient for timely discovery and timely resolution of the psychological problems of students, and plays a certain protective role for the students.

[0148] The present invention provides a student emotion analysis system based on face recognition, wherein the face acquisition module comprises:

[0149] A range determination unit, used to determine the effective shooting range of each image acquisition device in the target area;

[0150] A range classification unit, used to perform human body sensing on students in the target area based on a human body sensing sensor arranged in the target area, and to perform range classification on the human body sensing result based on the position layout of the human body sensing sensor and the image acquisition device;

[0151] A device triggering unit is used to trigger the image acquisition device corresponding to the effective shooting range to collect the face of the students according to the range classification result;

[0152] A cross-border judgment unit, used to capture the movement trajectory of the student based on the target area and judge whether there is a cross-border connection point;

[0153] If not, continuously capture the student according to the triggered image capture device to obtain a first image set;

[0154] If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

[0155] The beneficial effect of the above technical solution is: through human body sensing, the device that needs to be triggered is determined, and then the faces of students within the range are collected, which provides a basis for subsequent face recognition and provides a certain basis for timely discovery and timely resolution of students' psychological problems.

[0156] The present invention provides a student emotion analysis system based on face recognition, wherein the face recognition module comprises:

[0157] An identity recognition unit, used to recognize the face collection image based on a face recognition model to obtain the identity of the student;

[0158] A behavior analysis unit is used to retrieve daily emotional behaviors matching the student from a daily database based on the identity identifier, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors;

[0159] A first correction unit is used to recognize the face collection image based on an emotion recognition model, and to perform a first correction on the recognition result based on a daily emotion habit factor to obtain the facial emotion of the student;

[0160] The second correction unit is used to identify the face collection image based on the behavior recognition model, and to perform a second correction on the recognition result based on the daily behavior habit factor to obtain the behavior state of the student.

[0161] The beneficial effects of the above technical solution are: through preliminary recognition, the identity is determined, and then daily emotional behavior is extracted, and through analysis, habit factors based on emotions and emotions are obtained, and then the subsequent recognition results are corrected to ensure the authenticity of the student's facial emotions and behavioral states, providing a certain accurate foundation for timely discovery and timely resolution of students' existing psychological problems.

[0162] The present invention provides a student emotion analysis system based on face recognition, wherein the similarity determination module comprises:

[0163] An acquisition unit, used to acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state;

[0164] A vector building unit, used for building a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude;

[0165] A matrix construction unit, used to construct a relationship matrix based on all relationship vectors, and at the same time, construct a first sub-matrix based on the emotion attribute and the facial expression amplitude, and a second sub-matrix based on the state attribute and the behavior amplitude;

[0166] Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature;

[0167]

[0168] Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1 Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3;

[0169] When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

[0170] The beneficial effect of the above technical solution is: by constructing a relationship matrix based on facial emotions and behavioral states, and separately constructing matrices of facial emotions and behavioral states, and then performing feature similarity comparison to calculate the similarity, the reliability of emotion determination is verified, thereby effectively improving the accuracy of discovering students' psychological problems and facilitating timely resolution.

[0171] The present invention provides a student emotion analysis system based on face recognition, wherein the emotion determination module comprises:

[0172] A human judgment result acquisition unit is used to obtain the emotion judgment results of the professional instructor on the collected face images;

[0173] A first emotion result acquisition unit, configured to combine the facial emotion, behavior state and emotion judgment result to acquire the emotion result of the student when the similarity is greater than or equal to a preset degree;

[0174] A feature extraction unit, configured to extract the first psychological feature of the student from the emotion judgment result when the similarity is less than a preset degree;

[0175] A psychological analysis unit, configured to perform a psychological analysis on the facial emotion to obtain a second psychological characteristic, and simultaneously perform a psychological analysis on the behavioral state to obtain a third psychological characteristic;

[0176] a feature matching unit, configured to determine a first matching degree between the first psychological feature and the second psychological feature and a second matching degree between the first psychological feature and the third psychological feature;

[0177] A matching degree comparison unit, configured to select a third matching degree with a larger matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree;

[0178] a weight configuration unit, configured to configure a first weight to the emotion analysis content corresponding to the third matching degree, and configure a second weight to the emotion analysis content corresponding to the fourth matching degree, according to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree;

[0179] The second emotion result acquisition unit is used to acquire the emotion result of the student based on the facial emotion, behavior state, configuration result and emotion judgment result.

[0180] The beneficial effect of the above technical solution is: by extracting features from three aspects and calculating the matching degree between different features and various differences with the standard matching degree, different weights are configured to provide an effective basis for determining emotional results and provide a certain basis for timely discovery and timely resolution of students' existing psychological problems.

[0181] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A student emotion analysis method based on face recognition, characterized in that: include: Step 1: Based on an image acquisition device set in the target area, face acquisition is performed on students in the target area; Step 2: Performing face recognition based on the face collection image to determine the student's facial emotion and behavior state; Step 3: Determine the similarity between the facial emotion and the behavioral state; Step 4: Based on the facial emotions, behavioral states and similarities, the emotional results of the students are determined; Wherein, step 3: determining the similarity between the facial emotion and the behavioral state comprises: Acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state; Establishing a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude; Based on all the relationship vectors, a relationship matrix is ​​constructed, and at the same time, a first sub-matrix based on the emotion attributes and the facial expression amplitude, and a second sub-matrix based on the state attributes and the behavior amplitude is constructed; Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature; Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1 Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3; When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

2. The student emotion analysis method based on face recognition as claimed in claim 1, characterized in that: Step 1: Based on an image acquisition device set in the target area, face acquisition of students in the target area is performed, including: Determine the effective shooting range of each image acquisition device in the target area; Based on the human body sensing sensors arranged in the target area, human body sensing is performed on the students in the target area, and based on the position layout of the human body sensing sensors and the image acquisition device, the human body sensing results are classified into ranges; According to the range classification results, the image acquisition device corresponding to the effective shooting range is triggered to collect the faces of the students; At the same time, capturing the movement trajectory of the student based on the target area to determine whether there is a cross-border connection point; If not, continuously capture the student according to the triggered image capture device to obtain a first image set; If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

3. The student emotion analysis method based on face recognition as claimed in claim 1, characterized in that: Step 2: Performing face recognition based on the face collection image to determine the student's facial emotion and behavior state, including: Recognize the face collection image based on a face recognition model to obtain the identity of the student; Based on the identity identifier, retrieve daily emotional behaviors matching the student from a daily database, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors; Based on the emotion recognition model, the face collection image is recognized, and based on the daily emotion habit factor, the recognition result is first corrected to obtain the facial emotion of the student; Based on the behavior recognition model, the face collection image is recognized, and based on the daily behavior habit factor, the recognition result is corrected for the second time to obtain the behavior status of the student.

4. The student emotion analysis method based on face recognition as claimed in claim 1, characterized in that: Step 4: Based on the facial emotions, behavioral states and similarities, the emotional results of the students are determined, including: Obtain the emotion judgment results of the collected face images from professional instructors; When the similarity is greater than or equal to a preset degree, the facial emotion, behavior state and emotion judgment result are combined to obtain the emotion result of the student; When the similarity is less than a preset degree, extracting the first psychological characteristic of the student from the emotion judgment result; Performing a psychological analysis on the facial emotion to obtain a second psychological characteristic, and at the same time, performing a psychological analysis on the behavioral state to obtain a third psychological characteristic; Determining a first matching degree between the first psychological characteristic and the second psychological characteristic and a second matching degree between the first psychological characteristic and the third psychological characteristic; Selecting a third matching degree with a greater matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree; According to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree, a first weight is assigned to the emotion analysis content corresponding to the third matching degree, and a second weight is assigned to the emotion analysis content corresponding to the fourth matching degree; Based on the facial emotions, behavioral states, configuration results and emotion judgment results, the emotion results of the students are obtained.

5. A student emotion analysis system based on face recognition, characterized in that: include: A face acquisition module, used for acquiring faces of students in the target area based on an image acquisition device arranged in the target area; A face recognition module, used to perform face recognition based on the face collection image to determine the facial emotion and behavior status of the student; A similarity determination module, used to determine the similarity between the facial emotion and the behavioral state; An emotion determination module, used for determining the emotion result of the student based on the facial emotion, behavior state and similarity; Wherein, the similarity determination module includes: An acquisition unit, used to acquire the emotional attributes and facial expression amplitude of the facial emotion, and at the same time, acquire the state attributes and behavior amplitude of the behavior state; A vector building unit, used for building a relationship vector that simultaneously captures the student's emotional attributes, facial expression amplitude, state attributes, and behavior amplitude; A matrix construction unit, used to construct a relationship matrix based on all relationship vectors, and at the same time, construct a first sub-matrix based on the emotion attribute and the facial expression amplitude, and a second sub-matrix based on the state attribute and the behavior amplitude; Acquire a relation feature of the relation matrix, a first feature of the first submatrix, and a second feature of the second submatrix, and calculate a similarity between the relation feature and the first feature and the second feature; Among them, S represents similarity; n1 represents the number of relationship features; n2 represents the number of first features; n3 represents the number of second features; y i1 represents the eigenvalue of the i1th relation feature; s i1 Represents the standard conversion coefficient matching the feature attribute related to the i1th relationship feature; y i2 represents the eigenvalue of the i2th first feature; s i2 represents the standard conversion coefficient matching the feature attribute associated with the i2th first feature; i3 represents the eigenvalue of the i3th second feature; s i3 represents the standard conversion coefficient matched to the characteristic attribute associated with the i3th second characteristic; wherein n1 is greater than or equal to n2+n3; When the similarity is greater than a preset degree, the facial emotion and behavior state are retained.

6. The student emotion analysis system based on face recognition as claimed in claim 5, characterized in that: The face acquisition module comprises: A range determination unit, used to determine the effective shooting range of each image acquisition device in the target area; A range classification unit, used to perform human body sensing on students in the target area based on a human body sensing sensor arranged in the target area, and to perform range classification on the human body sensing result based on the position layout of the human body sensing sensor and the image acquisition device; A device triggering unit is used to trigger the image acquisition device corresponding to the effective shooting range to perform face acquisition on the students according to the range classification result; A cross-border judgment unit, used to capture the movement trajectory of the student based on the target area and judge whether there is a cross-border connection point; If not, continuously capture the student according to the triggered image capture device to obtain a first image set; If so, trigger the image acquisition device matching the cross-border connection point to perform cross-range acquisition of the student and obtain a second image set.

7. The student emotion analysis system based on face recognition as claimed in claim 5, characterized in that: The face recognition module comprises: An identity recognition unit, used to recognize the face collection image based on a face recognition model to obtain the identity of the student; A behavior analysis unit is used to retrieve daily emotional behaviors matching the student from a daily database based on the identity identifier, and analyze the daily emotional behaviors to obtain the student's daily emotional habit factors and daily behavioral habit factors; A first correction unit is used to recognize the face collection image based on an emotion recognition model, and to perform a first correction on the recognition result based on a daily emotion habit factor to obtain the facial emotion of the student; The second correction unit is used to identify the face collection image based on the behavior recognition model, and to perform a second correction on the recognition result based on the daily behavior habit factor to obtain the behavior state of the student.

8. The student emotion analysis system based on face recognition as claimed in claim 5, characterized in that: The emotion determination module comprises: A human judgment result acquisition unit is used to obtain the emotion judgment results of the professional instructor on the collected face images; A first emotion result acquisition unit, configured to combine the facial emotion, behavior state and emotion judgment result to acquire the emotion result of the student when the similarity is greater than or equal to a preset degree; A feature extraction unit, configured to extract the first psychological feature of the student from the emotion judgment result when the similarity is less than a preset degree; A psychological analysis unit, configured to perform a psychological analysis on the facial emotion to obtain a second psychological characteristic, and simultaneously perform a psychological analysis on the behavioral state to obtain a third psychological characteristic; a feature matching unit, configured to determine a first matching degree between the first psychological feature and the second psychological feature and a second matching degree between the first psychological feature and the third psychological feature; A matching degree comparison unit, configured to select a third matching degree with a greater matching degree and a fourth matching degree with a smaller matching degree from the first matching degree and the second matching degree; a weight configuration unit, configured to configure a first weight to the emotion analysis content corresponding to the third matching degree, and configure a second weight to the emotion analysis content corresponding to the fourth matching degree, according to a first difference between the third matching degree and the standard matching degree, a second difference between the fourth matching degree and the standard matching degree, and a third difference between the third matching degree and the fourth matching degree; The second emotion result acquisition unit is used to acquire the emotion result of the student based on the facial emotion, behavior state, configuration result and emotion judgment result.

Images