Emotion analysis method for ideological and political teaching

Abstract

The invention discloses an emotion analysis method for ideological and political teaching, and belongs to the technical field of image processing. The method comprises the following steps: firstly, pre-collecting front face images of students facing a blackboard, and collecting face images of students in an ideological and political teaching classroom in real time; calculating a head turning angle and a head raising angle of the real-time image according to key points of the front face image and the real-time face image, and acquiring a left eye deviation value and a right eye deviation value of the real-time image according to eye areas of the front face image and the real-time face image; then constructing a posture change degree time sequence based on the data at a plurality of continuous moments; and finally, inputting the angle, deviation value and posture change degree sequential sequence of the continuous moments into a sequential processing neural network to obtain a classroom interest value, thereby accurately evaluating the emotional state of students in ideological and political teaching, and improving the teaching condition evaluation precision.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an emotion analysis method for ideological and political teaching. Background Art

[0002] In the context of current educational informatization and intelligence, classroom teaching quality assessment has always been a key issue of concern in the education field. Traditional classroom teaching evaluation methods mainly rely on teachers' subjective judgments and limited manual observations, lacking means for objectively, real-time, and accurately evaluating students' learning states. At the same time, traditional classroom teaching monitoring technologies are mostly limited to simple attendance rate statistics and surface behavior observations, and it is difficult to deeply and accurately capture students' learning engagement.

[0003] Existing technical solutions for analyzing classroom situations based on students' facial recognition can attempt to analyze learning states by capturing facial images and extracting key points. However, relying solely on facial feature recognition, it is difficult to accurately judge students' learning interests and concentration levels. Moreover, existing technologies have deficiencies in obtaining dynamic change characteristics of students' postures and cannot comprehensively reflect students' true states in the classroom. Therefore, there is a problem of low accuracy in teaching situation assessment in existing technologies. Summary of the Invention

[0004] In view of the above deficiencies in the prior art, an emotion analysis method for ideological and political teaching provided by the present invention solves the problem of low accuracy in teaching situation assessment in existing technologies.

[0005] To achieve the above invention object, the technical solution adopted by the present invention is: an emotion analysis method for ideological and political teaching, including the following steps:

[0006] S1. Pre-collect an image of a student facing the blackboard to obtain a frontal facial image, and collect real-time facial images of students in an ideological and political teaching classroom;

[0007] S2. Obtain the turning angle and looking-up angle of the real-time facial image according to the key points on the frontal facial image and the real-time facial image;

[0008] S3. Obtain the left-eye deviation value and right-eye deviation value of the real-time facial image according to the eye regions on the frontal facial image and the real-time facial image;

[0009] S4. Construct a time series of posture change degrees according to the turning angle, looking-up angle, left-eye deviation value, and right-eye deviation value at consecutive moments;

[0010] S5. Obtain the interest value of the classroom based on the time series processing neural network according to the looking-up angle, turning angle, left-eye deviation value, and right-eye deviation value at consecutive moments, as well as the time series of posture change degrees.

[0011] Further, S2 includes the following sub-steps:

[0012] S21. Extract key points from the frontal face image and the real-time face image respectively, where the key points include: the outer left eye point, the outer right eye point, and the tip of the nose point;

[0013] S22. Obtain the central coordinates of multiple key points on the frontal face image and the real-time face image respectively;

[0014] S23. Calculate the turning angle according to the difference between the central coordinates of the key points of the real-time face image and the central coordinates of the key points of the frontal face image;

[0015] S24. Construct a normal vector of the reference plane according to multiple key points of the frontal face image;

[0016] S25. Construct a normal vector of the real-time plane according to multiple key points of the real-time face image;

[0017] S26. Calculate the head-up angle according to the normal vector of the real-time plane and the normal vector of the reference plane.

[0018] Further, the formula for calculating the turning angle in S23 is: , where R t is the turning angle at the t-th moment, arctan2 is the four-quadrant arctangent function, x o is the abscissa of the central coordinate of the key point of the frontal face image, y o is the ordinate of the central coordinate of the key point of the frontal face image, x t is the abscissa of the central coordinate of the key point of the real-time face image at the t-th moment, y t is the ordinate of the central coordinate of the key point of the real-time face image at the t-th moment, and t is the number of the moment.

[0019] Further, the formula for calculating the head-up angle in S26 is: , where θ t is the head-up angle at the t-th moment, arccos is the arccosine function, h t is the normal vector of the real-time plane at the t-th moment, h o is the normal vector of the reference plane, | | is the modulus operation, and t is the number of the moment.

[0020] Further, S3 includes the following sub-steps:

[0021] S31. Extract the iris region and the sclera region on the frontal face image and the real-time face image respectively;

[0022] S32. On the frontal face image or the real-time face image, splice the iris region and the sclera region to obtain the eyeball region;

[0023] S33. Extract the geometric center coordinates of the eyeball region to obtain the eyeball center;

[0024] S34. Extract the geometric center coordinates of the iris region to obtain the iris center;

[0025] S35. On the frontal face image, calculate the Euclidean distance between the iris center and the eyeball center to obtain the reference distance;

[0026] S36. On the real-time face image, calculate the Euclidean distance between the iris center and the eyeball center to obtain the real-time distance;

[0027] S37. According to the real-time distance and the reference distance of the same eye, obtain the left-eye deviation value and the right-eye deviation value.

[0028] Further, the formula for obtaining the left-eye deviation value in S37 is: , where γ L,t is the left-eye deviation value at the t-th moment, d L,t is the real-time distance of the left eye at the t-th moment, d L,o is the reference distance of the left eye, and D is the Euclidean distance between the outer lateral points of the left and right eyes of the same person;

[0029] The formula for obtaining the right-eye deviation value is: , where γ R,t is the right-eye deviation value at the t-th moment, d R,t is the real-time distance of the right eye at the t-th moment, d R,o is the reference distance of the right eye.

[0030] Further, for the same moment, respectively judge whether the absolute value of the head-turning angle is less than the head-turning angle threshold, whether the absolute value of the head-up angle is less than the head-up angle threshold, whether the absolute value of the left-eye deviation value is less than the deviation threshold, and whether the absolute value of the right-eye deviation value is less than the deviation threshold. If all four conditions are satisfied, the number of parameters that meet the conditions is recorded as 4. If only three conditions are satisfied, the number of parameters that meet the conditions is recorded as 3. If only two conditions are satisfied, the number of parameters that meet the conditions is recorded as 2. If only one condition is satisfied, the number of parameters that meet the conditions is recorded as 1. If all four conditions are not satisfied, the number of parameters that meet the conditions is recorded as 0. Take the number of parameters that meet the conditions as the attitude change degree at this moment, and form a sequence of the attitude change degrees at consecutive multiple moments to obtain the attitude change degree time series.

[0031] Further, the time series processing neural network in S5 includes: a fully connected unit, a first LSTM layer, a second LSTM layer, a time series feature fusion layer, a time series dilated convolutional unit, and an output layer;

[0032] The fully connected unit includes multiple fully connected layers, and each fully connected layer is used to process the turning angle, the looking-up angle, the left-eye deviation value, and the right-eye deviation value at a certain moment;

[0033] The input end of the first LSTM layer is connected to the output end of the fully connected unit; the input end of the second LSTM layer is used to input the time series of the attitude change degree; the input ends of the time series feature fusion layer are respectively connected to the output end of the first LSTM layer and the output end of the second LSTM layer, and its output end is connected to the input end of the time series dilated convolution unit; the output end of the time series dilated convolution unit is connected to the input end of the output layer; the output end of the output layer serves as the output end of the time series processing neural network.

[0034] Further, the expression of the time series feature fusion layer is: , , , where R is the output of the time series feature fusion layer, G1 is the output of the first LSTM layer, G2 is the output of the second LSTM layer, g 1,1 is the output of the first cell unit in the first LSTM layer, g 1,n is the output of the nth cell unit in the first LSTM layer, g 1,N is the output of the Nth cell unit in the first LSTM layer, g 2,1 is the output of the first cell unit in the second LSTM layer, g 2,n is the output of the nth cell unit in the second LSTM layer, g 2,N is the output of the Nth cell unit in the second LSTM layer, n is a positive integer, N is the sequence length, is the convolutional layer, is element-wise multiplication.

[0035] Further, the time series dilated convolution unit includes: the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the first dilated convolutional layer, the second dilated convolutional layer, the max pooling layer, and the adder A1;

[0036] The input end of the first convolutional layer is respectively connected to the input ends of the second convolutional layer, the third convolutional layer, and the fourth convolutional layer, and serves as the input end of the time series dilated convolution unit;

[0037] The output end of the first convolutional layer is connected to the input end of the first dilated convolutional layer; the output end of the second convolutional layer is connected to the input end of the second dilated convolutional layer; the output end of the third convolutional layer is connected to the input end of the max pooling layer; the input ends of the adder A1 are respectively connected to the output end of the first dilated convolutional layer, the output end of the second dilated convolutional layer, the output end of the max pooling layer, and the output end of the fourth convolutional layer, and its output end serves as the output end of the time series dilated convolution unit.

[0038] In summary, the beneficial effects of the present invention are as follows:

[0039] 1. By pre-collecting frontal facial images and comparing them with real-time facial images, and combining key point and eye region data, the present invention can accurately obtain multi-dimensional information such as turning angle, tilting angle, and binocular deviation value. Compared with traditional single facial feature recognition, the comprehensiveness of data collection is greatly improved; in terms of dynamic evaluation, a time series of pose change degrees is constructed to effectively capture the dynamic change characteristics of students' poses, realizing the dynamic monitoring of students' classroom states.

[0040] 2. The present invention inputs the tilting angles, turning angles, left eye deviation values, and right eye deviation values at multiple moments, as well as the time series of pose change degrees, into a time series processing neural network. The time series of pose change degrees is used to enhance the corresponding features of the tilting angles, turning angles, left eye deviation values, and right eye deviation values at multiple moments, making up for the defect of isolated single feature data information, thereby enhancing the temporal correlation and feature complementarity of the data. Through fusion enhancement, the neural network can capture richer and more representative information on students' classroom states, improving the accuracy of the time series processing neural network evaluation and solving the problem of low accuracy in teaching situation evaluation existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flowchart of an emotional analysis method for ideological and political teaching;

[0042] Figure 2 is a schematic structural diagram of a time series processing neural network;

[0043] Figure 3 is a schematic structural diagram of a fully connected unit;

[0044] Figure 4 is a schematic structural diagram of a time series dilated convolution unit. DETAILED DESCRIPTION OF THE INVENTION

[0045] The following describes the specific implementation manners of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation manners. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0046] As Figure 1 shown, an emotional analysis method for ideological and political teaching includes the following sub-steps:

[0047] S1. Pre-acquire an image of a student facing the blackboard to obtain a frontal face image, and acquire the real-time face image of the student in the ideological and political teaching classroom;

[0048] S2. According to the key points on the frontal face image and the real-time face image, obtain the turning angle and the looking-up angle of the real-time face image;

[0049] S3. According to the eye regions on the frontal face image and the real-time face image, obtain the left-eye deviation value and the right-eye deviation value of the real-time face image;

[0050] S4. According to the turning angles, looking-up angles, left-eye deviation values, and right-eye deviation values at consecutive multiple moments, construct a time series of pose change degrees;

[0051] S5. According to the looking-up angles, turning angles, left-eye deviation values, and right-eye deviation values at consecutive multiple moments, as well as the time series of pose change degrees, based on a time series processing neural network, obtain the interest value of the classroom.

[0052] In this embodiment, S2 includes the following sub-steps:

[0053] S21. Extract key points from the frontal face image and the real-time face image respectively. Among them, the key points include: the outer left-eye point, the outer right-eye point, and the tip-of-nose point;

[0054] S22. Take the central coordinates of multiple key points on the frontal face image and the real-time face image respectively;

[0055] S23. Calculate the turning angle according to the gap between the central coordinates of the key points of the real-time face image and the central coordinates of the key points of the frontal face image;

[0056] S24. Construct a normal vector of the reference plane according to multiple key points of the frontal face image;

[0057] S25. Construct a normal vector of the real-time plane according to multiple key points of the real-time face image;

[0058] S26. Calculate the looking-up angle according to the normal vector of the real-time plane and the normal vector of the reference plane.

[0059] In this embodiment, the outer left-eye point, the outer right-eye point, and the tip-of-nose point can be obtained through a facial feature point detection algorithm (such as Dlib, OpenCV tool, or Mediapipe, etc.).

[0060] In this embodiment, the formula for calculating the turning angle in S23 is: , where R t is the turning angle at the t-th moment, arctan2 is the four-quadrant arctangent function, x o is the abscissa of the central coordinate of the key point of the frontal face image, yo is the vertical coordinate of the center of the key point of the frontal face image, x t is the horizontal coordinate of the center of the key point of the real-time face image at the t-th moment, y t is the vertical coordinate of the center of the key point of the real-time face image at the t-th moment, and t is the number of the moment.

[0061] In this embodiment, the formula for calculating the head-up angle in S26 is: , where θ t is the head-up angle at the t-th moment, arccos is the inverse cosine function, h t is the normal vector of the real-time plane at the t-th moment, h o is the normal vector of the reference plane, | | is the modulus operation, and t is the number of the moment.

[0062] Reference plane normal vector: , , ; Real-time plane normal vector: , , ; where, RL o is the vector from the right eye to the left eye in the frontal face image, NL o is the vector from the tip of the nose to the left eye in the frontal face image, RL t is the vector from the right eye to the left eye in the real-time face image at the t-th moment, NL t is the vector from the tip of the nose to the left eye in the real-time face image at the t-th moment, x R,o is the horizontal coordinate of the outer point of the right eye in the frontal face image, y R,o is the vertical coordinate of the outer point of the right eye in the frontal face image, x L,o is the horizontal coordinate of the outer point of the left eye in the frontal face image, y L,o is the vertical coordinate of the outer point of the left eye in the frontal face image, x R,t is the horizontal coordinate of the outer point of the right eye in the real-time face image at the t-th moment, y R,t is the vertical coordinate of the outer point of the right eye in the real-time face image at the t-th moment, x L,t is the horizontal coordinate of the outer point of the left eye in the real-time face image at the t-th moment, y L,t is the vertical coordinate of the outer point of the left eye in the real-time face image at the t-th moment, x N,o is the horizontal coordinate of the tip of the nose point in the frontal face image, y N,o is the vertical coordinate of the tip of the nose point in the frontal face image, x N,t is the horizontal coordinate of the tip of the nose point in the real-time face image at the t-th moment, y N,t is the vertical coordinate of the tip of the nose point in the real-time face image at the t-th moment, is the vector cross product.

[0063] The present invention takes a human face as a two-dimensional plane, extracts the central coordinates of multiple key points (the central abscissa is equal to the average of the abscissas of multiple key points, and the central ordinate is equal to the average of the ordinates of multiple key points), calculates the turning angle according to the gap between the central coordinates of the key points in the real-time face image and the central coordinates of the key points in the frontal face image, and obtains the head-up angle through the normal vector of the reference plane and the normal vector of the real-time plane, so as to reflect the posture activities of students through head-up and turning.

[0064] In this embodiment, S3 includes the following sub-steps:

[0065] S31. Respectively extract the iris region and the sclera region on the frontal face image and the real-time face image;

[0066] S32. On the frontal face image or the real-time face image, splice the iris region and the sclera region to obtain the eyeball region;

[0067] S33. Extract the geometric center coordinates of the eyeball region to obtain the eyeball center;

[0068] S34. Extract the geometric center coordinates of the iris region to obtain the iris center;

[0069] S35. On the frontal face image, calculate the Euclidean distance between the iris center and the eyeball center to obtain the reference distance;

[0070] S36. On the real-time face image, calculate the Euclidean distance between the iris center and the eyeball center to obtain the real-time distance;

[0071] S37. Obtain the left-eye deviation value and the right-eye deviation value according to the real-time distance and the reference distance of the same eye.

[0072] The present invention extracts the pixel values of the pixel points in the iris region and the sclera region (the white part of the eye region) from the frontal face image, calculates the similarity between the pixel values of each pixel point on the real-time face image and the pixel values of the pixel points in the iris region, marks the pixel points with a similarity greater than the similarity threshold as suspected iris pixel points, and the connected region formed by each suspected iris pixel point is the iris region; calculates the similarity between the pixel values of each pixel point on the real-time face image and the pixel values of the pixel points in the sclera region, marks the pixel points with a similarity greater than the similarity threshold as suspected sclera pixel points, and the connected region formed by each suspected sclera pixel point is the sclera region.

[0073] In this embodiment, S33 is specifically: extract the edge contour pixel points of the eyeball region, that is, the contour where the eyeball region contacts the eyelid, and calculate the geometric center of each edge contour pixel point of the eyeball region to obtain the eyeball center.

[0074] Specifically, S34 is as follows: Extract the edge contour pixel points of the iris region, that is, the contour of the circle where the iris region contacts the sclera region, and calculate the geometric center of each edge contour pixel point of the iris region to obtain the iris center.

[0075] In this embodiment, the formula for obtaining the left - eye deviation value in S37 is: , where γ L,t is the left - eye deviation value at the t - th moment, d L,t is the real - time distance of the left eye at the t - th moment, d L,o is the reference distance of the left eye, and D is the Euclidean distance between the outer - lateral point of the left eye and the outer - lateral point of the right eye of the same person's eyes;

[0076] The formula for obtaining the right - eye deviation value is: , where γ R,t is the right - eye deviation value at the t - th moment, d R,t is the real - time distance of the right eye at the t - th moment, d R,o is the reference distance of the right eye.

[0077] The present invention takes the distance of the iris center in the eyeball region of the frontal face image as a reference, then extracts the relative distance of the iris center in the eyeball region of the real - time face image, and takes the difference between the two to measure the deviation of the iris center.

[0078] In this embodiment, for the same moment, respectively determine whether the absolute value of the head - turning angle is less than the head - turning angle threshold, whether the absolute value of the head - raising angle is less than the head - raising angle threshold, whether the absolute value of the left - eye deviation value is less than the deviation threshold, and whether the absolute value of the right - eye deviation value is less than the deviation threshold. If all four conditions are met, the number of parameters that meet the conditions is recorded as 4; if only three conditions are met, the number of parameters that meet the conditions is recorded as 3; if only two conditions are met, the number of parameters that meet the conditions is recorded as 2; if only one condition is met, the number of parameters that meet the conditions is recorded as 1; if all four conditions are not met, the number of parameters that meet the conditions is recorded as 0. Take the number of parameters that meet the conditions as the attitude change degree at this moment, and form a sequence of the attitude change degrees at consecutive multiple moments to obtain the attitude change degree time - series sequence.

[0079] The attitude change degree can intuitively reflect the degree of concentration of students in class. A high attitude change degree (close to 4) indicates that the student's posture is relatively stable and is more likely to be focused on the teaching content; a low attitude change degree (close to 0) indicates that the student's posture changes greatly and the possibility of being distracted is high. Forming a sequence of the attitude change degrees at consecutive multiple moments can show the dynamic process of the student's posture change from the time dimension. It can be used to analyze the change trend of the concentration of students in the entire ideological and political class.

[0080] In this embodiment, the turning angle threshold and the looking-up angle threshold are set to π / 12, that is, 15°. 15° is a relatively small angle range. Taking this as the threshold means allowing students to have a certain degree of natural posture changes. The deviation threshold is set to , where is the distance from the iris center of the same eye on the frontal face image to the outer point of the eye. There are differences in the eye structures of different students. By setting the threshold based on the distance related to their own eye structures (d max ), the individual differences can be better adapted. The turning angle threshold, the looking-up angle threshold, and the deviation threshold are not limited to those described in this embodiment and can be adjusted according to experiments or experience.

[0081] As Figure 2 shown, the time series processing neural network in S5 includes: a fully connected unit, a first LSTM layer, a second LSTM layer, a time series feature fusion layer, a time series dilated convolution unit, and an output layer;

[0082] The fully connected unit includes multiple fully connected layers, and each fully connected layer is used to process the turning angle, the looking-up angle, the left eye deviation value, and the right eye deviation value at one moment;

[0083] The input end of the first LSTM layer is connected to the output end of the fully connected unit; the input end of the second LSTM layer is used to input the time series of posture change degrees; the input ends of the time series feature fusion layer are respectively connected to the output end of the first LSTM layer and the output end of the second LSTM layer, and its output end is connected to the input end of the time series dilated convolution unit; the output end of the time series dilated convolution unit is connected to the input end of the output layer; the output end of the output layer is used as the output end of the time series processing neural network.

[0084] One cell unit in the first LSTM layer processes the output of one fully connected layer, and one cell unit in the second LSTM layer processes one element in the time series of posture change degrees.

[0085] As Figure 3 shown, the fully connected unit includes: multiple fully connected layers, and each fully connected layer processes the turning angle, the looking-up angle, the left eye deviation value, and the right eye deviation value at one moment. The specific expression is: , where y t is the output of the fully connected layer, R t is the turning angle at the t-th moment, θ t is the looking-up angle at the t-th moment, γ L,t is the left eye deviation value at the t-th moment, γ R,t is the right eye deviation value at the t-th moment, w R,t is the weight of R t , w θ,t is the weight of θ t is the weight ofγL,t is the weight of γ L,t , w γR,t is the weight of γ R,t , b t is the bias for the data at the t-th moment.

[0086] It is difficult to accurately judge the student's state with single-dimensional data. Each fully connected layer synthesizes data from 4 dimensions to obtain comprehensive features, and then the first LSTM layer processes the long-term dependencies in the time series data. The second LSTM layer inputs the time series of the attitude change degree to further mine the time series information in the sequence. The first LSTM layer receives the output of the fully connected unit and processes the attitude features of the turning angle, head-up angle, and deviation value integrated at each moment, focusing more on the learning and encoding of specific attitude information in the time dimension; the second LSTM layer inputs the time series of the attitude change degree and pays more attention to the overall time series pattern and long-term dependencies of the attitude change degree. The two capture attitude-related information from different angles. Through feature multiplication and fusion, they can complement each other, integrate information from different perspectives, and obtain a more comprehensive feature representation. Then, through the temporal dilation convolutional unit, the time series of the attitude change degree is observed with a larger "field of view" to mine more complex time series patterns.

[0087] The interest value is the quantization result output after the temporal processing neural network performs fusion analysis on multi-dimensional attitude data (including turning angle, head-up angle, binocular deviation value) and temporal features (time series of attitude change degree), and is used to reflect the student's attention concentration level and interest level in teaching content in the classroom.

[0088] In this embodiment, if the output layer uses a fully connected layer, the calculation formula for the interest value of the classroom is: , where y is the interest value of the classroom, s m is the m-th eigenvalue output by the temporal dilation convolutional unit, w m is the m-th weight in the fully connected layer, b m is the m-th bias in the fully connected layer, L is the number of eigenvalues output by the temporal dilation convolutional unit, and m is a positive integer. In this embodiment, the expression of the temporal feature fusion layer is: , , , where R is the output of the temporal feature fusion layer, G1 is the output of the first LSTM layer, G2 is the output of the second LSTM layer, g 1,1 is the output of the first cell unit in the first LSTM layer, g 1,n is the output of the n-th cell unit in the first LSTM layer, g 1,N is the output of the N-th cell unit in the first LSTM layer, g 2,1 is the output of the first cell unit in the second LSTM layer, g2,n is the output of the n-th cell in the second LSTM layer, g 2,N is the output of the N-th cell in the second LSTM layer, where n is a positive integer and N is the sequence length. is the convolutional layer is element-wise multiplication.

[0089] The present invention performs feature transformation on the outputs of the first LSTM layer and the second LSTM layer through convolution, and then fuses the two by element-wise multiplication. The element-wise multiplication operation can highlight the features that are commonly concerned in the outputs of the two layers.

[0090] Such as Figure 4 shown, the temporal dilation convolutional unit includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a first dilated convolutional layer, a second dilated convolutional layer, a max pooling layer, and an adder A1;

[0091] The input end of the first convolutional layer is respectively connected to the input ends of the second convolutional layer, the third convolutional layer, and the fourth convolutional layer, and serves as the input end of the temporal dilation convolutional unit;

[0092] The output end of the first convolutional layer is connected to the input end of the first dilated convolutional layer; the output end of the second convolutional layer is connected to the input end of the second dilated convolutional layer; the output end of the third convolutional layer is connected to the input end of the max pooling layer; the input ends of the adder A1 are respectively connected to the output ends of the first dilated convolutional layer, the output end of the second dilated convolutional layer, the output end of the max pooling layer, and the output end of the fourth convolutional layer, and its output end serves as the output end of the temporal dilation convolutional unit.

[0093] In this embodiment, the weights and biases in the temporal processing neural network can be trained by the gradient descent method.

[0094] In this embodiment, the convolutional kernel sizes of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are , the convolutional kernel size of the first dilated convolutional layer is , the dilation rate is 1, the convolutional kernel size of the second dilated convolutional layer is , the dilation rate is 2. The pooling window of the max pooling layer is .

[0095] Different dilation rates enable the model to extract features from different scales. The layer with a small dilation rate focuses on local and detailed pose features, and the layer with a large dilation rate captures more global and macroscopic pose patterns. Combining them can obtain a multi-scale pose feature representation.

[0096] By pre-collecting frontal facial images and comparing them with real-time facial images, and combining key point and eye region data, the present invention can accurately obtain multi-dimensional information such as head-turning angle, head-up angle, and binocular deviation value. Compared with traditional single facial feature recognition, the comprehensiveness of data collection is greatly improved; in terms of dynamic evaluation, a time series of pose change degrees is constructed to effectively capture the dynamic change characteristics of students' poses, realizing the dynamic monitoring of students' classroom states.

[0097] The present invention inputs the head-up angle, head-turning angle, left-eye deviation value, and right-eye deviation value at multiple moments, as well as the time series of pose change degrees into a time series processing neural network. The time series of pose change degrees is used to enhance the corresponding features of the head-up angle, head-turning angle, left-eye deviation value, and right-eye deviation value at multiple moments, making up for the defect of isolated single-feature data information, thereby enhancing the temporal correlation and feature complementarity of the data. Through the fusion enhancement, the neural network can capture richer and more representative information on students' classroom states, improving the accuracy of the time series processing neural network evaluation and solving the problem of low accuracy in teaching situation evaluation existing in the prior art.

[0098] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A sentiment analysis method for ideological and political teaching, characterized in that: The following steps are included: S1. Pre-collect an image of a student facing the blackboard to obtain a frontal facial image, and collect real-time facial images of students in the ideological and political teaching classroom; S2, obtaining the head turning angle and head raising angle of the real-time facial image according to the key points on the frontal facial image and the real-time facial image; S3, obtaining a left eye deviation value and a right eye deviation value of the real-time facial image according to the eye areas on the frontal facial image and the real-time facial image; S4, constructing a posture change time series sequence according to the head turning angle, head raising angle, left eye deviation value and right eye deviation value at multiple consecutive moments; In S4, for the same moment, it is judged whether the absolute value of the head turning angle is less than the head turning angle threshold, whether the absolute value of the head raising angle is less than the head raising angle threshold, whether the absolute value of the left eye deviation value is less than the deviation threshold, and whether the absolute value of the right eye deviation value is less than the deviation threshold. If all four conditions are met, the number of parameters that meet the conditions is recorded as 4, if only three conditions are met, the number of parameters that meet the conditions is recorded as 3, if only two conditions are met, the number of parameters that meet the conditions is recorded as 2, if only one condition is met, the number of parameters that meet the conditions is recorded as 1, if all four conditions are not met, the number of parameters that meet the conditions is recorded as 0, and the number of parameters that meet the conditions is used as the posture change degree at the moment, and the posture change degrees of multiple consecutive moments constitute a sequence to obtain a posture change degree time series sequence; S5. According to the head-up angle, head-turning angle, left eye deviation value and right eye deviation value at multiple consecutive moments, and the time series sequence of posture change degree, the interest value of the class is obtained based on the time series processing neural network.

2. The method for analyzing the emotions of ideological and political teaching according to claim 1 is characterized in that: S2 includes the following sub-steps: S21, extracting key points from the frontal facial image and the real-time facial image respectively, wherein the key points include: a left eye outer point, a right eye outer point and a nose tip point; S22, respectively obtaining center coordinates of a plurality of key points on the frontal facial image and the real-time facial image; S23, calculating the head turning angle according to the difference between the center coordinates of the key points of the real-time facial image and the center coordinates of the key points of the frontal facial image; S24, constructing a reference plane normal vector according to a plurality of key points of the frontal facial image; S25, constructing a real-time plane normal vector according to multiple key points of the real-time facial image; S26. Calculate the head-up angle according to the real-time plane normal vector and the reference plane normal vector.

3. The method for analyzing the emotions of ideological and political teaching according to claim 2 is characterized in that: The formula for calculating the head turning angle in S23 is: , where R t is the head turning angle at the tth moment, arctan2 is the four-quadrant inverse tangent function, x o is the horizontal coordinate of the center of the key point of the frontal facial image, y o is the vertical coordinate of the key point center of the frontal facial image, x t is the horizontal coordinate of the key point center of the real-time facial image at time t, y t is the central ordinate of the key point of the real-time facial image at the tth moment, and t is the number of the moment.

4. The method for analyzing the emotions of ideological and political education according to claim 2 is characterized in that: The formula for calculating the head-up angle in S26 is: , where θ t is the head-up angle at the tth moment, arccos is the inverse cosine function, h t is the real-time plane normal vector at the tth moment, h o is the normal vector of the reference plane, || is the modulo operation, and t is the time number.

5. The method for analyzing the emotions of ideological and political teaching according to claim 1 is characterized in that: S3 includes the following sub-steps: S31, extracting the iris region and the sclera region on the frontal facial image and the real-time facial image respectively; S32, on the frontal facial image or the real-time facial image, splicing the iris region and the sclera region to obtain an eyeball region; S33, extracting the geometric center coordinates of the eyeball area to obtain the eyeball center; S34, extracting the geometric center coordinates of the iris area to obtain the iris center; S35, on the frontal facial image, calculating the Euclidean distance between the iris center and the eyeball center to obtain a reference distance; S36, calculating the Euclidean distance between the iris center and the eyeball center on the real-time facial image to obtain a real-time distance; S37. Obtain a left eye deviation value and a right eye deviation value according to the real-time distance and the reference distance of the same eye.

6. The method for analyzing the emotions of ideological and political teaching according to claim 5 is characterized in that: The formula for obtaining the left eye deviation value in S37 is: , where γ L,t is the left eye deviation value at the tth moment, d L,t is the real-time distance of the left eye at time t, d L,o is the reference distance of the left eye, and D is the Euclidean distance between the outer points of the left eye and the outer points of the right eye of the same person; The formula for obtaining the right eye deviation value is: , where γ R,t is the right eye deviation value at the tth moment, d R,t is the real-time distance of the right eye at time t, d R,o is the reference distance for the right eye.

7. The method for analyzing the emotions of ideological and political education according to claim 1 is characterized in that: The time series processing neural network in S5 includes: fully connected units, the first LSTM layer, the second LSTM layer, the time series feature fusion layer, the time series expansion convolution unit and the output layer; The fully connected unit includes multiple fully connected layers, each of which is used to process the head turning angle, head raising angle, left eye deviation value and right eye deviation value at a moment; The input end of the first LSTM layer is connected to the output end of the fully connected unit; the input end of the second LSTM layer is used to input the posture change degree time series sequence; the input end of the time series feature fusion layer is respectively connected to the output end of the first LSTM layer and the output end of the second LSTM layer, and its output end is connected to the input end of the time series expansion convolution unit; the output end of the time series expansion convolution unit is connected to the input end of the output layer; the output end of the output layer serves as the output end of the time series processing neural network.

8. The method for analyzing the emotions of ideological and political education according to claim 7 is characterized in that: The expression of the temporal feature fusion layer is: , , , where R is the output of the temporal feature fusion layer, G1 is the output of the first LSTM layer, G2 is the output of the second LSTM layer, and g 1,1 is the output of the first cell in the first LSTM layer, g 1,n is the output of the nth cell in the first LSTM layer, g 1,N is the output of the Nth cell in the first LSTM layer, g 2,1 is the output of the first cell in the second LSTM layer, g 2,n is the output of the nth cell in the second LSTM layer, g 2,N is the output of the Nth cell in the second LSTM layer, n is a positive integer, N is the sequence length, for Convolutional layers, is element-wise multiplication.

9. The method for analyzing emotions in ideological and political education according to claim 7 is characterized in that: The temporal expansion convolution unit includes: a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a first expansion convolution layer, a second expansion convolution layer, a maximum pooling layer and an adder A1; The input end of the first convolutional layer is connected to the input end of the second convolutional layer, the input end of the third convolutional layer, and the input end of the fourth convolutional layer respectively, and serves as the input end of the time-series expansion convolution unit; The output end of the first convolution layer is connected to the input end of the first dilated convolution layer; the output end of the second convolution layer is connected to the input end of the second dilated convolution layer; the output end of the third convolution layer is connected to the input end of the maximum pooling layer; the input end of the adder A1 is respectively connected to the output end of the first dilated convolution layer, the output end of the second dilated convolution layer, the output end of the maximum pooling layer and the output end of the fourth convolution layer, and its output end serves as the output end of the temporal dilated convolution unit.

Images