Student attention and participation behavior monitoring method for terrace classroom teaching
Through comprehensive face detection and AdaBoost algorithm, combined with high-end camera equipment, the problem of poor head posture detection effect in step classrooms is solved, effectively monitoring students' attention and participation behaviors in large-scale classrooms is achieved, and the detection accuracy is improved sex and coverage.
Patent Information
- Application Number
- CN202410963153.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
AI Technical Summary
In a stepped classroom with a large depth, the head posture detection algorithm using ordinary monocular cameras is not effective, especially the detection effect of students in the back row is low, resulting in serious missed detection and missed detection, making it difficult to achieve a large-scale full coverage classroom Teaching attention and participation in behavior monitoring.
A comprehensive method of student attention and participation behavior monitoring, including face detection methods based on important facial features, template matching, and color information, combined with Haar features and AdaBoost algorithm for face and eye detection, and built using high-end camera equipment. The attention detection system realizes effective monitoring of students' attention and participation behaviors in the ladder classroom through face detection and positioning, camera control and focus, attention detection and reminder modules.
Effectively improve the accuracy of head posture detection for students in the back row, reduce false detection and missed detection, realize full coverage monitoring of students' classroom attention and participation behavior in large-scale ladder classrooms, and provide reliable data for teaching research. support.
Smart Images

Figure CN118918528A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of monitoring students' classroom attention and participation behavior, and in particular to a method for monitoring students' classroom attention and participation behavior for teaching in a stepped classroom. Background Art
[0002] At present, students' head posture detection algorithms are usually used to study classroom attention. However, in real classroom environments, especially in lecture halls with a large depth, the effect of head posture detection algorithms using ordinary monocular cameras is not ideal, which is specifically manifested in false detection and missed detection of head postures in the back row, especially in medium-sized classrooms with 100 to 140 people, and large lecture halls that can accommodate more than 160 people. In medium-sized classrooms and large lecture halls with a large depth, the resolution of students' heads in the front and back rows is quite different, with the front row being relatively clear and the back row being relatively blurry.
[0003] The target detection-based method does not perform well in large classrooms, and the false detection and missed detection in the back row are more serious. The reason for this is that compared with the back row, the front row students have richer head pixel information and higher resolution, which dominates the network training process. This will cause the model in the large classroom to have a better head posture detection accuracy for the front row students than the back row students.
[0004] In domestic universities, due to the high teacher-student ratio, lecture halls are the main form of teaching, and it is difficult to achieve small class teaching. Therefore, in order to achieve large-scale and full-coverage continuous analysis of classroom teaching, the measurement of students' classroom attention and participation behavior in lecture halls must be properly addressed.
[0005] Therefore, it is necessary to propose a method for monitoring students' attention and participation behavior in lecture halls to solve the above problems. Summary of the invention
[0006] The main purpose of the present invention is to provide a method for monitoring student attention and participation behavior for teaching in a stepped classroom, which can effectively solve the problems in the background technology.
[0007] To achieve the above object, the technical solution adopted by the present invention is:
[0008] A method for monitoring student attention and participation behavior in a stepped classroom, comprising the following steps:
[0009] S1: Optimization and characteristics of student face detection: including face detection methods based on important facial features, face detection methods based on template matching, and face detection methods based on color information. Among them, the face detection method based on important facial features: the distance between the students' eyes is recorded as s, and the vertical distance from the eyes to the mouth is 0.8-1.3 times s. The numerical description of s is used as the rule for the distribution of local facial features to perform face detection;
[0010] Face detection method based on template matching: Establish a standard student face template library, which is composed of sub-templates containing local student face features. Perform a global search on the input image of a single student face, and calculate the correlation coefficients with different parts of the standard face template for image windows of different sizes. Use a pre-set threshold to determine whether the student contains a face in the image window.
[0011] Face detection method based on color information: Establish and use the skin color model to segment skin color pixels, remove non-skin color pixels, and then segment the possible student face area according to the similarity of skin color pixels in color space and the correlation of spatial position. Finally, use other features of the face, including the aspect ratio, for verification, determine the final face area, and then perform tilt correction and normalization processing;
[0012] S2: Face statistics based on sample learning: Sample learning is performed based on Haar features. The integral graph of student face features is used to obtain Haar feature values. Haar features are used as input for the training process. The sample learning algorithm is used to select Haar features with strong classification capabilities to form their corresponding weak classifiers. The AdaBoost algorithm is used to combine several weak classifiers into a strong classifier. Several strong classifiers are connected in series to form a cascade classifier. The number of layers of the cascade classifier is adjusted to change the detection rate and false positive rate of the entire cascade classifier.
[0013] S3: Human eye detection based on AdaBoost algorithm: Haar wavelet transform features are introduced to replace the original pixel values as the input of the learning algorithm to reduce the intra-class differences. The feature set of the AdaBoost algorithm needs to select key feature quantities that meet three conditions: only the weighted sum of two rectangular areas, the weights of the rectangular areas have opposite signs to compensate for the area difference between the two rectangular areas, and the features extracted by Haar wavelet transform that imitate the human visual channel;
[0014] S4: training the strong classifier based on the algorithm in step S3: adjusting the detection rate and false positive rate of each strong classifier and the number of layers of the final cascade classifier at the same time;
[0015] S5: Construction of attention detection system: including high-end camera equipment, which includes rotation and focusing functions, and is equipped with corresponding interfaces for connecting and controlling with computers, and also equipped with SDK for secondary development. The attention detection system can be divided into three modules: face detection and positioning module, camera control and focusing module, and attention detection and reminder module;
[0016] The face detection and positioning module inputs the real-time image of the lecture hall collected from the camera, and outputs the face position of each student in the corresponding image. The original digital image is obtained by taking a snapshot of the lecture hall through a high-end camera device.
[0017] The input of the camera control and focus module is the output of the face detection and positioning module. Therefore, the input is a set of rectangles of detected faces, and the area is limited to a rectangular area 2-4 times the size of the detected face area.
[0018] The attention detection and reminder module adopts the AdaBoost-based eye detection algorithm in the attention detection module, and the training samples are open eye samples centered on the pupil and bounded by the upper edge of the eyebrows.
[0019] Preferably, in step S1, the template library is based on an elastic template, based on a parameterized adjustable template set according to the shape of the detected object and a corresponding energy function, the energy function is designed based on prior knowledge such as the grayscale information of the image and the contour of the detected object, and includes the following operating steps when used: moving the adjustable template in the image to be detected, and dynamically adjusting its parameters, calculating the energy function, when the energy function reaches the minimum value 1, the shape of the adjustable template determined by its position and parameters achieves the best fit with the face shape, thereby detecting the student's face.
[0020] Preferably, in step S1, the color space includes an RGB color space and an HSV color space, wherein the RGB color space is divided into three bands: red, green and blue, and the normalized RGB color space is changed by the following formula: In the HSV color space, H represents hue and S represents saturation. The conversion equation from RGB to HSV color space is:
[0021]
[0022] Preferably, in step S2, the detection process is the opposite process of the training process. During training, all normalized training samples, including size normalization and brightness normalization, are subjected to rectangular scanning, including movable scanning of the same size and scanning of variable size. Finally, the corresponding weak classifier is obtained by calculation of all samples. During the detection process, the obtained weak classifier is first used to scan the image to be detected, and the parameters of the weak classifier threshold are adjusted accordingly when the size increases.
[0023] Preferably, in step S5, the preset position of the high-end camera device must ensure that the student's face can be covered and the pixel size of the student's face must be larger than the minimum detection size in the face detection system.
[0024] Beneficial Effects
[0025] Compared with the prior art, the present invention provides a method for monitoring student attention and participation behavior in a stepped classroom, which has the following beneficial effects:
[0026] 1. The method for monitoring students' attention and participation behavior in lecture hall teaching can effectively solve the problem of attention measurement in traditional lecture-type classroom teaching environment. It includes head posture detection algorithm and attention model. The head posture detection algorithm provides basic data for the whole scheme. The attention model is a composite structure built on this data, which is used to characterize the abstract concept of classroom attention and provide data support for subsequent teaching research. In view of the fact that the head posture detection effect in the back row of the lecture hall is significantly lower than that in the front row, the lecture hall plans to adopt a two-stage model. First, a target detection-based method is used to obtain the student head area, and then the head area is divided into small-scale head area and normal-scale head area according to the size of the head area, and the classifiers of the corresponding scale are used to classify the head posture. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 It is a block diagram of the attention detection system of the present invention. DETAILED DESCRIPTION
[0028] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.
[0029] like Figure 1 As shown, a method for monitoring student attention and participation behavior for teaching in a stepped classroom includes the following steps:
[0030] S1: Optimization and characteristics of student face detection: including face detection methods based on important facial features, face detection methods based on template matching, and face detection methods based on color information. Among them, the face detection method based on important facial features: the distance between the students' eyes is recorded as s, and the vertical distance from the eyes to the mouth is 0.8-1.3 times s. The numerical description of s is used as the rule for the distribution of local facial features to perform face detection;
[0031] Face detection method based on template matching: establish a standard student face template library, which is composed of sub-templates containing local student face features, perform a global search on the input image of a single student face, and calculate the correlation coefficients with different parts of the standard face template corresponding to image windows of different sizes. Use a preset threshold to determine whether the student contains a face in the image window. The template library is based on an elastic template, which is based on a parameterized adjustable template set according to the shape of the object to be detected and the corresponding energy function. The energy function is designed based on prior knowledge such as the grayscale information of the image and the contour of the object to be detected. The following steps are included in its use: move the adjustable template in the image to be detected, and dynamically adjust its parameters, calculate the energy function, and when the energy function reaches the minimum value of 1, the adjustable template shape determined by its position and parameters achieves the best fit with the face shape, thereby detecting the student face;
[0032] Face detection method based on color information: Establish and use the skin color model to segment skin color pixels, remove non-skin color pixels, and then segment the possible student face area according to the similarity of skin color pixels in color space and the correlation of spatial position. Finally, use other features of the face, including the aspect ratio, for verification, determine the final face area, and then perform tilt correction and normalization processing;
[0033] The color space includes RGB color space and HSV color space. The RGB color space is divided into three bands: red, green, and blue. The normalized RGB color space is changed by the following formula: In the HSV color space, H represents hue and S represents saturation. The conversion equation from RGB to HSV color space is:
[0035] S2: Face statistics based on sample learning: Sample learning is performed based on Haar features. The integral graph of student face features is used to obtain Haar feature values. Haar features are used as input for the training process. The sample learning algorithm is used to select Haar features with strong classification capabilities to form their corresponding weak classifiers. The AdaBoost algorithm is used to combine several weak classifiers into a strong classifier. Several strong classifiers are connected in series to form a cascade classifier. The number of layers of the cascade classifier is adjusted to change the detection rate and false positive rate of the entire cascade classifier.
[0036] The detection process is the opposite of the training process. During training, all normalized training samples, including size normalization and brightness normalization, are scanned in rectangles, including mobile scanning of the same size and scanning of variable sizes. Finally, the corresponding weak classifier is obtained through calculation of all samples. In the detection process, the obtained weak classifier is first used to scan the image to be detected, and the parameters of the weak classifier threshold are adjusted accordingly when the size is increased.
[0037] S3: Human eye detection based on AdaBoost algorithm: Haar wavelet transform features are introduced to replace the original pixel values as the input of the learning algorithm to reduce the intra-class differences. The feature set of the AdaBoost algorithm needs to select key feature quantities that meet three conditions: only the weighted sum of two rectangular areas, the weights of the rectangular areas have opposite signs to compensate for the area difference between the two rectangular areas, and the features extracted by Haar wavelet transform that imitate the human visual channel;
[0038] S4: training the strong classifier based on the algorithm in step S3: adjusting the detection rate and false positive rate of each strong classifier and the number of layers of the final cascade classifier at the same time;
[0039] S5: Construction of attention detection system: including high-end camera equipment, which includes rotation and focusing functions, and is equipped with corresponding interfaces for connecting and controlling with computers, and also equipped with SDK for secondary development. The attention detection system can be divided into three modules: face detection and positioning module, camera control and focusing module, attention detection and reminder module. The preset position of the high-end camera equipment must ensure that the student's face can be covered, and the pixel size of the student's face must be larger than the minimum detection size in the face detection system;
[0040] The face detection and positioning module inputs the real-time image of the lecture hall collected from the camera, and outputs the face position of each student in the corresponding image. The original digital image is obtained by taking a snapshot of the lecture hall through a high-end camera device.
[0041] The input of the camera control and focus module is the output of the face detection and positioning module. Therefore, the input is a set of rectangles of detected faces, and the area is limited to a rectangular area 2-4 times the size of the detected face area.
[0042] The attention detection and reminder module adopts the AdaBoost-based eye detection algorithm in the attention detection module, and the training samples are open eye samples centered on the pupil and bounded by the upper edge of the eyebrows.
[0043] Based on this method, human eye detection is performed, and the following is the detection table:
[0044]
[0045] It can be seen that the classifier trained with pupil-centered samples has the best detection effect, the detector trained with samples featuring the eye's circumscribed rectangle has the second best detection effect, and the binocular detection effect is the worst. However, the detection effect of binocular samples is obviously worse than the other two classifications. The main reason is that it is not as sensitive to the opening and closing of the eyes as the other two classifiers, and the characteristics of the binoculars are more easily disturbed by the angle of the hair and face compared to the other two classifiers.
[0046] This method can effectively solve the problem of attention measurement in traditional lecture-type classroom teaching environments. It includes a head posture detection algorithm and an attention model. The head posture detection algorithm provides basic data for the entire solution. The attention model is a composite structure built on this data, which is used to characterize the abstract concept of classroom attention and provide data support for subsequent teaching research. In view of the fact that the head posture detection effect in the back row of the stepped classroom is significantly lower than that in the front row, a two-stage model is proposed for the stepped classroom. First, a target detection-based method is used to obtain the student head area, and then the head area is divided into a small-scale head area and a normal-scale head area according to the size of the head area, and the classifiers of the corresponding scale are used to classify the head posture.
[0047] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for monitoring student attention and participation behavior in a stepped classroom, characterized in that: The steps include: S1: Optimization and characteristics of student face detection: including face detection methods based on important facial features, face detection methods based on template matching, and face detection methods based on color information. Among them, the face detection method based on important facial features: the distance between the students' eyes is recorded as s, and the vertical distance from the eyes to the mouth is 0.8-1.3 times s. The numerical description of s is used as the rule for the distribution of local facial features to perform face detection; Face detection method based on template matching: Establish a standard student face template library, which is composed of sub-templates containing local student face features. Perform a global search on the input image of a single student face, and calculate the correlation coefficients with different parts of the standard face template for image windows of different sizes. Use a pre-set threshold to determine whether the student contains a face in the image window. Face detection method based on color information: Establish and use the skin color model to segment skin color pixels, remove non-skin color pixels, and then segment the possible student face area according to the similarity of skin color pixels in color space and the correlation of spatial position. Finally, use other features of the face, including the aspect ratio, for verification, determine the final face area, and then perform tilt correction and normalization processing; S2: Face statistics based on sample learning: Sample learning is performed based on Haar features. The integral graph of student face features is used to obtain Haar feature values. Haar features are used as input for the training process. The sample learning algorithm is used to select Haar features with strong classification capabilities to form their corresponding weak classifiers. The AdaBoost algorithm is used to combine several weak classifiers into a strong classifier. Several strong classifiers are connected in series to form a cascade classifier. The number of layers of the cascade classifier is adjusted to change the detection rate and false positive rate of the entire cascade classifier. S3: Human eye detection based on AdaBoost algorithm: Haar wavelet transform features are introduced to replace the original pixel values as the input of the learning algorithm to reduce the intra-class differences. The feature set of the AdaBoost algorithm needs to select key feature quantities that meet three conditions: only the weighted sum of two rectangular areas, the weights of the rectangular areas have opposite signs to compensate for the area difference between the two rectangular areas, and the features extracted by Haar wavelet transform that imitate the human visual channel; S4: training the strong classifier based on the algorithm in step S3: adjusting the detection rate and false positive rate of each strong classifier and the number of layers of the final cascade classifier at the same time; S5: Construction of attention detection system: including high-end camera equipment, which includes rotation and focusing functions, and is equipped with corresponding interfaces for connecting and controlling with computers, and also equipped with SDK for secondary development. The attention detection system can be divided into three modules: face detection and positioning module, camera control and focusing module, and attention detection and reminder module; The face detection and positioning module inputs the real-time image of the lecture hall collected from the camera, and outputs the face position of each student in the corresponding image. The original digital image is obtained by taking a snapshot of the lecture hall through a high-end camera device. The input of the camera control and focus module is the output of the face detection and positioning module. Therefore, the input is a set of rectangles of detected faces, and the area is limited to a rectangular area 2-4 times the size of the detected face area. The attention detection and reminder module adopts the AdaBoost-based eye detection algorithm in the attention detection module, and the training samples are open eye samples centered on the pupil and bounded by the upper edge of the eyebrows.
2. A method for monitoring student attention and participation behavior in a stepped classroom according to claim 1, characterized in that: In the step S1, the template library is based on an elastic template, a parameterized adjustable template set according to the shape of the detected object and a corresponding energy function. The energy function is designed based on prior knowledge such as the grayscale information of the image and the contour of the detected object. When used, it includes the following operating steps: moving the adjustable template in the image to be detected, and dynamically adjusting its parameters, calculating the energy function, when the energy function reaches the minimum value 1, the shape of the adjustable template determined by its position and parameters achieves the best fit with the face shape, thereby detecting the student's face.
3. The method for monitoring student attention and participation behavior in a stepped classroom according to claim 1, characterized in that: In step S1, the color space includes an RGB color space and an HSV color space, wherein the RGB color space is divided into three bands: red, green, and blue. The normalized RGB color space is changed by the following formula: In the HSV color space, H represents hue and S represents saturation. The conversion equation from RGB to HSV color space is:
4. The method for monitoring student attention and participation behavior in a stepped classroom according to claim 1, characterized in that: In step S2, the detection process is the opposite process of the training process. During training, all normalized training samples, including size normalization and brightness normalization, are subjected to rectangular scanning, including movable scanning of the same size and scanning of variable size. Finally, the corresponding weak classifier is obtained by calculation of all samples. During the detection process, the obtained weak classifier is first used to scan the image to be detected, and the parameters of the weak classifier threshold are adjusted accordingly when the size increases.
5. The method for monitoring student attention and participation behavior in a stepped classroom according to claim 1, characterized in that: In step S5, the preset position of the high-end camera device must ensure that the student's face can be covered and the pixel size of the student's face must be larger than the minimum detection size in the face detection system.
Citation Information
Patent Citations
Human face detection and tracking method based on Gaussian skin color model and feature analysis
CN102324025A
Method and system for detecting classroom attention of student
CN104517102A
Online education scene student attention recognition method for CPU operation optimization
CN112597888A
Classroom analysis system and method based on face recognition technology
CN113450369A
Identity recognition method based on AI education system
CN114419711A
