Classroom attention monitoring method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a classroom attention monitoring method and device, equipment and a storage medium. In a teaching mode, a first video shot by a first camera capable of shooting a human face and a second video shot by a second camera capable of shooting a full view of a classroom are utilized to obtain a moving path of a head of a teacher and a sight path of sight of each student; the classroom attention is monitored through the similarity between the moving path and the sight line path, whether the sight line of the student follows the movement of the teacher in the teaching mode can be effectively monitored, the attention of the student on the teaching content is reflected by following the movement of the teacher, and the attention monitoring accuracy is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of educational information technology, and in particular to a classroom attention monitoring method and device, equipment and storage medium. Background Art

[0002] In the classroom, the effectiveness of teaching is closely related to the students' attention. Teachers not only have to teach the course, but also pay attention to the students' attention. It is inevitable that they will overlook the attention of some students. In order to improve the quality of teaching, it is particularly important to pay attention to students' attention.

[0003] However, the existing attention monitoring methods all have low accuracy. Therefore, how to improve the accuracy of attention monitoring of students in the classroom is an urgent problem to be solved. Summary of the Invention

[0004] The main purpose of the present invention is to provide a classroom attention monitoring method and device, equipment and storage medium, which can solve the problem of low accuracy of monitoring students' attention in the classroom in the prior art.

[0005] To achieve the above objectives, the present invention provides a first aspect of a method for monitoring classroom attention, the method comprising:

[0006] If it is detected that the current classroom environment has entered a teaching mode, a first video of the classroom captured by a first camera is obtained, and a second video captured by a second camera during the same time period as the first video is obtained, wherein the first camera is set in front of the student seats in the classroom, and the second camera is located above the student seats. The teaching mode refers to a classroom learning mode in which the teacher teaches knowledge points;

[0007] Performing face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, performing head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system of the three-dimensional space formed by the classroom;

[0008] Determine a movement path of the teacher's head in the first coordinate system based on the second position; determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0009] The attention of each student is monitored according to the similarity between the moving path of the teacher and the sight path of each student.

[0010] Optionally, determining the movement path of the teacher's head in the first coordinate system according to the second position includes:

[0011] Obtaining the height information of the teacher and the height information of the teaching area of ​​the classroom;

[0012] Determining the head position information of the teacher in the first coordinate system at each time point according to the height information of the teacher, the height information of the teaching area, and the second position;

[0013] The moving path of the teacher is formed by using the head position information at each time point in the first coordinate system.

[0014] Optionally, determining the sight path of each student in the first coordinate system according to the face orientation angle, the first position, and the second position includes:

[0015] Obtaining the height of each student and the height of the desk and chair, and determining the height of each student's eye area when they are in a seated position based on the height of each student and the height of the desk and chair;

[0016] determining the coordinates of the eye parts of each student in the first coordinate system at each time point according to the height of the eye parts of each student and the first position;

[0017] Determining the coordinates of the sight points of each student at each time point in the first coordinate system according to the coordinates of the eye parts of each student at each time point in the first coordinate system, the face orientation angle, and the second position;

[0018] The sight path of each student is determined using the coordinates of the sight point at each time point in the first coordinate system.

[0019] Optionally, determining the coordinates of the sight points of each student in the first coordinate system at each time point based on the coordinates of the eye parts of each student in the first coordinate system, the face orientation angle, and the second position includes:

[0020] Determine the sight line vector of each student in the first coordinate system according to the coordinates of the students' eyes at the same time point and the face orientation angle;

[0021] determining a target plane in the first coordinate system that is parallel to a plane formed by YZ axes of the first coordinate system and includes the second position;

[0022] The intersection point of the sight line vector and the target plane is used as the coordinate of the student's sight line point in the first coordinate system.

[0023] Optionally, performing face recognition processing on the first video to obtain the face orientation angle of each student's face region in each frame of the first video includes:

[0024] Traversing the first video, performing face recognition on the traversed target video frame using a face deep learning model, and extracting images of each face region contained in the target video frame;

[0025] Using the OpenCV cross-platform computer vision library to perform key point recognition on each facial region image, determining a first set of coordinate points of the key points of each facial region image in a camera coordinate system, wherein the key point set includes: a left eye center point, a right eye center point, a chin center point, and an eyebrow center point;

[0026] Performing coordinate transformation on the first set of key points of each facial region image to obtain a second set of coordinates in a second coordinate system;

[0027] The coordinates of the left eye center point, right eye center point, chin center point and eyebrow center of each facial area in the second set are calculated to obtain the facial orientation angle of each facial area in the second coordinate system, and the directions of the second coordinate system are parallel to the directions of the first coordinate system.

[0028] Optionally, calculating the coordinates of the left eye center point, the right eye center point, the chin center point, and the eyebrow center of each facial region in the second set to obtain the facial orientation angle of each facial region in the second coordinate system includes:

[0029] Determine a first line segment between the coordinates of the left eye center point and the right eye center point, and determine a second line segment between the coordinates of the chin center point and the coordinates of the eyebrow center point;

[0030] Determine a first plane perpendicular to the first line segment and intersecting the midpoint of the first line segment, and determine a second plane perpendicular to the second line segment and including the midpoint of the first line segment;

[0031] A three-dimensional vector formed by a line where the first plane intersects the second plane and the midpoint of the first line segment is taken as the starting point and is directed toward the outside of the face is used as the face orientation angle.

[0032] Optionally, monitoring the attention of each student based on the similarity between the teacher's movement path and the student's sight path includes:

[0033] Calculating the similarity between the moving path and each student's sight path respectively to obtain a similarity value of each student's sight tracking;

[0034] If the target similarity value is greater than or equal to a preset threshold, determining that the student corresponding to the target similarity is focused;

[0035] If the target similarity is less than a preset threshold, it is determined that the student corresponding to the target similarity value is not paying attention.

[0036] To achieve the above objectives, the present invention provides a second aspect of a classroom attention monitoring device, comprising:

[0037] an acquisition module, configured to, upon detecting that the current classroom environment has entered a teaching mode, acquire a first video of the classroom captured by a first camera, and acquire a second video captured by a second camera during the same time period as the first video, wherein the first camera is located in front of a student seat in the classroom, and the second camera is located above the student seat. The teaching mode refers to a classroom learning mode in which a teacher lectures on knowledge points;

[0038] a first determination module configured to perform face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, and to perform head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system formed by the three-dimensional space of the classroom;

[0039] a second determining module, configured to determine a movement path of the teacher's head in the first coordinate system based on the second position; and determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0040] A monitoring module is used to monitor the attention of each student based on the similarity between the teacher's movement path and the student's sight path.

[0041] To achieve the above objectives, the third aspect of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes each step of the method provided in the first aspect.

[0042] To achieve the above-mentioned objectives, the fourth aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes each step in the method provided in the first aspect.

[0043] The embodiments of the present invention have the following beneficial effects:

[0044] The present invention provides a classroom attention monitoring method. In this method, if it is detected that the current classroom environment enters the teaching mode, a first video of the classroom taken by a first camera is obtained, and a second video taken by a second camera in the same time period as the first camera is obtained. The first camera is set in front of the student seats in the classroom, so that the first camera faces the students and can capture the students' faces. The second camera is located above the student seats, so that the second camera can capture the whole picture of the classroom from a bird's-eye view. In addition, the teaching mode refers to a classroom learning mode in which the teacher teaches knowledge points, indicating that students all need to look up at the teacher and listen to the teacher's teaching content. In this mode, students need to follow the teacher's position in order to better understand the teaching content. Further, face recognition is performed on the first video. Processing is performed to obtain the face orientation angle of each student's face area in each video frame of the first video, and head recognition processing is performed on the second video to determine the first position of the head of each student located in the student seat area corresponding to the time point of each video frame of the second video, and the second position of the head of the teacher located in the lecture area corresponding to the time point of each video frame of the second video, the first position and the second position both include x values ​​and y values ​​in a first coordinate system, and the first coordinate system is a three-dimensional coordinate system of a three-dimensional space formed by a classroom, the moving path of the teacher's head in the first coordinate system is determined according to the second position, the sight path of each student's video in the first coordinate system is determined according to the face orientation angle, the first position and the second position, and the attention of each student is monitored according to the similarity of the moving path and the sight path. By using the first video captured by the first camera that can capture people's faces and the second video captured by the second camera that can capture the entire classroom in the teaching mode, the movement path of the teacher's head and the line of sight path of each student are obtained, and the classroom attention is monitored by the similarity between the movement path and the line of sight path. It is possible to effectively monitor whether the students' eyes follow the movement of the teacher in the teaching mode, so as to reflect the attention to the teaching content by following the movement of the teacher, thereby effectively improving the accuracy of attention monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] in:

[0047] Figure 1 Schematic diagram of the flow of a method for monitoring classroom attention in an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of a flow chart of a method for determining a face orientation angle in an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of a flow chart of a method for determining a moving path of a teacher's head in a first coordinate system according to a second position in an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of the structure of the program module of the classroom attention monitoring device in an embodiment of the present invention;

[0051] Figure 5 4 is a structural block diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] To better understand the technical solutions of this application, the following describes a scenario in which the technical solutions of this application are applied. This scenario can be a school teaching scenario, which can be applied to daily teaching work in schools, such as teaching work in classrooms of different grades, such as elementary schools, junior high schools, high schools, or universities. In order to monitor students' attention in class, it is necessary to set up cameras in the classroom, including a first camera and a second camera. The first camera is set in front of all student seats in the classroom. In other words, students usually face the blackboard or display screen in class. In order to effectively capture students' faces, the first camera can be set in the area where the blackboard or display screen is located or in an adjacent area, so that the first camera can capture images of the classroom from the front. Furthermore, the number of first cameras arranged in the classroom can be one or more. For example, the length of the plane where the blackboard or display screen is located in the classroom can be divided into two equal sections, that is, the middle position and the end positions on both sides are determined in the length direction. The first cameras are respectively set at the middle position and the end positions on both sides. The images captured by the multiple first cameras can be integrated and recognized to obtain more complete image information of each student, thereby better realizing face recognition. In addition, the second camera can be located above the student seat, specifically, it can be located on the ceiling of the classroom, so that the second camera can capture the entire classroom from a bird's-eye view. Furthermore, the area in the classroom can be divided into two parts, one part is the student desk and chair area and the lecture area. The second camera can be located in the middle of the ceiling of the entire classroom, or it can be located in the middle of the student desk and chair area. It should be noted that in actual applications, the positions of the first camera and the second camera can be set according to specific needs, and the number of the first camera and the second camera can be set, which is not limited here. In addition, a control device is also provided, which is in communication connection with the first camera and the second camera, so that the video captured by the first camera and the second camera can be obtained.

[0054] See also Figure 1 , is a flow chart of a classroom attention monitoring method in an embodiment of the present application, the method comprising:

[0055] 101. If it is detected that the current classroom environment has entered a teaching mode, a first video of the classroom captured by a first camera is obtained, and a second video captured by a second camera during the same time period as the first video is obtained. The teaching mode refers to a classroom learning mode in which a teacher teaches knowledge points.

[0056] 102. Perform face recognition processing on the first video to obtain the face orientation angle of each student's face region in each video frame of the first video, perform head recognition processing on the second video to determine the first position of each student's head located in the student desk and chair area at a corresponding time point in each video frame of the second video, and the second position of the teacher's head located in the lecture area at a corresponding time point in each video frame of the second video;

[0057] 103. Determine a movement path of the teacher's head in the first coordinate system based on the second position; determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0058] 104. Monitor the attention of each student based on the similarity between the teacher's moving path and the student's sight path.

[0059] In an embodiment of the present application, the above-mentioned classroom attention monitoring method can be implemented by a classroom attention monitoring device, which is a program module. The program module is located in the above-mentioned control device, and the control device can call the program module to implement the above-mentioned classroom attention monitoring method.

[0060] Among them, the classroom usually refers to the state of being in class, and the duration of a class is usually 45 minutes. In order to better monitor classroom attention, the use of classroom time can be divided into teaching mode and non-teaching mode. Among them, the teaching mode refers to the classroom learning mode in which the teacher teaches knowledge points. In the teaching mode, students are required to listen carefully to the teacher's lectures, and in this mode, the concentration of attention is manifested in being able to keep up with the teacher's rhythm. The student's eyes move with the movement of the teacher. The inventor found through creative research that when the listener pays attention to the lecturer's speech, the eyes are unconsciously fixed on the lecturer, which can help the listener better understand the lecturer's speech. Therefore, in the teaching mode, the student's eyes follow the movement of the teacher, and the higher the similarity between the student's eyesight movement path and the teacher's movement path, the more focused the student's attention is. This application is based on this principle to monitor students' attention in the classroom, and in the teaching mode, the teacher is required to be in the lecture area to better monitor students' attention. In addition, the non-teaching mode refers to a mode in which the teacher does not need to teach knowledge points. For example, the teacher arranges students to preview knowledge, do exercises, take exams, etc. In these situations, students are usually in a state of lowering their heads. Other methods can be used to monitor students' attention in the non-teaching mode, which will not be elaborated here.

[0061] It should be noted that, in actual applications, the mode can be manually switched by the teacher according to the class arrangement. For example, after the class, the teacher can start the teaching mode by operating the control device, such as through a physical button or a virtual button. After the control device detects the teacher's operation of starting the teaching mode, it responds to the operation and monitors the students' classroom attention according to the method in the embodiment of the present application. When it is detected that the teaching mode has reached the length of a class, the control device can automatically end the teaching mode, or when the teaching mode has not reached the length of a class, if a mode switching operation is detected, it switches from the teaching mode to the non-teaching mode in response to the mode switching operation. In actual applications, the start and end of the teaching mode can be set as needed, which is not limited here.

[0062] In an embodiment of the present application, if it is detected that the current classroom environment enters the teaching mode, the first video in the classroom shot by the first camera and the second video shot by the second camera can be obtained, and the starting time point of the first video is the same as the starting time point of the second video, and the ending time point of the first video is the same as the ending time point of the second video, so that the first video and the second video are videos of the same time period, and further, in order to ensure that the time period of the first video and the second video is the same, the time of the first camera and the second camera can be synchronized each time after entering the teaching mode.

[0063] After obtaining the first video and the second video, face recognition processing is performed on the first video to obtain the face orientation angle of each student's face area in each video frame of the first video, and head recognition processing is performed on the second video to determine the first position of the head of each student located in the student desk and chair area corresponding to the time point of each video frame of the second video, and the second position of the head of the teacher located in the lecture area corresponding to the time point of each video frame of the second video. Among them, the first video is mainly used to determine the face orientation angle of the students in class, and the second video is used to locate the position of the students and the teacher in the classroom. The position refers to the position in the first coordinate system, wherein both are positioned with the student's head and the teacher's head as calibration, and in teaching mode, the student's line of sight follows the teacher, usually looking at the teacher's head. Therefore, the movement path of the teacher and the student's line of sight can be effectively determined by head positioning.

[0064] Furthermore, the above-mentioned first position and second position both include x values ​​and y values ​​in a first coordinate system, and the first coordinate system is a three-dimensional coordinate system of the three-dimensional space formed by the classroom. In a feasible implementation method, the three-dimensional coordinate system of the three-dimensional space formed by the classroom can be set to have the lower left corner of the classroom as the origin, the direction toward the ceiling from the lower left corner as the Z axis, the direction toward the blackboard from the lower left corner as the X axis, and the direction perpendicular to the Z axis and the X axis and passing through the classroom toward the outside of the classroom as the Y axis. The first coordinate systems involved later are all based on this setting as an example and will not be repeated here.

[0065] It should be noted that each video frame has a corresponding time point. Since the first video and the second video are videos of the same time period, the parameters obtained from the video frames at the same time point in the first video and the second video can be comprehensively processed based on the time point to obtain the required parameters. And for students, since the student's facial orientation angle can be obtained from the first video and the student's first position can be obtained from the second video, when the facial orientation angle and the first position are used simultaneously in the subsequent description, they all refer to the facial orientation angle obtained from the video frame in the first video at the same time point, and the first position obtained from the video frame in the second video, so that the parameters of the same student at the same time point can be obtained.

[0066] See also Figure 2 , is a flow chart of a method for determining a face orientation angle in this application, the method comprising:

[0067] 201. Traverse the first video, perform face recognition on the traversed target video frame using a face deep learning model, and determine each face region image contained in the target video frame;

[0068] 202. Using the OpenCV cross-platform computer vision library to perform key point recognition on each of the facial region images, determine a first set of coordinate points of the key points of each of the facial region images in a camera coordinate system, wherein the key point set includes: a left eye center point, a right eye center point, a chin center point, and an eyebrow center point;

[0069] 203. Perform coordinate transformation on the first set of key points of each facial region image to obtain a second set of coordinates in a second coordinate system;

[0070] 204. Calculate the coordinates of the left eye center point, the right eye center point, the chin center point, and the eyebrow center of each facial region in the second set to obtain a facial orientation angle of each facial region in the second coordinate system, where each direction of the second coordinate system is parallel to each direction of the first coordinate system.

[0071] In an embodiment of the present application, it is necessary to determine the facial orientation angle of each student contained in each video frame of the first video. Specifically, each video frame in the first video is traversed in chronological order, and a face deep learning model is used to perform face recognition on the traversed target video frame to determine each face area image contained in the target video frame. For each determined face area image, OpenCV (cross-platform computer vision library) is used to perform key point recognition on each face area image to determine a first set of coordinate points of the key points of each face area image in the camera coordinate system, wherein the key point set includes: the center point of the left eye, the center point of the right eye, the center point of the chin and the center of the eyebrows.

[0072] Among them, OpenCV is a cross-platform computer vision library released under the BSD (Berkly Software Distribution, BSD license) license. It can run on many different types of operating systems and can effectively realize key point recognition.

[0073] It can be understood that when performing face recognition on the first video, the recognized facial features can be compared with the preset facial features of each student to determine the correspondence between each facial area image and the student, so that the facial orientation angle of each student in each video frame of the first video can be obtained.

[0074] In addition, considering that the orientation angle of the face changes little in a short period of time, in order to reduce the amount of data processed and reduce the occupancy of system resources, face recognition can be performed first through a face deep learning model to obtain a set of face area images of each student in the classroom in the first video. The set contains the correspondence between the time point of the video frame and the face area image. If there is a frame of video in which the face area image of student B is not detected, the set of face area images of student B does not contain the correspondence between the video frame and the face area image. After obtaining the set of face area images of each student, image sampling is performed according to a preset step size, for example, sampling is performed at a step size of every 3 seconds to obtain a set of sampled face area images, and the sampled face area image set is used to complete the method in this application to achieve the purpose of reducing data processing volume, reducing system resource occupancy, and improving monitoring efficiency.

[0075] In an embodiment of the present application, by traversing the first video, a set of facial area images of each student in the first video can be obtained, and further key point recognition can be performed on each facial area image based on OpenCV to effectively obtain a first set of key points of each student in each video frame. It should be noted that the coordinate points of the key points in the first set are determined based on the camera coordinate system, the origin of the camera coordinate system is the optical center of the first camera, the x-axis and y-axis are parallel to the x-axis and y-axis of the video frame, the z-axis is the optical axis of the first camera, and is perpendicular to the plane of the video frame. The intersection of the optical axis and the image plane is the above-mentioned origin, which can constitute a camera coordinate system of a rectangular coordinate system.

[0076] In order to better determine the face orientation angle of each student, the key points of the face area image of each student in each video frame will be transformed to obtain a second set of coordinates of each face area image in the second coordinate system, where the second coordinate system is a three-dimensional coordinate system with the human body as the coordinate origin.

[0077] Among them, the above-mentioned second coordinate system can be set to have the center point of the left eye in the face area image as the origin, the x-axis is parallel to the x-axis of the first coordinate system, the y-axis is parallel to the y-axis of the first coordinate system, and the z-axis is parallel to the z-axis of the first coordinate system, that is, the directions of the second coordinate system are parallel to the directions of the first coordinate system. It should be noted that through this setting method, the second coordinate system can be converted into the first coordinate system, so that the moving path of the teaching teacher and the line of sight path of each student can be determined in the first coordinate system, so as to effectively monitor the students' attention.

[0078] It should be noted that in the embodiment of the present application, the coordinate point of the key point in the camera coordinate system is converted to the coordinate point in the second coordinate system. The conversion method is the existing technology and will not be described here.

[0079] In an embodiment of the present invention, after converting the first set of key points of each facial area image to obtain a second set of coordinates in a second coordinate system, the coordinates of the left eye center point, right eye center point, chin center point and eyebrow center of each facial area image in the second set can be calculated to obtain the facial orientation angle of each facial area in the second coordinate system.

[0080] In an embodiment of the present application, the following method can be used to calculate the coordinates of the left eye center point, the right eye center point, the chin center point, and the eyebrow center of each face region in the second set to obtain the face orientation angle of each face region in the second coordinate system, specifically including:

[0081] Step A, determining a first line segment between the coordinates of the left eye center and the right eye center, and determining a second line segment between the coordinates of the chin center and the eyebrow center;

[0082] Step B: determining a first plane perpendicular to the first line segment and intersecting the midpoint of the first line segment, and determining a second plane perpendicular to the second line segment and including the midpoint of the first line segment;

[0083] Step C: taking the midpoint of the first line segment as the starting point and a three-dimensional vector formed by the line where the first plane intersects the second plane and points outward from the face as the face orientation angle.

[0084] In an embodiment of the present application, after obtaining the second set, the second set contains a subset of key points of each facial area in each video frame, wherein one subset contains the coordinates of the key points of the facial image area of ​​a student in a video frame, and the subset contains the coordinates of the student's left eye center point, right eye center point, chin center point and eyebrow center. In order to obtain the facial orientation angle of the student in the video frame, the coordinates of the left eye center point and the right eye center point in the subset can be first formed into a first line segment, and the coordinates of the chin center point and the eyebrow center in the subset can be formed into a second line segment. Two line segments, among which the left eye center point and the right eye center point used in the first line segment are for determining the student's head rotation angle, and for facilitating determining the starting point of the face orientation angle based on the midpoint of the left eye center point and the right eye center point (i.e., the midpoint of the first line segment), so as to obtain a more accurate face orientation angle. The chin center point and the eyebrow center used in the second line segment can be used to determine whether the student raises his chin, and can effectively determine the angle at which the student raises his head. Since the angle at which the student raises his head is consistent with the angle at which the student's eyes rise, the second line segment can be used to determine the angle at which the eyes rise. Specifically, a first plane that is perpendicular to the first line segment and intersects at the midpoint of the first line segment is determined. The angle of the first plane represents the student's head rotation angle, and a second plane that is perpendicular to the second line segment and includes the midpoint of the first line segment is determined. The angle formed by the second plane represents the angle at which the student raises his head. In this way, the line where the first plane intersects the second plane represents the direction of sight. Specifically, the midpoint of the first line segment can be used as the starting point of the facial orientation angle, and the starting point is on the line where the first plane and the second plane intersect. The three-dimensional vector formed by the line where the first plane and the second plane intersect and the direction toward the outside of the face is used as the facial orientation angle.

[0085] In an embodiment of the present invention, by identifying the left eye center point, right eye center point, chin center point and eyebrow center in the student's facial area image, the student's head rotation angle can be determined based on the left eye center point and the right eye center point, and the student's line of sight upward angle can be determined based on the chin center point and eyebrow center, and the student's facial orientation angle can be effectively determined based on the formed first plane and second plane, thereby improving the accuracy of the facial orientation angle determination.

[0086] In an embodiment of the present invention, after performing face recognition processing on the first video and obtaining the face orientation angle of each student in each video frame, the second video needs to be processed. Since the second video is shot by a second camera located on the ceiling of the classroom and capable of capturing the entire classroom from a bird's-eye view, the second video will be subjected to head recognition processing. The function of the second video is mainly to locate the positions of students and teachers in the first coordinate system. After head recognition processing, the heads located in the student desk and chair area are regarded as the students' heads, and the first positions of the heads of each student in each video frame of the second video are determined. The heads located in the lecture area are identified as the teachers' heads, and the second positions of the teachers' heads corresponding to the time points of each video frame in the second video are determined. The moving path of the teachers' heads in the first coordinate system is further determined based on the second position, and the line of sight path of the students in the first coordinate system is determined based on the face orientation angle, the first position, and the second position.

[0087] In the embodiment of the present application, the first position and the second position are both positions of the heads of the student and the teacher in the video frame converted to positions in the XY axis of the first coordinate system.

[0088] In order to better obtain the first position and the second position, the length and width information of the classroom preset in the system, as well as the length information of the lecture area and the length information of the student desk and chair area in the actual scene will be obtained, and the head recognition processing will be performed on each video frame of the second video through the head recognition model based on the deep learning algorithm to determine the head image area of ​​each head in each video frame of the second video, and the video frame will be divided into regions based on the ratio formed by the length information of the lecture area and the length information of the desk and chair area, the head located in the lecture area will be determined as the head of the teacher, the head located in the student desk and chair area will be determined as the head of the student, and the second position of the teacher's head and the first position of each student's head will be further determined.

[0089] The second position of the teacher's head may be determined as follows:

[0090] First, determine the head area image of the teaching teacher in each video frame of the second video, and determine the minimum pixel point of the head area image in the X-axis direction in the image coordinate system as the calibration point of the teaching teacher's head. It can be understood that since in the teaching mode, the teaching teacher usually lectures towards the students, and the students look at the teaching teacher's head. Therefore, the pixel point with the minimum x value is closest to the focus point when the students look at the teaching teacher, that is, using this pixel point can improve the accuracy of determining the moving path of the teaching teacher. Here, the image coordinate system refers to the coordinate system formed with the lower left corner of the classroom in the captured video frame as the origin, the length direction of the classroom as the X-axis, and the width direction of the classroom as the Y-axis.

[0091] Secondly, after obtaining the pixel point A of the lecturer in each video frame of the second video (i.e., the pixel point with the minimum x value mentioned above), the distance between two endpoints parallel to the X-axis and containing pixel point A will be determined in each video frame. The two endpoints are point B located on the wall in front of the classroom in the video frame, and point C located on the wall behind the classroom in the video frame. The first length of the line segment formed by BC and the second length of the line segment formed by pixel points A and C are determined. The ratio formed by the second length and the first length is determined, and the ratio is multiplied by the actual length of the classroom in the first coordinate system to obtain the x value of the lecturer in the first coordinate system. To achieve the purpose of converting the teaching teacher's position from the X-axis position in the video frame to the X-axis position in the classroom, similarly, the distance between the other two endpoints parallel to the Y-axis and containing pixel point A can be determined. Point E is located on the left wall of the classroom in the video frame, and point F is located on the right wall of the classroom in the video frame. The third length of the line segment formed by EF and the fourth length of the line segment formed by point E and pixel point A are determined. The ratio of the fourth length to the third length is multiplied by the actual width of the classroom in the first coordinate system to obtain the y value of the teaching teacher in the first coordinate system, and the obtained x value and y value constitute the second position of the teaching teacher's head in the first coordinate system.

[0092] Furthermore, the method for determining the first position of each student's head may include:

[0093] First, taking the determination of the first position of a single student's head as an example, the head area image of the student in each video frame of the second video can be determined, and the maximum pixel point of the head area image in the direction of the X-axis in the image coordinate system is determined as the calibration point of the student's head. The pixel point G corresponding to the calibration point is used to determine the first position of the student, that is, the distance between the two endpoints parallel to the X-axis and containing the pixel point G is determined, and the two endpoints, one point K, is located on the wall in front of the classroom in the video frame, and the other point L is located on the wall behind the classroom in the video frame. The fifth length of the line segment formed by KL and the sixth length of the line segment formed by the pixel points G and L are determined, the ratio formed by the sixth length and the fifth length is determined, and the ratio is multiplied by The actual length of the classroom in the first coordinate system is used to obtain the x value of the student in the first coordinate system, so as to achieve the purpose of converting the student's position from the X-axis position in the video frame to the X-axis position in the classroom. Similarly, the distance between the other two endpoints parallel to the Y-axis and containing the pixel point G can be determined. A point M is located on the left wall of the classroom in the video frame, and a point N is located on the right wall of the classroom in the video frame. The seventh length of the line segment formed by MN and the eighth length of the line segment formed by point M and pixel point G are determined. The ratio of the eighth length to the seventh length is multiplied by the actual width of the classroom in the first coordinate system to obtain the y value of the student in the first coordinate system, and the obtained x value and y value constitute the first position of the student's head in the first coordinate system.

[0094] In an embodiment of the present invention, after obtaining the facial orientation angle and first position of each student and the second position of the teacher through the method described above, the movement path of the teacher's head and the student's sight path will be further obtained.

[0095] See also Figure 3 , which is a flow chart of a method for determining a moving path of a teacher's head in a first coordinate system according to a second position in an embodiment of the present invention, comprising:

[0096] Step 301: Obtain the height information of the teacher and the height information of the lecture area in the classroom;

[0097] Step 302: Determine the head position information of the teacher at each time point in the first coordinate system based on the teacher's height information, the height information of the teaching area, and the second position;

[0098] Step 303: Use the head position information at each time point in the first coordinate system to form the movement path of the teacher.

[0099] In an embodiment of the present application, the height information of the teaching teacher, the height information of the classroom's teaching area, and the length and width information of the classroom are first obtained. The system pre-sets the height information of the teaching teachers of each subject, the height information of the classroom's teaching area, and the length and width information of the classroom. The name of the teaching teacher in the current class can be determined by face recognition, and the corresponding relationship between the preset name and height information can be searched by name to obtain the corresponding height information.

[0100] After obtaining the height information of the teaching teacher and the height information of the teaching area of ​​the classroom, the head position information of the teaching teacher in the first coordinate system at each time point can be determined based on the height information of the teaching teacher, the height information of the teaching area and the second position.

[0101] In the embodiment of the present application, the second position already includes the x value and y value of the teacher's head. Therefore, using the teacher's height information, the height information of the classroom's teaching area is required to determine the z value of the teacher's head in the first coordinate system, so as to determine the teacher's head position information. Specifically: the teacher's height information and the height information of the teaching area can be used as the z value of the teacher's head position information. For example, if the teacher's height is 1.6m and the height information of the teaching area is 0, then the z value of the teacher's head position information is determined to be 1.6m. Through the above method, the teacher's head position information at each time point can be effectively obtained.

[0102] In an embodiment of the present invention, after obtaining the head position information of the teaching teacher at each time point in the first coordinate system, the head position information will be used to form the movement path of the teaching teacher. The specific method can be: since each time point corresponds to a video frame, the head position information of multiple video frames of a single second can be deduplicated, and the remaining head position information after deduplication is calculated as the average value, and the head position information of the average value is used as the head position information of the single second. In this way, the head position information of the teaching teacher in seconds is obtained, and the head position information of the teaching teacher is connected in chronological order to obtain the above-mentioned movement path of the teaching teacher.

[0103] In the embodiment of the present application, the line of sight path of each student in the first coordinate system is determined based on the face orientation angle of each student, the first position and the second position of each student, specifically including the following steps:

[0104] Step D: Obtaining the height of each student and the height of the desk and chair, and determining the height of the eye area of ​​each student when the student is in a seated position based on the height of each student and the height of the desk and chair;

[0105] Step E: determining the coordinates of the eye parts of each student in the first coordinate system according to the height of the eye parts of each student and the first position;

[0106] Step F, determining the coordinates of the sight points of each student in the first coordinate system at each time point based on the coordinates of the eye parts of each student in the first coordinate system, the face orientation angle, and the second position;

[0107] Step G: Determine the sight path of each student using the coordinates of the sight point at each time point in the first coordinate system.

[0108] In an embodiment of the present application, the height of each student and the height of the desks and chairs used are pre-set in the system, so that the height of the students when sitting in front of the desks can be determined according to the height of the students and the height of the desks and chairs according to a preset algorithm. Since the height difference between the eyes of different students and the top of the head is relatively small, the statistical average value can be used to obtain the height of the eyes of each student when sitting. Specifically, the height of the students when sitting in front of the desks can be subtracted from the above average value, and the value obtained is the height of the eyes of the students when sitting.

[0109] Furthermore, according to the height of each student's eye part and the first position of each student, the coordinates of each student's eye part in the first coordinate system are determined, wherein the first position already includes the student's x value and y value in the first coordinate system, and the height of the student's eye part can be used as the student's z value in the first coordinate system to obtain the coordinates of the student's eye part. After obtaining the coordinates, the sight line vector of each student in the first coordinate system at each time point can be determined according to the coordinates of each student's eye part and the face orientation angle. Specifically: the coordinates are combined with the face orientation angle to obtain the student's sight line vector in the first coordinate system, wherein, since the coordinates used for the face orientation angle are in the second coordinate system, and the directions of the axes of the coordinate system are the same as and parallel to the directions of the axes of the first coordinate system, the components of the face orientation angle on the X-axis, the Y-axis, and the Z-axis can be added to the x-value, y-value, and z-value of the coordinates of the student's eye part respectively to obtain the above-mentioned sight line vector.

[0110] After obtaining the aforementioned gaze vector, the student's gaze vector and the second position of the instructor at the same time point corresponding to the same video frame are further used to determine the coordinates of the student's gaze point in the first coordinate system at the time point corresponding to the video frame. The coordinates of this gaze point are the student's gaze point. Specifically, a target plane parallel to the YZ axis and containing the second position in the first coordinate system can be determined. The intersection of the student's gaze vector and the target plane is used as the coordinates of the student's gaze point. The coordinates of each student's gaze point at each time point are then connected to obtain each student's gaze path.

[0111] Based on this, after obtaining the face orientation angle, the first position and the second position, the moving path of the teacher's head in the first coordinate system can be determined according to the second position, and the line of sight path of each student in the first coordinate system can be determined according to the face orientation angle, the first position and the second position, and the similarity between the moving path of the teacher and the line of sight path of each student can be calculated to obtain the similarity between the moving path of the teacher and the line of sight path of each student, so as to determine the attention of each student based on the similarity. Specifically, a preset threshold value of the similarity can be set, and the similarity between the moving path of the teacher and the line of sight path of each student can be calculated respectively to obtain the similarity of the line of sight tracking of each student. If the target similarity value is greater than or equal to the preset threshold, it is determined that the attention of the student corresponding to the target similarity is concentrated. The target similarity is the similarity of any one of the above-mentioned students. If the target similarity value is greater than the preset threshold, it is determined that the attention of the student corresponding to the target similarity is not concentrated. For example, if the similarity between the moving path of the teaching teacher and the sight path of student A is greater than or equal to the preset threshold, it is determined that student A is in a state of concentration; if the similarity between the moving path of the teaching teacher and the sight path of student A is less than the preset threshold, it is determined that student A is in a state of inattention, so that the student's attention can be effectively monitored.

[0112] In an embodiment of the present application, by using a first video captured by a first camera capable of capturing faces and a second video captured by a second camera capable of capturing the entire classroom in a teaching mode, the movement path of the teacher's head and the video path of each student's sight are obtained, and classroom attention is monitored by the similarity between the movement path of the teacher and the sight path of each student. Since in the teaching mode, students need to pay attention to the teacher's lecture content and follow the teacher's movement with their eyes in order to better listen to and understand the lecture content, it is possible to monitor the attention of each student by determining the similarity between the movement path of the teacher and the video path of each student, thereby effectively improving the accuracy of student attention monitoring.

[0113] See also Figure 4 , is a schematic diagram of the structure of a program module of a classroom attention monitoring device according to an embodiment of the present application, the device comprising:

[0114] Acquisition module 401 is configured to, upon detecting that the current classroom environment has entered a teaching mode, acquire a first video of the classroom captured by a first camera, and acquire a second video captured by a second camera during the same time period as the first video, wherein the first camera is located in front of a student seat in the classroom, and the second camera is located above the student seat. The teaching mode refers to a classroom learning mode in which a teacher lectures on knowledge points.

[0115] A first determination module 402 is configured to perform face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, and to perform head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system formed by the classroom.

[0116] A second determining module 403 is configured to determine a movement path of the teacher's head in the first coordinate system based on the second position; and determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0117] The monitoring module 404 is used to monitor the attention of each student based on the similarity between the movement path of the teacher and the sight path of each student.

[0118] It should be noted that the specific implementation methods of each module in the device of the present application can refer to the contents of each step in the aforementioned classroom attention monitoring method, which will not be repeated here.

[0119] In an embodiment of the present application, by using a first video captured by a first camera capable of capturing faces and a second video captured by a second camera capable of capturing the entire classroom in a teaching mode, the movement path of the teacher's head and the video path of each student's sight are obtained, and classroom attention is monitored by the similarity between the movement path of the teacher and the sight path of each student. Since in the teaching mode, students need to pay attention to the teacher's lecture content and follow the teacher's movement with their eyes in order to better listen to and understand the lecture content, it is possible to monitor the attention of each student by determining the similarity between the movement path of the teacher and the video path of each student, thereby effectively improving the accuracy of student attention monitoring.

[0120] Figure 5 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 5As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the various steps in the above method embodiment. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the various steps in the above method embodiment. It will be understood by those skilled in the art that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0121] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0122] If it is detected that the current classroom environment has entered a teaching mode, a first video of the classroom captured by a first camera is obtained, and a second video captured by a second camera during the same time period as the first video is obtained, wherein the first camera is set in front of the student seats in the classroom, and the second camera is located above the student seats. The teaching mode refers to a classroom learning mode in which the teacher teaches knowledge points;

[0123] Performing face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, performing head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system of the three-dimensional space formed by the classroom;

[0124] Determine a movement path of the teacher's head in the first coordinate system based on the second position; determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0125] The attention of each student is monitored according to the similarity between the moving path of the teacher and the sight path of each student.

[0126] It should be noted that the specific implementation of the above steps can refer to the description of each step in the aforementioned classroom attention monitoring method, which will not be repeated here.

[0127] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor performs the following steps:

[0128] If it is detected that the current classroom environment has entered a teaching mode, a first video of the classroom captured by a first camera is obtained, and a second video captured by a second camera during the same time period as the first video is obtained, wherein the first camera is set in front of the student seats in the classroom, and the second camera is located above the student seats. The teaching mode refers to a classroom learning mode in which the teacher teaches knowledge points;

[0129] Performing face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, performing head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system of the three-dimensional space formed by the classroom;

[0130] Determine a movement path of the teacher's head in the first coordinate system based on the second position; determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position;

[0131] The attention of each student is monitored according to the similarity between the moving path of the teacher and the sight path of each student.

[0132] It should be noted that the specific implementation of the above steps can refer to the description of each step in the aforementioned classroom attention monitoring method, which will not be repeated here.

[0133] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0134] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for monitoring classroom attention, characterized in that: The method comprises: If it is detected that the current classroom environment has entered a teaching mode, a first video of the classroom captured by a first camera is obtained, and a second video captured by a second camera during the same time period as the first video is obtained, wherein the first camera is set in front of the student seats in the classroom, and the second camera is located above the student seats. The teaching mode refers to a classroom learning mode in which the teacher teaches knowledge points; Performing face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, performing head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system of the three-dimensional space formed by the classroom; Determine a movement path of the teacher's head in the first coordinate system based on the second position; determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position; The attention of each student is monitored according to the similarity between the moving path of the teacher and the sight path of each student.

2. The method according to claim 1, characterized in that Determining the movement path of the teacher's head in the first coordinate system according to the second position includes: Obtaining the height information of the teacher and the height information of the teaching area of ​​the classroom; Determining the head position information of the teacher in the first coordinate system at each time point according to the height information of the teacher, the height information of the teaching area, and the second position; The moving path of the teacher is formed by using the head position information at each time point in the first coordinate system.

3. The method according to claim 1, characterized in that The determining, based on the face orientation angle, the first position, and the second position, of a sight line path of each student in the first coordinate system includes: Obtaining the height of each student and the height of the desk and chair, and determining the height of each student's eye area when they are in a seated position based on the height of each student and the height of the desk and chair; determining the coordinates of the eye parts of each student in the first coordinate system at each time point according to the height of the eye parts of each student and the first position; Determining the coordinates of the sight points of each student at each time point in the first coordinate system according to the coordinates of the eye parts of each student at each time point in the first coordinate system, the face orientation angle, and the second position; The sight path of each student is determined using the coordinates of the sight point at each time point in the first coordinate system.

4. The method according to claim 3, characterized in that The determining, based on the coordinates of the eye parts of each student in the first coordinate system, the face orientation angle, and the second position, the coordinates of the sight points of each student at each time point in the first coordinate system includes: Determine the sight line vector of each student in the first coordinate system according to the coordinates of the students' eyes at the same time point and the face orientation angle; determining a target plane in the first coordinate system that is parallel to a plane formed by YZ axes of the first coordinate system and includes the second position; The intersection point of the sight line vector and the target plane is used as the coordinate of the student's sight line point in the first coordinate system.

5. The method according to claim 1, wherein The performing face recognition processing on the first video to obtain the face orientation angle of each student's face region in each frame of the first video includes: Traversing the first video, performing face recognition on the traversed target video frame using a face deep learning model, and extracting images of each face region contained in the target video frame; Using the OpenCV cross-platform computer vision library to perform key point recognition on each facial region image, determining a first set of coordinate points of the key points of each facial region image in a camera coordinate system, wherein the key point set includes: a left eye center point, a right eye center point, a chin center point, and an eyebrow center point; performing coordinate transformation on the first set of key points of each facial region image to obtain a second set of coordinates in a second coordinate system, where the second coordinate system is a three-dimensional coordinate system with the human body as the coordinate origin; The coordinates of the left eye center point, right eye center point, chin center point and eyebrow center of each facial area in the second set are calculated to obtain the facial orientation angle of each facial area in the second coordinate system, and the directions of the second coordinate system are parallel to the directions of the first coordinate system.

6. The method according to claim 5, characterized in that The calculation of the coordinates of the left eye center point, the right eye center point, the chin center point, and the eyebrow center of each face region in the second set to obtain the face orientation angle of each face region in the second coordinate system includes: Determine a first line segment between the coordinates of the left eye center point and the right eye center point, and determine a second line segment between the coordinates of the chin center point and the coordinates of the eyebrow center point; Determine a first plane perpendicular to the first line segment and intersecting the midpoint of the first line segment, and determine a second plane perpendicular to the second line segment and including the midpoint of the first line segment; A three-dimensional vector formed by a line where the first plane intersects the second plane and the midpoint of the first line segment is taken as the starting point and is directed toward the outside of the face is used as the face orientation angle.

7. The classroom attention monitoring method according to claim 1, characterized in that: The monitoring of the attention of each student according to the similarity between the movement path of the teacher and the sight path of each student includes: Calculating the similarity between the moving path and each student's sight path respectively to obtain a similarity value of each student's sight tracking; If the target similarity value is greater than or equal to a preset threshold, determining that the student corresponding to the target similarity is focused; If the target similarity is less than a preset threshold, it is determined that the student corresponding to the target similarity value is not paying attention.

8. A classroom attention monitoring device, characterized in that: The device comprises: an acquisition module, configured to, upon detecting that the current classroom environment has entered a teaching mode, acquire a first video of the classroom captured by a first camera, and acquire a second video captured by a second camera during the same time period as the first video, wherein the first camera is located in front of a student seat in the classroom, and the second camera is located above the student seat. The teaching mode refers to a classroom learning mode in which a teacher lectures on knowledge points; a first determination module configured to perform face recognition processing on the first video to obtain a face orientation angle of each student's face region in each video frame of the first video, and to perform head recognition processing on the second video to determine a first position of each student's head located in the student desk and chair area at a time point in each video frame of the second video, and a second position of the teacher's head located in the lecture area at a time point in each video frame of the second video, wherein the first position and the second position both include x and y values ​​in a first coordinate system, where the first coordinate system is a three-dimensional coordinate system formed by the three-dimensional space of the classroom; a second determining module, configured to determine a movement path of the teacher's head in the first coordinate system based on the second position; and determine a sight line path of each student in the first coordinate system based on the face orientation angle, the first position, and the second position; The monitoring module is used to monitor the attention of each student based on the similarity between the movement path of the teacher and the sight path of each student.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images