Classroom attendance checking method based on face recognition, computer equipment and storage medium
The classroom attendance system, which combines panoramic cameras with close-up shots, solves the problem of identity recognition difficulties caused by students' varying postures, and achieves higher attendance accuracy.
Patent Information
- Application Number
- CN202510673648.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-23
- Publication Date
- 2025-10-31
AI Technical Summary
In existing classroom attendance systems based on facial recognition, students' varied postures make it difficult for cameras to identify their identities, affecting the accuracy of attendance tracking.
Classroom images are captured by combining panoramic cameras and close-up shots. The panoramic images are used to identify students in the front row, while the close-up images are used to identify students in the back row. Facial features are extracted and matched during class time, and the facial identity database is updated to improve recognition accuracy.
It improved the accuracy of student identification in the back row of the classroom, increased the probability of matching facial features with corresponding identity identifiers, and improved the accuracy of attendance tracking.
Smart Images
Figure CN120877397A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent attendance technology, and in particular to a classroom attendance method, computer equipment, and storage medium based on facial recognition. Background Technology
[0002] Contactless attendance in educational settings is a method that automatically records student attendance using non-contact technology. It aims to improve attendance efficiency, reduce manual operations, and ensure the accuracy and security of attendance data. Contactless attendance is typically implemented based on technologies such as radio frequency identification (RFID) and biometrics.
[0003] Facial recognition technology is commonly used in biometrics to achieve contactless attendance. This technology uses cameras installed in the teacher's area to capture images of students' faces. These images are then compared to a database of facial images to determine attendance. However, because students' postures vary, the images captured by the cameras may show students in profile or with their heads down, making it difficult to identify students and affecting the accuracy of attendance tracking. Summary of the Invention
[0004] This application provides a face recognition-based classroom attendance method, computer equipment, and storage medium, aiming to improve the accuracy of attendance tracking.
[0005] Firstly, a classroom attendance method based on facial recognition is provided, including:
[0006] Facial features are extracted from classroom images during a specific class period to obtain a facial feature set. The specific class period is the time between the start and end of classroom teaching, and the facial feature set includes multiple facial features.
[0007] The facial features in the facial feature set are matched with preset facial features in the facial identity database to obtain a first result set; the facial identity database includes multiple identity identifiers and preset facial features corresponding to the multiple identity identifiers, and the first result set includes facial matching results corresponding to the multiple facial features, the facial matching results being used to indicate whether an identity identifier corresponding to the facial feature is matched;
[0008] The preset facial features in the facial identity database are updated based on the first facial feature in the facial feature set; the first facial feature is the facial feature corresponding to the first facial matching result, and the first facial matching result is used to indicate the identity identifier corresponding to the matched facial feature;
[0009] The second facial feature in the facial feature set is matched with the preset facial features in the updated facial identity database to obtain a second result set; the second facial feature is the facial feature corresponding to the second facial matching result, and the second facial matching result is used to indicate the identity identifier corresponding to the unmatched facial feature; the second result set includes the facial matching result corresponding to the second facial feature;
[0010] By combining the first result set and the second result set, the attendance results corresponding to the class period are generated.
[0011] In this technical solution, after extracting facial features from classroom images during class time to obtain a facial feature set, the facial features in the facial feature set are matched with preset facial features in a facial identity database to obtain a first result set. The first result set includes the face matching results corresponding to the facial features in the facial feature set. Then, based on the facial features in the facial feature set that match the identity identifier, the preset facial features in the facial identity database are updated. Finally, the facial features in the facial feature set that do not match the identity identifier are matched with the updated preset facial features in the facial identity database to obtain a second result set. The second result set includes the faces corresponding to the facial features in the facial feature set that do not match the identity identifier. The matching results are then combined with the first and second result sets to generate attendance results for each class period, achieving seamless attendance tracking. By extracting and matching facial features from classroom images throughout the entire class period, after obtaining the first result set, the facial features of those matching the identity identifiers in the facial feature set are used to update the preset facial features in the facial identity database. This facilitates real-time updates to the facial identity database and improves its accuracy. A second matching is performed on facial features of those not matching the identity identifiers in the facial feature set, which in turn increases the probability of matching the identity identifiers corresponding to the facial features, thereby increasing the probability of successful attendance tracking and improving attendance accuracy.
[0012] In conjunction with the first aspect, in one possible implementation, updating the preset facial features in the facial identity database based on the first facial features in the facial feature set includes: determining a subset of facial features corresponding to a first identity identifier in the facial feature set, the subset of facial features including at least one first facial feature, each of the at least one first facial feature corresponding to the first identity identifier; calculating an average facial feature corresponding to the subset of facial features, the average facial feature being the average value of the at least one first facial feature; and updating the preset facial features corresponding to the first identity identifier in the facial identity database to the average facial features.
[0013] By calculating the average value of facial features corresponding to the same identity identifier and updating the preset facial features corresponding to the corresponding identity identifier in the facial identity database, the facial features in the facial identity database are made up to be the latest facial features.
[0014] In conjunction with the first aspect, in one possible implementation, generating the attendance result corresponding to the class period by combining the first result set and the second result set includes: determining an identity identifier set based on the first result set and the second result set, the identity identifier set including the identity identifier corresponding to the first face matching result in the first result set and the identity identifier corresponding to the first face matching result in the second result set; and generating the attendance result corresponding to the class period based on the attendance list corresponding to the class period and the identity identifier set.
[0015] Combining the identity identifiers obtained from two matches into an identity identifier set, and generating attendance results corresponding to class periods based on the attendance list and the identity identifier set, helps to increase the probability of successful attendance and thus improve the accuracy of attendance.
[0016] In conjunction with the first aspect, in one possible implementation, the step of matching the facial features in the facial feature set with preset facial features in the facial identity database to obtain a first result set includes: calculating the similarity between a target facial feature and each preset facial feature in the facial identity database to obtain multiple similarities corresponding to the target facial feature, wherein the target facial feature is any facial feature in the facial feature set; determining the maximum similarity corresponding to the target facial feature among the multiple similarities; and determining the facial matching result corresponding to the target facial feature based on the maximum similarity.
[0017] Determining the face matching result corresponding to the face feature based on the maximum similarity helps to match the identity identifier corresponding to the most similar face.
[0018] In conjunction with the first aspect, in one possible implementation, the step of extracting facial features from classroom images within a class period to obtain a facial feature set includes: performing head detection on a target classroom image to obtain a head image corresponding to the target classroom image, wherein the target classroom image is any classroom image within the class period; performing facial key point detection on the head image to obtain facial key points in the head image; correcting the facial pose in the head image to a frontal pose based on the facial key points; and extracting facial features from the corrected head image to obtain the facial features corresponding to the target classroom image.
[0019] After correcting the facial pose in the head image to a frontal pose based on the facial key points, facial features are extracted from the corrected head image to obtain facial features, which helps to extract facial features containing more feature information.
[0020] In conjunction with the first aspect, in one possible implementation, the step of extracting facial features from the corrected head image to obtain the facial features corresponding to the target classroom image includes: evaluating the facial quality of the corrected head image to obtain a facial quality score corresponding to the corrected head image; and, if the facial quality score is greater than a preset score threshold, extracting facial features from the corrected head image to obtain the facial features corresponding to the target classroom image.
[0021] If the face quality score corresponding to the head image is greater than the preset score threshold, facial features are extracted from the corrected head image to obtain facial features, which helps to improve the probability of successful facial feature matching in the future.
[0022] In conjunction with the first aspect, in one possible implementation, the step of evaluating the face quality of the corrected head image to obtain a face quality score corresponding to the corrected head image includes: calculating Euler angles based on the facial key points, the Euler angles being used to reflect the face pose; determining a first quality score based on the Euler angles; inputting the corrected head image into a preset face quality model to obtain a second quality score; and performing a weighted summation of the first quality score and the second quality score to obtain the face quality score corresponding to the corrected head image.
[0023] By weighting the first quality score determined based on Euler angles and the second quality score determined by the face quality model, the face quality score corresponding to the corrected head image is obtained. This evaluates the face quality in the head image from multiple dimensions, enabling a more accurate assessment of the face quality in the head image.
[0024] In conjunction with the first aspect, in one possible implementation, the classroom image includes a panoramic image and a partial image, wherein the panoramic image is a classroom image acquired by a panoramic image acquisition device, and the partial image is a classroom image acquired by a partial image acquisition device.
[0025] Classroom images include panoramic and partial images, which can capture facial images of all students, avoiding missed attendance due to missing student images.
[0026] Secondly, a classroom attendance device based on facial recognition is provided, comprising:
[0027] The facial feature extraction module is used to extract facial features from classroom images during a class period to obtain a facial feature set. The class period is the time between the start and end of classroom teaching, and the facial feature set includes multiple facial features.
[0028] The matching module is used to match the facial features in the facial feature set with the preset facial features in the facial identity database to obtain a first result set; the facial identity database includes multiple identity identifiers and preset facial features corresponding to the multiple identity identifiers, and the first result set includes facial matching results corresponding to the multiple facial features, and the facial matching results are used to indicate whether the identity identifier corresponding to the facial feature is matched;
[0029] The database update module is used to update the preset face features in the face identity database according to the first face feature in the face feature set; the first face feature is the face feature corresponding to the first face matching result, and the first face matching result is used to indicate the identity identifier corresponding to the matched face feature;
[0030] The matching module is further configured to match the second face feature in the face feature set with the preset face features in the updated face identity database to obtain a second result set; the second face feature is the face feature corresponding to the second face matching result, and the second face matching result is used to indicate the identity identifier corresponding to the unmatched face feature; the second result set includes the face matching result corresponding to the second face feature;
[0031] The attendance result generation module is used to combine the first result set and the second result set to generate the attendance results corresponding to the class period.
[0032] Thirdly, a computer device is provided, including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, wherein when the processor executes the one or more computer programs, the computer device performs the face recognition-based classroom attendance method described in the first aspect.
[0033] Fourthly, a computer-readable storage medium is provided, which stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the face recognition-based classroom attendance method of the first aspect.
[0034] This application can achieve the following technical effects: enabling seamless classroom attendance; by extracting and matching facial features from classroom images throughout the entire class period, after obtaining the first result set, updating the preset facial features in the facial identity database using facial features that match the identity identifiers in the facial feature set, which helps to update the facial identity database in real time and improve its accuracy; and by performing a secondary matching between facial features that do not match the identity identifiers in the facial feature set and the updated facial identity database, it helps to increase the probability of matching the identity identifiers corresponding to the facial features, thereby increasing the probability of successful attendance and improving the accuracy of attendance. Attached Figure Description
[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 A system block diagram of a classroom attendance system provided in an embodiment of this application;
[0037] Figure 2 A flowchart illustrating a classroom attendance method based on face recognition, provided as an embodiment of this application;
[0038] Figure 3 A schematic diagram of classroom images and head images provided in embodiments of this application;
[0039] Figure 4 This is a schematic diagram of the structure of a classroom attendance device based on face recognition provided in an embodiment of this application;
[0040] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0042] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.
[0043] The technical solution of this application is applicable to classroom attendance scenarios.
[0044] In classroom attendance scenarios, the attendance period is typically defined as a timeframe at the beginning, middle, or end of class. During this period, a camera installed near the podium captures images of the area where students are located. Facial recognition is then performed on the students in these images to determine attendance. However, these cameras are usually monocular cameras. In images captured by monocular cameras, the faces of students in the back rows occupy a smaller proportion of the classroom, resulting in lower recognition accuracy and impacting attendance accuracy. Furthermore, during the attendance period, students may turn their heads to the side or lower their heads, causing their faces to not appear fully visible in the image, making it difficult to identify them and leading to attendance failures, further affecting attendance accuracy.
[0045] To improve attendance accuracy, this application proposes a classroom attendance scheme based on a binocular camera. It acquires classroom images using a panoramic camera and a close-up lens, obtaining panoramic and partial images. The panoramic image is primarily used to identify the identity information of students in the front row, while the partial image is mainly used to identify the identity information of students in the back row. This solves the problem of low recognition accuracy caused by the small proportion of the faces of students in the back row in the panoramic image, thus improving the accuracy of identifying the identities of students in the back row. Furthermore, the entire class period is used as the attendance period. Classroom images are acquired throughout the entire class period using the panoramic camera and the close-up lens. Student facial features in the classroom images throughout the entire class period are matched for identity verification. The matching facial features are then used to update the facial identity database, ensuring that the facial features in the database are up-to-date. The updated facial identity database is then used to perform a secondary matching on the facial features of students who did not match their identities, increasing the probability of matching the identity corresponding to the facial features, thereby improving the probability of successful attendance and thus improving attendance accuracy.
[0046] The technical solution of this application is applied to a classroom attendance system. For ease of understanding, the classroom attendance system of this application will be introduced first.
[0047] See Figure 1 , Figure 1 A schematic diagram of the system composition of a classroom attendance system provided in this application embodiment is shown below. Figure 1 As shown, the classroom attendance system 10 includes a classroom attendance device 101, a panoramic image acquisition device 102, and a partial image acquisition device 103. The classroom attendance device 101 is used to implement classroom attendance. The classroom attendance device 101 is connected to the panoramic image acquisition device 102 and the partial image acquisition device 103 to acquire classroom images captured by the panoramic image acquisition device 102 and the partial image acquisition device 103. The connection between the classroom attendance device 101 and the panoramic image acquisition device 102 and the partial image acquisition device 103 can be wired or wireless.
[0048] The panoramic image acquisition device 102 is used to acquire panoramic images. A panoramic image is an image that reflects the entire classroom space. A panoramic image can comprehensively display the content in the classroom space and typically has a large field of view. In this application, the panoramic image acquisition device 102 can be a wide-angle lens. The horizontal field of view of the wide-angle lens exceeds a preset field of view, such as 120°. Thus, the panoramic image acquired by the panoramic image acquisition device 102 can capture the content in the first horizontal direction of the classroom space. The first horizontal direction reflects the direction of the field of view on the horizontal plane, which is a plane perpendicular to the direction of gravity.
[0049] The local image acquisition device 103 is used to acquire local images of the classroom space. A local image refers to an image reflecting a specific area within the classroom space; it can also be called a close-up image, displaying a portion of the content within the classroom space. The local image acquisition device 103 can be a telephoto lens, capable of capturing and magnifying distant objects, allowing them to occupy more pixels in the image. The focal length of the telephoto lens is greater than a preset focal length, such as 85 mm, with a maximum focal length reaching several hundred millimeters. Thus, the local image acquisition device 103 can acquire close-up content from a distance, ensuring that the distant content meets detection requirements in the local image. In this application, the orientation of the local image acquisition device 103 is adjustable. By adjusting its orientation, the local image acquisition device 103 can acquire images of any area within the classroom space. For example, the local image acquisition device 103 can be mounted on a gimbal, a support device used to stabilize the image acquisition device, allowing the local image acquisition device 103 to rotate freely in any direction, thereby enabling multi-directional adjustments.
[0050] In some specific implementations, the panoramic image acquisition device 102 and the local image acquisition device 103 are positioned at the same location in the classroom space. "Same location" means that the distance between the panoramic image acquisition device 102 and the local image acquisition device 103 is less than a preset distance; for example, the distance between the panoramic image acquisition device 102 and the local image acquisition device 103 is less than 0.05 meters. Positioning the panoramic image acquisition device 102 and the local image acquisition device 103 at the same location in the classroom space ensures that the local image acquired by the local image acquisition device 103 is a part of the panoramic image acquired by the panoramic image acquisition device 102.
[0051] The classroom attendance device 101 can be an independent device from the panoramic image acquisition device 102 and the local image acquisition device 103. For example, the panoramic image acquisition device 102 and the local image acquisition device 103 can be integrated into one image acquisition device, and the classroom attendance device 101 can be connected to this integrated image acquisition device via wireless or wired connection. The classroom attendance device 101 can be, for example, a teaching terminal host or a personal computer. Alternatively, the classroom attendance device 101 can be integrated into one device, for example, on a computer device capable of both acquisition and recognition, such as an interactive flat panel device. This application does not limit the connection and integration methods between the classroom attendance device 101 and the panoramic image acquisition device 102 and the local image acquisition device 103.
[0052] based on Figure 1 The classroom attendance system 10 shown can implement the technical solution of this application, and the technical solution of this application is specifically applied to the classroom attendance device 101 in the classroom attendance system 10. The technical solution of this application is described in detail below.
[0053] See Figure 2 , Figure 2 This application provides a flowchart illustrating a face recognition-based classroom attendance method. The method is applied to a classroom attendance device, such as... Figure 1 Classroom attendance equipment 101; such as Figure 2 As shown, the method includes the following steps:
[0054] S201, extract facial features from classroom images during class time to obtain a facial feature set.
[0055] Here, a class period refers to the time between the start and end of a class. In one feasible implementation, an electronic timetable is obtained, which records the course names, start times, and end times. The time period between the start and end times of a course is defined as the class period. An image acquisition device is controlled to acquire images during the class period, thereby obtaining classroom images within that period. There are multiple classroom images within the class period.
[0056] In some cases, classroom images during class time may include panoramic images and partial images. Panoramic images are those captured by a panoramic image acquisition device, while partial images are those captured by a partial image acquisition device. For definitions of panoramic and partial image acquisition devices, please refer to the preceding text. Figure 1 The corresponding description.
[0057] Classroom images include panoramic and partial images, which can capture facial images of all students, avoiding missed attendance due to missing student images.
[0058] A facial feature set includes multiple facial features. These multiple facial features include the facial features corresponding to faces in each classroom image within a specific class period. Each classroom image within a specific class period corresponds to one or more facial features. A single facial feature represents the feature attributes of a face in a classroom image. Facial features are represented as n-dimensional vectors, where n represents the feature dimension of the facial feature vector. The larger n is, the richer the feature information contained in the facial feature.
[0059] Each facial feature in the facial feature set is extracted using a facial feature extraction algorithm, which includes, but is not limited to, deep learning-based algorithms such as ArcFace and FaceNet.
[0060] In some embodiments, facial features are extracted from classroom images during class time using the following steps A1-A4 to obtain a facial feature set:
[0061] A1. Perform head detection on the target classroom image to obtain the head image corresponding to the target classroom image.
[0062] Here, the target classroom image is any classroom image within the classroom period; the target classroom image can be either the aforementioned panoramic image or the aforementioned partial image.
[0063] The head image corresponding to the target classroom image refers to the image used to represent a person's head in the target classroom image. For example, see... Figure 3 Assuming the target classroom image is as follows: Figure 3 As shown in P1, the human head image can be as follows: Figure 3 As shown in the rectangular regions, the image within each rectangular region is cropped to obtain the head image.
[0064] The process involves using an object detection algorithm to detect heads in the target classroom image, obtaining the location information of the head bounding boxes. This location information includes the coordinates of the head bounding box within the target classroom image, as well as its width and height. Based on this location information, the image within the head bounding box is cropped to obtain the corresponding head image from the target classroom image. Object detection algorithms include, but are not limited to, region-based convolutional neural networks (RCNN), YOLO, and SSD algorithms.
[0065] A2. Perform facial landmark detection on the head image to obtain the facial landmarks in the head image.
[0066] Here, facial key points in a head image refer to points in a head image that have specific semantics and salient features. Facial key points in a head image are used to describe the structure and shape of the face in the head image.
[0067] Facial key points in a head image include facial contour points and facial feature points. Facial contour points are distributed on the outer contour of the face and are used to describe the overall shape and outline of the face. Facial feature points include points representing the positions of the eyes, nose, mouth, and ears.
[0068] This involves using facial landmark detection algorithms to detect facial landmarks in a head image, thereby obtaining the facial landmarks within the image. Facial landmark detection algorithms include, but are not limited to, the Active Shape Model (ASM) algorithm, the Active Appearance Model (AAM) algorithm, the VGGNet algorithm, and the ResNet algorithm.
[0069] A3. Based on the key facial features in the head image, correct the facial pose in the head image to a frontal pose.
[0070] Here, frontal pose refers to a face facing directly forward, with the facial plane parallel to the imaging plane of the image acquisition device; in frontal pose, facial features are in a relatively symmetrical and horizontal state.
[0071] Among them, the affine transformation relationship between the facial key points in the head image and the standard template key points (referring to the facial key points in the frontal face pose) can be determined, and the facial pose in the head image can be corrected to the frontal face pose based on the affine transformation relationship.
[0072] A4. Extract facial features from the corrected head image to obtain the facial features corresponding to the target classroom image.
[0073] In some possible cases, facial features can be extracted directly from the corrected head image using a facial feature extraction algorithm to obtain the facial features corresponding to the target classroom image.
[0074] In some possible cases, facial features can be extracted from the corrected head image through the following steps a1-a2 to obtain the facial features corresponding to the target classroom image:
[0075] a1. Evaluate the face quality of the corrected head image to obtain the face quality score corresponding to the corrected head image.
[0076] The face quality score corresponding to the corrected head image is used to evaluate the quality of the face in the head image. The higher the face quality score corresponding to the corrected head image, the better the face quality of the corrected head image, and the easier it is to identify the identity of the face. The lower the face quality score corresponding to the corrected head image, the worse the face quality of the corrected head image, and the more difficult it is to identify the identity of the face.
[0077] In one feasible implementation, the face quality of the corrected head image is evaluated through the following steps a11-a14 to obtain the face quality score corresponding to the corrected head image:
[0078] a11. Calculate Euler angles based on the facial landmarks in the head image.
[0079] Here, Euler angles are used to reflect the facial pose in a head image, which includes rotational and / or pitch poses.
[0080] In one specific implementation, Euler angles are calculated based on key eye points in a head image. These Euler angles, determined by the key eye points, reflect the rotational posture of the face in the head image, i.e., the head tilt. For example, the formula for calculating the Euler angles is: θ represents Euler angles, (x1, y1) represents the coordinates of the left eye key point in the head image, and (x2, y2) represents the coordinates of the right eye key point in the head image.
[0081] Optionally, Euler angles can be determined based on the position of the nose tip keypoint in the head image relative to the line connecting the two eye keypoints (left and right eye keypoints). These Euler angles, determined by the position of the nose tip keypoint in the head image relative to the line connecting the two eye keypoints (left and right eye keypoints), are used to reflect the tilt posture of the face in the head image, i.e., the head-down posture. This application does not impose any limitations on this.
[0082] a12. Determine the first quality score based on Euler angles.
[0083] Specifically, the angle difference between the Euler angles and the preset face angle can be calculated. The preset face angle is the face angle in a frontal pose, and for example, the preset face angle is 0°. A first quality score is determined based on the angle difference between the Euler angles and the preset face angle. The smaller the angle difference between the Euler angles and the preset face angle, the higher the first quality score; the larger the angle difference, the lower the first quality score.
[0084] For example, the first quality score can be determined according to the following formula: s1=(180° / |θ-α|)*K, where s1 represents the first quality score, α represents the preset angle, and K represents the constant score value.
[0085] a13. Input the corrected head image into the preset face quality model to obtain the second quality score.
[0086] Here, the face quality model is a pre-trained model used to evaluate image quality. For example, a face quality model can be a binary classification model that assesses whether an image is good or bad. Face quality models include, but are not limited to, face sharpness evaluation models, face illumination evaluation models, and face occlusion evaluation models. Face sharpness evaluation models are used to evaluate image sharpness to assess image quality, face illumination evaluation models are used to evaluate lighting conditions to assess image quality, and face occlusion evaluation models are used to evaluate the degree of face occlusion to assess image quality.
[0087] a14. The first quality score and the second quality score are weighted and summed to obtain the face quality score corresponding to the corrected head image.
[0088] The face quality score corresponding to the corrected head image is represented as S = w1*s1 + w2*s2, where S represents the face quality score corresponding to the corrected head image, w1 and w2 represent the weights corresponding to the first quality score and the second quality score, respectively, and the sum of the weights corresponding to the first quality score and the second quality score is 1. s2 represents the second quality score.
[0089] In steps a11-a14 above, the first quality score determined based on Euler angles and the second quality score determined by the face quality model are weighted to obtain the face quality score corresponding to the corrected head image. This evaluates the face quality in the head image from multiple dimensions, enabling a more accurate assessment of the face quality in the head image.
[0090] Alternatively, other implementation methods can be used to evaluate the face quality of the corrected head image and obtain a face quality score corresponding to the corrected head image. For example, the resolution of the corrected head image can be determined, and the face quality score corresponding to the corrected head image can be determined based on the resolution of the corrected head image. This application does not limit this approach.
[0091] a2. When the face quality score is greater than the preset score threshold, extract the face features from the corrected head image to obtain the face features corresponding to the target classroom image.
[0092] In this process, facial features are extracted from the corrected head image using a facial feature extraction algorithm to obtain the facial features corresponding to the target classroom image.
[0093] In steps a1-a2 above, if the face quality score corresponding to the head image is greater than a preset score threshold, facial features are extracted from the corrected head image to obtain facial features, which helps to improve the probability of successful facial feature matching in the future.
[0094] Understandably, since there are one or more students in the classroom image, head detection is performed on the target classroom image, resulting in one or more head images corresponding to the target classroom image. In the case of multiple head images corresponding to the target classroom image, for each head image corresponding to the target classroom image, the facial features corresponding to each head image can be obtained by extracting facial features through steps A2-A4 above, thus obtaining multiple facial features corresponding to the target classroom image.
[0095] In steps A1-A4 above, after correcting the facial pose in the head image to a frontal pose based on the facial key points in the head image, facial features are extracted from the corrected head image to obtain facial features, which helps to extract facial features containing more feature information.
[0096] Understandably, since there are multiple classroom images during a class period, facial features are extracted for each classroom image during the class period through the above steps A1-A4. This yields the facial features corresponding to each classroom image during the class period, and the facial features corresponding to each classroom image during the class period constitute a facial feature set.
[0097] Alternatively, facial features can be extracted from classroom images during class time using other implementation methods to obtain a facial feature set. For example, after performing head detection on the classroom images to obtain the corresponding head images, the facial pose in the head images can be evaluated. If the facial pose is close to a frontal pose, facial features can be directly extracted from the head images to obtain the facial features corresponding to the classroom images. This application does not impose any limitations on this.
[0098] S202, Match the facial features in the facial feature set with the preset facial features in the facial identity database to obtain the first result set.
[0099] Here, the facial identity database is a pre-built database used to store identity information, containing the facial features of different students. The facial identity database includes multiple identity identifiers and corresponding preset facial features. The preset facial features are the student's facial features pre-extracted using a facial feature extraction algorithm. One preset facial feature in the facial identity database corresponds to one identity identifier, and one identity identifier represents one student's identity.
[0100] The first result set includes face matching results corresponding to multiple face features in the face feature set. Each face matching result is the result corresponding to a single face feature, and it indicates whether a match was found for the identity identifier corresponding to the face feature. There are two types of face matching results: a first face matching result, which indicates a match for the identity identifier corresponding to the face feature, and a second face matching result, which indicates a match for the identity identifier corresponding to the face feature.
[0101] In some embodiments, the facial features in the facial feature set are matched with preset facial features in the facial identity database through the following steps B1-B3 to obtain a first result set:
[0102] B1. Calculate the similarity between the target face feature and each preset face feature in the face identity database to obtain multiple similarity values corresponding to the target face feature.
[0103] Here, the target facial feature is any facial feature in the set of facial features. The multiple similarities corresponding to the target facial feature are the similarities corresponding to multiple identity identifiers in the facial identity database.
[0104] Among them, the Euclidean distance between the target face feature and each preset face feature in the face identity database can be calculated as multiple similarities corresponding to the target face feature; the similarity is negatively correlated with the Euclidean distance, that is, the larger the Euclidean distance, the smaller the similarity, and the smaller the Euclidean distance, the larger the similarity.
[0105] Alternatively, the cosine similarity between the target facial feature and each preset facial feature in the facial identity database can be calculated as multiple similarities corresponding to the target facial feature.
[0106] Alternatively, the Manhattan distance between the target facial feature and each preset facial feature in the facial identity database can be calculated as multiple similarities corresponding to the target facial feature; similarity is negatively correlated with Manhattan distance, that is, the larger the Manhattan distance, the smaller the similarity, and the smaller the Manhattan distance, the larger the similarity.
[0107] This application does not limit the specific calculation method for calculating the similarity between the target facial features and the preset facial features.
[0108] B2. Determine the maximum similarity among the multiple similarities corresponding to the target facial features.
[0109] B3. Determine the face matching result corresponding to the target face feature based on the maximum similarity.
[0110] Specifically, if the maximum similarity corresponding to the target facial feature is greater than the preset similarity threshold, the face matching result corresponding to the target facial feature is determined as the first face matching result, and the identity identifier corresponding to the preset facial feature with the maximum similarity is determined as the identity identifier corresponding to the target facial feature; if the maximum similarity corresponding to the target facial feature is less than or equal to the preset similarity threshold, the face matching result corresponding to the target facial feature is determined as the second face matching result.
[0111] In steps B1-B3 above, determining the face matching result corresponding to the face feature based on the maximum similarity helps to match the identity identifier corresponding to the most similar face.
[0112] Understandably, for each facial feature in the facial feature set, the facial matching result is determined through the above steps B1-B3, thus obtaining the facial matching result corresponding to each facial feature in the facial feature set. The facial matching results corresponding to each facial feature in the facial feature set constitute the first result set.
[0113] S203, Update the preset face features in the face identity database based on the first face feature in the face feature set.
[0114] Here, the first face feature is the face feature corresponding to the first face matching result. For the definition of the first face matching result, please refer to the aforementioned step S202.
[0115] In some embodiments, the preset facial features in the facial identity database are updated through the following steps C1-C3:
[0116] C1. In the set of facial features, determine the subset of facial features corresponding to the first identity identifier.
[0117] Here, the first identity identifier is the identity identifier corresponding to any first face feature in the face feature set. The face feature subset corresponding to the first identity identifier includes at least one first face feature. The face features in the face feature subset correspond to the same identity identifier, and each of the at least one first face features corresponds to the first identity identifier.
[0118] C2. Calculate the average facial features corresponding to the subset of facial features.
[0119] Here, the average facial feature corresponding to the subset of facial features is the average value of at least one first facial feature in the subset of facial features.
[0120] The average facial features are represented as (X1, X2, ..., Xn). 1≤i≤n, where n represents the dimension of the facial features, x ji Let represent the feature value of the j-th first face feature in the i-th feature dimension of the face feature subset, and m represent the number of first face features in the face feature subset.
[0121] C3. Update the preset facial features corresponding to the first identity identifier in the facial identity database to the average facial features corresponding to the subset of facial features.
[0122] In steps C1-C3 above, the average value of facial features corresponding to the same identity is calculated, and the preset facial features corresponding to the corresponding identity in the facial identity database are updated, so that the facial features in the facial identity database are the latest facial features.
[0123] Understandably, since the identity identifiers corresponding to different first facial features in the facial feature set may be different, there may be multiple identity identifiers corresponding to the facial feature set. For multiple identity identifiers corresponding to the facial feature set, the preset facial features corresponding to the identity identifiers in the facial identity database are updated through the above steps C1-C3 respectively.
[0124] Optionally, after determining the subset of facial features corresponding to the first identity in the facial feature set, the preset facial features corresponding to the first identity in the facial identity database can be updated to one of the first facial features in the subset of facial features corresponding to the first identity. This one first facial feature can be any first facial feature in the subset of facial features corresponding to the first identity, or the first facial feature with the highest similarity in the subset of facial features corresponding to the first identity, or the last extracted first facial feature in the subset of facial features corresponding to the first identity, etc. This application does not impose any limitations on this.
[0125] S204, match the second facial features in the facial feature set with the preset facial features in the updated facial identity database to obtain the second result set.
[0126] Here, the second face feature is the face feature corresponding to the second face matching result. For the definition of the second face matching result, please refer to the aforementioned step S202.
[0127] The second result set includes the face matching results corresponding to the second face feature.
[0128] In some embodiments, the second facial features in the facial feature set are matched with the preset facial features in the updated facial identity database through the following steps D1-D3 to obtain a second result set:
[0129] D1. Calculate the similarity between the second facial feature and each preset facial feature in the updated facial identity database to obtain multiple similarities corresponding to the second facial feature.
[0130] The calculation of the similarity between the second facial feature and each preset facial feature in the updated facial identity database is similar to the calculation of the similarity between the target facial feature and each preset facial feature in the facial identity database in step B1 above. Please refer to the relevant description in step B1 above, which will not be repeated here.
[0131] D2. Determine the maximum similarity among the multiple similarities corresponding to the second facial feature.
[0132] D3. Determine the face matching result corresponding to the second face feature based on the maximum similarity.
[0133] The process of determining the face matching result corresponding to the second face feature is similar to the process of determining the face matching result corresponding to the target face feature in step B3 above. Please refer to the relevant description in step B3 above, which will not be repeated here.
[0134] Understandably, when there are multiple second face features in the face feature set, for each second face feature in the face feature set, the face matching result is determined through the above steps D1-D3, and the face matching result corresponding to each second face feature is obtained. The face matching result corresponding to each second face feature forms the second result set.
[0135] S205, combine the first result set and the second result set to generate the attendance results corresponding to the class period.
[0136] In some embodiments, attendance results for class periods are generated through the following steps E1-E2:
[0137] E1. Determine the identity set based on the first result set and the second result set.
[0138] Here, the identity identifier set includes the identity identifier corresponding to the first face matching result in the first result set and the identity identifier corresponding to the first face matching result in the second result set.
[0139] E2. Generate attendance results for each class period based on the attendance list and identity set corresponding to that class period.
[0140] The attendance list includes information such as each student's name or student ID. It can identify the name or student ID corresponding to each identity in the identity set, thus obtaining the attendance name set or attendance student ID set. The names or student IDs in the attendance list are compared with the attendance name set or attendance student ID set. If a student's name or student ID in the attendance list exists in the attendance name set or attendance student ID set, then the student's attendance result is determined to be present. If a student's name or student ID in the attendance list does not exist in the attendance name set or attendance student ID set, then the student's attendance result is determined to be absent.
[0141] In steps E1-E2 above, the identity identifiers obtained from the two matches are combined into an identity identifier set, and the attendance results corresponding to the class period are generated based on the attendance list and the identity identifier set. This helps to increase the probability of successful attendance and thus improve the accuracy of attendance.
[0142] In the above Figure 2In the corresponding technical solution, after extracting facial features from classroom images during class time to obtain a facial feature set, the facial features in the facial feature set are matched with preset facial features in a facial identity database to obtain a first result set. The first result set includes the face matching results corresponding to the facial features in the facial feature set. Then, based on the facial features in the facial feature set that match the identity identifier, the preset facial features in the facial identity database are updated. Finally, the facial features in the facial feature set that do not match the identity identifier are matched with the updated preset facial features in the facial identity database to obtain a second result set. The second result set includes the faces in the facial feature set that do not match the identity identifier. The face matching results are combined with the first and second result sets to generate attendance results for the corresponding class period, achieving seamless attendance tracking. By extracting and matching facial features from classroom images throughout the entire class period, after obtaining the first result set, the facial features of those matching the identity identifier in the facial feature set are used to update the preset facial features in the facial identity database. This helps to update the facial identity database in real time and improve its accuracy. The facial features of those not matching the identity identifier in the facial feature set are then matched a second time with the updated facial identity database, which helps to increase the probability of matching the identity identifier corresponding to the facial features, thereby increasing the probability of successful attendance tracking and improving attendance accuracy.
[0143] The method of this application has been described above; the apparatus of this application will be described below.
[0144] See Figure 4 , Figure 4 This is a schematic diagram of a classroom attendance device based on face recognition provided in an embodiment of this application, as shown below. Figure 4 As shown, the classroom attendance device 30 based on facial recognition includes:
[0145] The face feature extraction module 301 is used to extract face features from classroom images during a class period to obtain a face feature set. The class period is the time between the start and end of classroom teaching. The face feature set includes multiple face features.
[0146] The matching module 302 is used to match the facial features in the facial feature set with the preset facial features in the facial identity database to obtain a first result set; the facial identity database includes multiple identity identifiers and preset facial features corresponding to the multiple identity identifiers, and the first result set includes facial matching results corresponding to the multiple facial features, and the facial matching results are used to indicate whether the identity identifier corresponding to the facial feature is matched;
[0147] The database update module 303 is used to update the preset face features in the face identity database according to the first face feature in the face feature set; the first face feature is the face feature corresponding to the first face matching result, and the first face matching result is used to indicate the identity identifier corresponding to the matched face feature;
[0148] The matching module 302 is further configured to match the second face feature in the face feature set with the preset face features in the updated face identity database to obtain a second result set; the second face feature is the face feature corresponding to the second face matching result, and the second face matching result is used to indicate the identity identifier corresponding to the unmatched face feature; the second result set includes the face matching result corresponding to the second face feature;
[0149] The attendance result generation module 304 is used to combine the first result set and the second result set to generate the attendance result corresponding to the class period.
[0150] It should be noted that the aforementioned face recognition-based classroom attendance device 30 can execute the face recognition-based classroom attendance method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments can be found in the face recognition-based classroom attendance method provided in the embodiments of this application.
[0151] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device 40 provided in an embodiment of this application. The computer device 40 includes a processor 401 and a memory 402. The memory 402 is connected to the processor 401, for example, via a bus.
[0152] Processor 401 is configured to support the computer device 40 in performing the corresponding functions in the methods described in the above method embodiments. Processor 401 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0153] Memory 402 is used to store program code, etc. Memory 402 may include volatile memory (VM), such as random access memory (RAM); memory 402 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 402 may also include combinations of the above types of memory.
[0154] The memory 402 is used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the face recognition-based classroom attendance method in the embodiments of this application. The processor executes various functional applications and data processing of the face recognition-based classroom attendance method by running the non-volatile software programs, instructions, and modules stored in the memory, thereby realizing the functions of the face recognition-based classroom attendance method provided in the above method embodiments.
[0155] Memory 402 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the annotation display device, etc. In some embodiments, the memory may include memory remotely located relative to the processor, which can be connected to a face recognition-based classroom attendance device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0156] The one or more modules are stored in the memory. When executed by the one or more processors, they perform the classroom attendance method based on face recognition in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.
[0157] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.
[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing associated hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0159] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A classroom attendance method based on facial recognition, characterized in that, include: Facial features are extracted from classroom images during a specific class period to obtain a facial feature set. The specific class period is the time between the start and end of classroom teaching, and the facial feature set includes multiple facial features. The facial features in the facial feature set are matched with preset facial features in the facial identity database to obtain a first result set; the facial identity database includes multiple identity identifiers and preset facial features corresponding to the multiple identity identifiers, and the first result set includes facial matching results corresponding to the multiple facial features, the facial matching results being used to indicate whether an identity identifier corresponding to the facial feature is matched; The preset facial features in the facial identity database are updated based on the first facial feature in the facial feature set; the first facial feature is the facial feature corresponding to the first facial matching result, and the first facial matching result is used to indicate the identity identifier corresponding to the matched facial feature; The second facial feature in the facial feature set is matched with the preset facial features in the updated facial identity database to obtain a second result set; the second facial feature is the facial feature corresponding to the second facial matching result, and the second facial matching result is used to indicate the identity identifier corresponding to the unmatched facial feature; the second result set includes the facial matching result corresponding to the second facial feature; By combining the first result set and the second result set, the attendance results corresponding to the class period are generated.
2. The method according to claim 1, characterized in that, The step of updating the preset facial features in the facial identity database based on the first facial feature in the facial feature set includes: In the set of facial features, a subset of facial features corresponding to the first identity identifier is determined. The subset of facial features includes at least one first facial feature, and each of the at least one first facial features corresponds to the first identity identifier. Calculate the average face feature corresponding to the subset of face features, wherein the average face feature is the average value of the at least one first face feature; The preset facial features corresponding to the first identity identifier in the facial identity database are updated to the average facial features.
3. The method according to claim 1, characterized in that, The step of combining the first result set and the second result set to generate the attendance results corresponding to the class period includes: Based on the first result set and the second result set, an identity identifier set is determined, wherein the identity identifier set includes the identity identifier corresponding to the first face matching result in the first result set and the identity identifier corresponding to the first face matching result in the second result set; Based on the attendance list corresponding to the class period and the identity set, the attendance result corresponding to the class period is generated.
4. The method according to claim 1, characterized in that, The step of matching the facial features in the facial feature set with preset facial features in the facial identity database to obtain a first result set includes: Calculate the similarity between the target facial feature and each preset facial feature in the facial identity database to obtain multiple similarities corresponding to the target facial feature, wherein the target facial feature is any facial feature in the set of facial features; Determine the maximum similarity among the multiple similarities corresponding to the target facial features; Based on the maximum similarity, the face matching result corresponding to the target face feature is determined.
5. The method according to any one of claims 1-4, characterized in that, The process of extracting facial features from classroom images during class time to obtain a facial feature set includes: Perform head detection on the target classroom image to obtain the head image corresponding to the target classroom image, wherein the target classroom image is any classroom image within the classroom time period; Facial landmark detection is performed on the head image to obtain the facial landmarks in the head image; Based on the facial key points, the facial pose in the head image is corrected to a frontal pose; Facial features are extracted from the corrected head image to obtain the facial features corresponding to the target classroom image.
6. The method according to claim 5, characterized in that, The step of extracting facial features from the corrected head image to obtain the facial features corresponding to the target classroom image includes: The corrected head image is subjected to a face quality evaluation to obtain a face quality score corresponding to the corrected head image; If the face quality score is greater than a preset score threshold, facial features are extracted from the corrected head image to obtain the facial features corresponding to the target classroom image.
7. The method according to claim 6, characterized in that, The step of evaluating the face quality of the corrected head image to obtain a face quality score corresponding to the corrected head image includes: Euler angles are calculated based on the facial key points, and the Euler angles are used to reflect the facial pose. Determine the first quality score based on the Euler angles; The corrected head image is input into a preset face quality model to obtain a second quality score; The first quality score and the second quality score are weighted and summed to obtain the face quality score corresponding to the corrected head image.
8. The method according to any one of claims 1-4, characterized in that, The classroom images include panoramic images and partial images. The panoramic images are classroom images captured by a panoramic image acquisition device, and the partial images are classroom images captured by a partial image acquisition device.
9. A computer device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, and the processor, when executing the one or more computer programs, causing the computer device to perform the method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-8.