Classroom attendance checking method based on face recognition, computer equipment and storage medium

Abstract

The invention provides a classroom attendance checking method based on face recognition, computer equipment and a storage medium, and the method comprises the following steps: matching a current classroom face feature with a preset face feature in a first face identity database to obtain a first face matching result; under the condition that the first face matching result is used for indicating that the identity identification corresponding to the face feature of the current classroom is not obtained through matching, matching the face feature of the current classroom with a real-time face feature in a second face identity database to obtain a second face matching result; the second face identity database is used for storing identities and real-time face features of a plurality of students; and under the condition that the second face matching result is used for indicating that the identity identifier corresponding to the face feature of the current classroom is obtained through matching, generating an attendance checking result corresponding to the current classroom period according to the second face matching result. According to the technical scheme, the identity label can be matched under the non-front-face postures of the student such as head measurement or head lowering so as to determine the identity of the student.

Description

Technical Field

[0001] This application relates to the field of intelligent attendance technology, and in particular to a classroom attendance method, computer equipment, and storage medium based on facial recognition. Background Technology

[0002] Contactless attendance in educational settings is a method that automatically records student attendance using non-contact technology. It aims to improve attendance efficiency, reduce manual operations, and ensure the accuracy and security of attendance data. Contactless attendance is typically implemented based on technologies such as radio frequency identification (RFID) and biometrics.

[0003] Facial recognition technology is commonly used in biometrics to achieve contactless attendance. This technology uses cameras installed in the teacher's area to capture images of students' faces. These images are then compared to a database of facial images to determine attendance. However, because students' postures vary, the images captured by the cameras may show students in profile or with their heads down, making it difficult to identify students and affecting the accuracy of attendance tracking. Summary of the Invention

[0004] This application provides a face recognition-based classroom attendance method, computer equipment, and storage medium, aiming to improve the accuracy of attendance tracking.

[0005] Firstly, a classroom attendance method based on facial recognition is provided, including: The facial features of the current classroom are matched with the preset facial features in the first facial identity database to obtain the first facial matching result; the facial features of the current classroom are obtained by extracting facial features from classroom images during the current classroom period; the first facial identity database is used to store the identity identifiers and preset facial features of multiple pre-registered students. If the first face matching result indicates that no identity identifier corresponding to the current classroom face feature is matched, the current classroom face feature is matched with the real-time face feature in the second face identity database to obtain a second face matching result. The second face identity database is used to store the identity identifiers and real-time face features of the multiple students. Each identity identifier in the second face identity database corresponds to multiple sets of real-time face features. The multiple sets of real-time face features belong to the same student. The real-time face features of different groups in the multiple sets of real-time face features belong to different face feature clusters. The face feature clusters are obtained by clustering the face features of historical classrooms. The face features of historical classrooms are obtained by extracting face features from classroom images in the previous classroom period of the current classroom period. When the second face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, the attendance result corresponding to the current classroom time period is generated based on the second face matching result.

[0006] In this technical solution, after matching the current classroom facial features with preset facial features in a first facial identity database to obtain a first facial matching result, and if the first facial matching result indicates that no identity identifier corresponding to the current classroom facial features was matched, the current classroom facial features are matched with real-time facial features in a second facial identity database to obtain a second facial matching result. If the second facial matching result indicates that an identity identifier corresponding to the current classroom facial features was matched, the attendance result corresponding to the current class period is generated based on the second facial matching result, achieving seamless classroom attendance. Since the second facial identity database stores multiple student identity identifiers and real-time facial features, with each identity identifier corresponding to multiple sets of real-time facial features, multiple sets of real-time facial features... The facial features belong to the same student, and different groups of real-time facial features belong to different facial feature clusters. The facial feature clusters are obtained by clustering the facial features of the history classroom. The facial features of the history classroom are obtained by extracting facial features from the classroom images of the previous class period of the current class period. The multiple groups of real-time facial features belonging to the same student encompass the student's facial features in different facial poses. Matching the current classroom facial features with the real-time facial features in the second facial identity database is equivalent to matching the current classroom facial features with the student's facial features in different facial poses. This increases the probability of matching the identity identifier when the student is in a non-frontal facial pose such as turning their head to the side or looking down, thereby increasing the probability of successful attendance and improving the accuracy of attendance.

[0007] In conjunction with the first aspect, in one possible implementation, the method further includes: when the first face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, generating the attendance result corresponding to the current classroom time period based on the first face matching result.

[0008] Once the identity identifier corresponding to the facial features of the person in the current classroom is obtained through matching, attendance results are generated based on the matching results, enabling seamless attendance tracking.

[0009] In conjunction with the first aspect, in one possible implementation, the method further includes: when the first face matching result or the second face matching result indicates that the identity identifier corresponding to the face feature of the current classroom is obtained, acquiring all face features corresponding to the current identity identifier within the current classroom time period, as multiple real-time face features corresponding to the current identity identifier; the current identity identifier is the identity identifier corresponding to the face feature of the current classroom; clustering the multiple real-time face features to obtain multiple face feature clusters corresponding to the current identity identifier; updating multiple sets of real-time face features corresponding to the current identity identifier in the second face identity database according to the multiple face feature clusters.

[0010] The system acquires all facial features of individuals with the same identity identifier during class time and clusters them into multiple facial feature clusters. Then, it updates multiple sets of real-time facial features corresponding to the identity identifiers in the second facial identity database based on these multiple facial feature clusters. This enables real-time updates to the second facial identity database, ensuring that the facial features in the second facial identity database can cover the facial features of students in the latest and different facial states.

[0011] In conjunction with the first aspect, in one possible implementation, clustering the plurality of real-time facial features to obtain a plurality of facial feature clusters corresponding to the current identity includes: calculating the similarity between each of the plurality of real-time facial features and a current preset facial feature; the current preset facial feature is a preset facial feature corresponding to the current identity in the first facial identity database; and grouping real-time facial features with similarity belonging to the same similarity range into the same facial feature cluster to obtain a plurality of facial feature clusters corresponding to the current identity.

[0012] Using the similarity between real-time facial features and facial features in the first facial identity database as the clustering index, real-time facial features with similarity within the same range are grouped into the same facial feature cluster. This results in multiple facial feature clusters corresponding to the same identity identifier, enabling multiple sets of real-time facial features corresponding to the same identity identifier in the second facial database to cover different facial qualities and facial poses, thereby improving the probability of successful face matching.

[0013] In conjunction with the first aspect, in one possible implementation, the step of clustering the plurality of real-time facial features to obtain a plurality of facial feature clusters corresponding to the current identity includes: determining the facial quality score corresponding to each of the plurality of real-time facial features; and classifying real-time facial features whose facial quality scores belong to the same quality score range into the same facial feature cluster to obtain a plurality of facial feature clusters corresponding to the current identity.

[0014] Using the face quality score corresponding to real-time face features as the clustering index, real-time face features with face quality scores belonging to the same quality score range are divided into the same face feature cluster. This results in multiple face feature clusters corresponding to the same identity, which enables multiple sets of real-time face features corresponding to the same identity in the second face database to cover different face poses, thereby improving the probability of successful face matching.

[0015] In conjunction with the first aspect, in one possible implementation, clustering the multiple real-time facial features to obtain multiple facial feature clusters corresponding to the current identity includes: traversing the multiple real-time facial features; calculating the distance between the traversed real-time facial features and the facial feature cluster centers corresponding to the multiple feature clusters, assigning the traversed real-time facial features to target feature clusters among the multiple feature clusters, until all the multiple real-time facial features have been traversed, and the target feature cluster is the feature cluster corresponding to the facial feature cluster center with the smallest distance; updating the facial feature cluster center corresponding to each feature cluster based on the real-time facial features in each of the multiple feature clusters; and determining the multiple feature clusters as the multiple facial feature clusters corresponding to the current identity when the facial feature cluster centers corresponding to the multiple feature clusters no longer change.

[0016] By traversing real-time facial features and calculating the feature cluster corresponding to the cluster center of the facial feature with the smallest distance from the real-time facial feature, the facial feature cluster center is updated until the facial feature cluster center no longer changes. In this way, multiple facial feature clusters corresponding to the same identity are obtained, which can achieve adaptive division of facial features.

[0017] In conjunction with the first aspect, in one possible implementation, updating the multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database based on the multiple facial feature clusters includes: extracting N real-time facial features from each facial feature cluster in the multiple facial feature clusters as multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database, where N is the number of a set of real-time facial features in the second facial identity database, and N > 1.

[0018] Multiple real-time facial features are extracted from each facial feature cluster to form multiple sets of real-time facial features, which can cover facial features under more facial poses as much as possible, thereby improving the probability of successful face matching.

[0019] In conjunction with the first aspect, in one possible implementation, the step of extracting N real-time face features from each of the plurality of face feature clusters as multiple sets of real-time face features corresponding to the current identity in the second face identity database includes: dividing the real-time face features in the target face feature cluster into N face feature groups according to the feature scores corresponding to the real-time face features in the target face feature cluster; the target face feature cluster is any face feature cluster among the plurality of face feature clusters; different face feature groups correspond to different feature score ranges; and extracting one real-time face feature from each of the N face feature groups to obtain a set of real-time face features corresponding to the current identity in the second face identity database.

[0020] The face features are grouped according to the feature scores corresponding to the real-time face features in the face feature clusters, and the real-time face features are extracted from the face feature groups as multiple sets of real-time face features corresponding to the same identity. This enables multiple sets of real-time face features for the same identity to cover different face poses.

[0021] Secondly, a classroom attendance device based on facial recognition is provided, comprising: The facial feature matching module is used to match the facial features of the current classroom with the preset facial features in the first facial identity database to obtain the first facial matching result; the facial features of the current classroom are obtained by extracting facial features from classroom images during the current classroom period; the first facial identity database is used to store the identity identifiers and preset facial features of multiple pre-registered students. The face feature matching module is further configured to, when the first face matching result indicates that no identity identifier corresponding to the face feature of the current classroom is matched, match the face feature of the current classroom with the real-time face features in the second face identity database to obtain a second face matching result; the second face identity database is used to store the identity identifiers and real-time face features of the multiple students, each identity identifier in the second face identity database corresponds to multiple sets of real-time face features, the multiple sets of real-time face features belong to the same student, and the real-time face features of different groups in the multiple sets of real-time face features belong to different face feature clusters, the face feature clusters are obtained by clustering the face features of historical classrooms, and the face features of historical classrooms are obtained by extracting face features from classroom images in the previous classroom period of the current classroom period; The attendance module is used to generate the attendance result corresponding to the current class period based on the second face matching result, when the second face matching result indicates that the identity identifier corresponding to the face feature of the current class is obtained.

[0022] Thirdly, a computer device is provided, including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, wherein when the processor executes the one or more computer programs, the computer device performs the face recognition-based classroom attendance method described in the first aspect.

[0023] Fourthly, a computer-readable storage medium is provided, which stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the face recognition-based classroom attendance method of the first aspect.

[0024] This application can achieve the following technical effects: It enables seamless classroom attendance tracking. Because the second facial identity database stores multiple student identifiers and real-time facial features, each identifier corresponds to multiple sets of real-time facial features. These multiple sets of real-time facial features belong to the same student, and different groups within these sets belong to different facial feature clusters. These clusters are obtained by clustering historical classroom facial features, which are extracted from classroom images from the previous classroom period. The multiple sets of real-time facial features belonging to the same student encompass the student's facial features in different facial poses. Matching the current classroom facial features with the real-time facial features in the second facial identity database is equivalent to matching the current classroom facial features with the student's facial features in different facial poses. This increases the probability of matching the identifier when the student is tilting their head or looking down (non-frontal poses), thereby increasing the probability of successful attendance tracking and improving attendance accuracy. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A system block diagram of a classroom attendance system provided in an embodiment of this application; Figure 2 A flowchart illustrating a classroom attendance method based on face recognition, provided as an embodiment of this application; Figure 3 A schematic diagram illustrating the process of updating the second face identity database provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a classroom attendance device based on face recognition provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0029] The technical solution of this application is applicable to classroom attendance scenarios.

[0030] In classroom attendance scenarios, cameras typically installed near the podium capture images of the area where students are located. Facial recognition is then performed on the students' faces in these images to determine attendance. However, during the recording process, students may turn their heads to the side or look down, resulting in their faces not being fully visible in the image. This makes it difficult to identify the students, leading to attendance failures and affecting the accuracy of attendance tracking.

[0031] To improve attendance accuracy, this application proposes a classroom attendance scheme. This scheme involves clustering facial features of students who successfully matched their identities in the previous class period, forming multiple facial feature clusters. A subset of facial features from each of these clusters is then saved to a second set of real-time facial identity data, updated in real-time with each class period, serving as multiple sets of real-time facial features corresponding to the identity identifier. In the next class period, classroom facial features that failed to match with the fixed first facial identity database are then matched a second time with facial features from the second set of facial identity data. Since these multiple sets of real-time facial features are extracted from multiple facial feature clusters, they can cover students' facial features in different facial poses. This second matching of classroom facial features with those from the second set of facial identity data is equivalent to matching classroom facial features with students' facial features in various poses, increasing the probability of matching the identity identifier when students are tilting their heads or looking down (non-frontal poses), thereby increasing the probability of successful attendance and improving attendance accuracy. By increasing the probability of matching identification tags when students are not facing forward, such as when their heads are turned to the side or down, students can be identified with fewer scans, thus reducing the number of scans and extending the lifespan of the camera-mounted gimbal.

[0032] The technical solution of this application is applied to a classroom attendance system. For ease of understanding, the classroom attendance system of this application will be introduced first.

[0033] See Figure 1 , Figure 1A schematic diagram of the system composition of a classroom attendance system provided in this application embodiment is shown below. Figure 1 As shown, the classroom attendance system 10 includes a classroom attendance device 101 and an image acquisition device 102. The classroom attendance device 101 is used to implement classroom attendance. The classroom attendance device 101 is connected to the image acquisition device 102 to acquire classroom images captured by the image acquisition device 102. The connection between the classroom attendance device 101 and the image acquisition device 102 can be wired or wireless.

[0034] The image acquisition device 102 is used to acquire classroom images. The posture and position of the image acquisition device 102 are adjustable. By adjusting its posture and position, the image acquisition device 102 can acquire images of any area in the classroom space. For example, the image acquisition device 102 is mounted on a pan-tilt head, which is a support device used to stabilize the image acquisition device and supports the image acquisition device 102 to rotate freely in any direction, thereby making multi-directional adjustments.

[0035] The classroom attendance device 101 and the image acquisition device 102 can be independent devices; for example, the image acquisition device 102 can be an independent camera, and the classroom attendance device 101 can be connected to the independent camera wirelessly or via wired connection. The classroom attendance device 101 can be, for example, a teaching terminal host or a personal computer. Alternatively, the classroom attendance device 101 and the image acquisition device 102 can be integrated into a single device. For example, they can be integrated into a computer device capable of both acquisition and recognition, such as an interactive flat panel device. This application does not limit the connection method or integration method between the classroom attendance device 101 and the image acquisition device 102.

[0036] based on Figure 1 The classroom attendance system 10 shown can implement the technical solution of this application, and the technical solution of this application is specifically applied to the classroom attendance device 101 in the classroom attendance system 10. The technical solution of this application is described in detail below.

[0037] See Figure 2 , Figure 2 This application provides a flowchart illustrating a face recognition-based classroom attendance method. The method is applied to a classroom attendance device, such as... Figure 1 Classroom attendance equipment 101; such as Figure 2 As shown, the method includes the following steps: S201, Match the current classroom facial features with the preset facial features in the first facial identity database to obtain the first facial matching result.

[0038] Here, the facial features of the current classroom are obtained by extracting facial features from classroom images within the current classroom time period. A classroom time period refers to the time between the start and end of class, and the current classroom time period refers to the current classroom time period for which attendance needs to be taken. The current classroom time period can be any classroom time period for which attendance needs to be taken. Specifically, in a real-time classroom observation scenario, the current classroom time period can be the currently ongoing classroom time period.

[0039] In one feasible implementation, an electronic timetable is obtained, which records the name of the course, as well as the start time and end time of the course. The time period between the start time and the end time of the course is taken as the class period. Any one of the class periods is taken as the current class period, or the class period that includes the current time is taken as the current class period.

[0040] The classroom images during the current class period are obtained by controlling the image acquisition device to capture images within the current class period. There is more than one image captured during the current class period. For example, if the class period is 45 minutes long, and the image acquisition device captures images every 30 seconds (i.e., twice per minute), scanning and photographing each student in the classroom during each capture, and there are 50 students in the classroom (i.e., the image acquisition device takes 50 photos in one capture), then with continuous scanning and acquisition throughout the current class period, there will be 2 * 45 * 50 = 4500 classroom images.

[0041] For each classroom image within the current class period, after head detection is performed on each image to obtain head images, facial features are extracted from each head image using a facial feature extraction algorithm to obtain the facial features corresponding to each classroom image. Facial feature extraction algorithms include, but are not limited to, deep learning-based algorithms such as ArcFace and FaceNet.

[0042] The facial features in the current classroom are any facial features corresponding to any classroom image within the current classroom time period.

[0043] The first facial identity database is used to store the identity identifiers and preset facial features of multiple pre-registered students. Pre-registration means that the identity identifiers and preset facial features in the first facial identity database are manually entered and saved. For example, a user (teacher or student) inputs a student's facial image / head image, along with the student's name or student ID, into the classroom attendance device. The attendance device then generates the student's identity identifier and preset facial features based on the information entered by the user and saves them to the first facial identity database.

[0044] The default facial features in the first facial identity database are frontal facial features, which are obtained by extracting facial features from head images in a frontal pose. A frontal pose refers to a face facing directly forward, with the facial plane parallel or nearly parallel to the imaging plane. Each default facial feature in the first facial identity database corresponds to one identity identifier, and each identity identifier is used to identify a student.

[0045] In some embodiments, the facial features of the current classroom are matched with preset facial features in the first facial identity database through the following steps A1-A3 to obtain the first facial matching result: A1. Calculate the similarity between the current classroom facial features and each preset facial feature in the first facial identity database to obtain multiple similarity values ​​corresponding to the current classroom facial features.

[0046] Here, the multiple similarities corresponding to the facial features in the current classroom are the similarities corresponding to multiple identity identifiers in the first facial identity database, with one similarity corresponding to one identity identifier in the first facial identity database.

[0047] Specifically, the Euclidean distance between the current classroom facial features and each preset facial feature in the first facial identity database can be calculated as multiple similarities corresponding to the current classroom facial features. The similarity is negatively correlated with the Euclidean distance, that is, the larger the Euclidean distance, the smaller the similarity, and the smaller the Euclidean distance, the larger the similarity.

[0048] Alternatively, the cosine similarity between the current classroom facial features and each preset facial feature in the first facial identity database can be calculated as multiple similarities corresponding to the current classroom facial features.

[0049] Alternatively, the Manhattan distance between the current classroom facial features and each preset facial feature in the first facial identity database can be calculated as multiple similarities corresponding to the current classroom facial features; similarity is negatively correlated with Manhattan distance, that is, the larger the Manhattan distance, the smaller the similarity, and the smaller the Manhattan distance, the larger the similarity.

[0050] This application does not limit the specific calculation method for calculating the similarity between the facial features of the current classroom and the preset facial features in the first facial identity database.

[0051] A2. Among the multiple similarities corresponding to the facial features in the current classroom, determine the maximum similarity corresponding to the facial features in the current classroom.

[0052] A3. Based on the maximum similarity of the facial features in the current classroom, determine the first face matching result corresponding to the facial features in the current classroom.

[0053] Specifically, if the maximum similarity of the facial features in the current classroom is greater than a preset similarity threshold, the first face matching result is used to indicate the identity identifier corresponding to the facial features in the current classroom that has been matched, and the identity identifier corresponding to the preset facial features with the maximum similarity is determined as the identity identifier corresponding to the facial features in the current classroom; if the maximum similarity of the facial features in the current classroom is less than or equal to the preset similarity threshold, the first face matching result is used to indicate the identity identifier corresponding to the facial features in the current classroom that has not been matched.

[0054] In steps A1-A3 above, determining the face matching result corresponding to the face feature based on the maximum similarity helps to match the identity identifier corresponding to the most similar face.

[0055] S202, if the first face matching result indicates that no identity identifier corresponding to the current classroom face features has been matched, the current classroom face features are matched with the real-time face features in the second face identity database to obtain the second face matching result.

[0056] Here, the second face identity database is used to store the identity identifiers and real-time facial features of multiple students. The identity identifiers of multiple students in the second face identity database are derived from the identity identifiers of multiple students in the first face identity database.

[0057] The real-time facial features in the second facial identity database are understood as facial features that change and update over time. Real-time facial features can include facial features in various poses such as looking down or turning the head. Each identity in the second facial identity database corresponds to multiple sets of real-time facial features. Multiple sets of real-time facial features corresponding to the same identity belong to the same student. Different groups of real-time facial features within the multiple sets of real-time facial features corresponding to the same identity belong to different facial feature clusters. These facial feature clusters are obtained by clustering facial features from history classrooms. The facial features from history classrooms are obtained by extracting facial features from classroom images from the previous classroom period.

[0058] The identity identifiers and real-time facial features in the second facial identity database are obtained through facial feature filtering during classroom teaching. For details on how to filter facial features during classroom teaching to obtain real-time facial features and identity identifiers, please refer to [link to relevant documentation]. Figure 3 The corresponding description.

[0059] The current classroom facial features are matched with the real-time facial features in the second facial identity database to obtain the second facial matching result. This is similar to the previous method of matching the current classroom facial features with the preset facial features in the first facial identity database to obtain the first facial matching result. The implementation methods described in steps A1-A3 above can be referred to, and will not be repeated here.

[0060] S203, if the second face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, the attendance result corresponding to the current classroom time period is generated based on the second face matching result.

[0061] S204, if the first face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, the attendance result corresponding to the current classroom time period is generated based on the first face matching result.

[0062] Specifically, the attendance list corresponding to the current class period is obtained. The attendance list includes information such as the name or student ID of each student. When the first face matching result or the second face matching result is used to indicate the identity identifier corresponding to the face feature of the current class, the identity identifier corresponding to the face feature of the current class is determined according to the first face matching result or the second face matching result. The name or student ID corresponding to the identity identifier corresponding to the face feature of the current class is determined as the current student's name or current student ID. The attendance result of the student whose name or student ID in the attendance list is the current student's name or current student ID is determined as attendance.

[0063] In the above Figure 2 In the corresponding technical solution, after matching the current classroom facial features with preset facial features in the first facial identity database to obtain the first facial matching result, if the first facial matching result indicates that no identity identifier corresponding to the current classroom facial features has been matched, the current classroom facial features are matched with real-time facial features in the second facial identity database to obtain the second facial matching result. If the second facial matching result indicates that an identity identifier corresponding to the current classroom facial features has been matched, the attendance result corresponding to the current class period is generated based on the second facial matching result, realizing seamless attendance tracking in the classroom. Since the second facial identity database stores the identity identifiers and real-time facial features of multiple students, each identity identifier corresponds to multiple sets of real-time facial features, and multiple sets of real-time facial features... Real-time facial features belonging to the same student, and different groups of real-time facial features belonging to different facial feature clusters, are obtained by clustering facial features from historical classrooms. Historical classroom facial features are obtained by extracting facial features from classroom images from the previous classroom period. Multiple groups of real-time facial features belonging to the same student encompass the student's facial features in different facial poses. Matching the current classroom facial features with real-time facial features in the second facial identity database is equivalent to matching the current classroom facial features with the student's facial features in different facial poses. This increases the probability of matching identity identifiers when the student is in a non-frontal pose such as tilting their head or looking down, thereby increasing the probability of successful attendance and improving attendance accuracy.

[0064] In some embodiments, when the first face matching result or the second face matching result is used to indicate the identity identifier corresponding to the facial features of the current classroom, the above... Figure 2 The corresponding technical solutions also include Figure 3 The process steps shown include the following steps S301-S303: S301, Obtain all facial features corresponding to the current identity during the current class period, and use them as multiple real-time facial features corresponding to the current identity.

[0065] Here, the current identity identifier is the identity identifier corresponding to the facial features of the current classroom, that is, the identity identifier corresponding to the first face matching result or the second face matching result.

[0066] For each classroom image within the current class period, facial feature matching is performed in accordance with the methods described in steps S201-S202 above. This yields the facial matching results for all facial features within the current class period. Based on these results, facial features that match the identity identifier are selected from all facial features within the current class period to form a facial feature set. From this set, all facial features whose identity identifier is the current identity identifier are selected as multiple real-time facial features corresponding to the current identity identifier.

[0067] S302, cluster the multiple real-time facial features corresponding to the current identity identifier to obtain multiple facial feature clusters corresponding to the current identity identifier.

[0068] Here, clustering multiple real-time facial features corresponding to the current identity identifier to obtain multiple facial feature clusters corresponding to the current identity identifier means: dividing multiple real-time facial features corresponding to the current identity identifier into different facial feature clusters according to a certain characteristic, so that real-time facial features in the same facial feature cluster have the same characteristics, and real-time facial features in different facial feature clusters have different characteristics.

[0069] In one feasible implementation, multiple real-time facial features corresponding to the current identity can be divided according to the similarity between real-time facial features and preset facial features in the first facial identity database; the multiple real-time facial features corresponding to the current identity are then clustered through the following steps B1-B2 to obtain multiple facial feature clusters corresponding to the current identity: B1. Calculate the similarity between each real-time facial feature and the current preset facial feature among multiple real-time facial features corresponding to the current identity identifier.

[0070] Here, the current preset facial feature is the preset facial feature corresponding to the current identity identifier in the first facial identity database. After matching the current identity identifier from the first facial identity database, the preset facial feature corresponding to the current identity identifier in the first facial identity database is determined as the current preset facial feature.

[0071] Specifically, the Euclidean distance between each real-time face feature and the current preset face feature can be calculated from among the multiple real-time face features corresponding to the current identity identifier, and used as the similarity between each real-time face feature and the current preset face feature.

[0072] Alternatively, the cosine similarity between each real-time face feature among the multiple real-time face features corresponding to the current identity and the current preset face feature can be calculated as the similarity between each real-time face feature among the multiple real-time face features corresponding to the current identity and the current preset face feature.

[0073] Alternatively, the Manhattan distance between each of the multiple real-time facial features corresponding to the current identity and the current preset facial feature can be calculated as the similarity between each of the multiple real-time facial features corresponding to the current identity and the current preset facial feature.

[0074] This application does not limit the specific method for calculating the similarity between each of the multiple real-time facial features corresponding to the current identity and the current preset facial features.

[0075] B2. Real-time facial features with similarity within the same similarity range are grouped into the same facial feature cluster to obtain multiple facial feature clusters corresponding to the current identity.

[0076] Here, the multiple facial feature clusters corresponding to the current identity identifier are multiple facial feature clusters corresponding to multiple similarity ranges. Each facial feature cluster corresponds to a similarity range, and different similarity ranges correspond to different facial poses.

[0077] Taking cosine similarity as the similarity metric, the total range of cosine similarity is [0, 1]. Assuming the similarity threshold is 0.6 (meaning that if the cosine similarity is less than 0.6, no identity identifier is obtained), it can be divided into 3 similarity ranges, namely [0.6, 0.8), [0.8, 0.95), and [0.95, 1]. These 3 similarity ranges correspond to the head-down posture, the side-face posture, and the front-face posture, respectively.

[0078] Based on the three similarity ranges mentioned above, real-time face features with similarity values ​​of [0.6, 0.8) are assigned to face feature cluster 1, real-time face features with similarity values ​​of [0.8, 0.95) are assigned to face feature cluster 2, and real-time face features with similarity values ​​of [0.95, 1] ​​are assigned to face feature cluster 3, thus obtaining three face feature clusters.

[0079] In steps B1-B2 above, the similarity between real-time facial features and facial features in the first facial identity database is used as the clustering index. Real-time facial features with similarity within the same range are grouped into the same facial feature cluster, thereby obtaining multiple facial feature clusters corresponding to the same identity. This allows multiple sets of real-time facial features corresponding to the same identity in the second facial database to cover different facial poses, thereby increasing the probability of successful face matching.

[0080] In another feasible implementation, the multiple real-time facial features corresponding to the current identity can be divided according to the facial quality score corresponding to the real-time facial features; the multiple real-time facial features corresponding to the current identity can be clustered through the following steps C1-C2 to obtain the multiple facial feature clusters corresponding to the current identity: C1. Determine the face quality score for each of the multiple real-time face features corresponding to the current identity identifier.

[0081] Here, the face quality score is used to indicate the face quality in the head image from which real-time face features are extracted; the higher the face quality score, the higher the face quality in the head image from which real-time face features are extracted; the lower the face quality score, the lower the face quality in the head image from which real-time face features are extracted.

[0082] The face quality score corresponding to each real-time face feature can be output by the face feature extraction model that extracts the real-time face features; or, during the process of extracting face features from the face image using the face feature extraction algorithm, the head image can be input into a preset face quality extraction model to obtain the face quality score corresponding to the head image, thereby obtaining the face quality score corresponding to the real-time face features extracted from the head image.

[0083] C2. Real-time facial features with facial quality scores belonging to the same quality score range are divided into the same facial feature cluster, resulting in multiple facial feature clusters corresponding to the current identity.

[0084] Here, the multiple facial feature clusters corresponding to the current identity identifier are multiple facial feature clusters corresponding to multiple quality score ranges. Each facial feature cluster corresponds to a quality score range. Different quality score ranges correspond to different facial qualities, and different facial qualities reflect different facial poses.

[0085] For example, the total range of face quality scores is [1, 100]. Considering that face quality scores less than 50 cannot match identity identifiers, three quality score ranges can be divided: [50, 70), [70, 90), and [90, 100]. Based on these three quality score ranges, real-time face features with a quality score of [50, 70) are assigned to face feature cluster 1, those with a quality score of [70, 90) are assigned to face feature cluster 2, and those with a quality score of [90, 100] are assigned to face feature cluster 3, thus obtaining three face feature clusters.

[0086] In steps C1-C2 above, the face quality score corresponding to the real-time face features is used as the clustering index. Real-time face features with face quality scores belonging to the same quality score range are divided into the same face feature cluster. This results in multiple face feature clusters corresponding to the same identity, which enables multiple sets of real-time face features corresponding to the same identity in the second face database to cover different face quality and face poses, thereby increasing the probability of successful face matching.

[0087] In another feasible implementation, multiple real-time facial features corresponding to the current identity can be clustered through the following steps D1-D5 to obtain multiple facial feature clusters corresponding to the current identity: D1. Among the multiple real-time facial features corresponding to the current identity identifier, randomly select multiple real-time facial features as the facial feature cluster centers corresponding to multiple feature clusters.

[0088] For example, among the multiple real-time facial features corresponding to the current identity identifier, three real-time facial features are randomly selected as the facial feature cluster centers corresponding to the multiple feature clusters, resulting in three facial feature cluster centers.

[0089] D2. Traverse multiple real-time facial features corresponding to the current identity identifier.

[0090] D3. Calculate the distance between the real-time face features traversed and the face feature cluster centers corresponding to multiple feature clusters, and divide the traversed real-time face features into the target feature clusters in multiple feature clusters until all real-time face features have been traversed.

[0091] The distance between the real-time face features traversed and the face feature cluster centers corresponding to multiple feature clusters can be any one of Euclidean distance, Manhattan distance, or cosine distance.

[0092] The target feature cluster is the feature cluster corresponding to the face feature cluster center with the smallest distance. The face feature cluster center with the smallest distance is the face feature cluster center that is most similar to the real-time face features traversed.

[0093] D4. Based on the real-time face features in each of the multiple feature clusters, update the face feature cluster center corresponding to each feature cluster.

[0094] In one feasible implementation, the mean of real-time facial features in each feature cluster is calculated and used as the facial feature cluster center corresponding to each feature cluster.

[0095] The updated feature clusters correspond to the face feature cluster centers represented as (X1, X2, ..., Xn). , 1≤i≤n, where n represents the dimension of the facial features, x ji Let represent the feature value of the j-th real-time face feature in the i-th feature dimension, and m represent the number of real-time face features in the feature cluster.

[0096] D5. Determine whether the face feature cluster centers corresponding to the updated multiple feature clusters are completely identical to the face feature cluster centers corresponding to the original multiple feature clusters.

[0097] If the face feature cluster centers corresponding to the updated multiple feature clusters are not completely the same as those corresponding to the original multiple feature clusters, proceed to step D2; if the face feature cluster centers corresponding to the updated multiple feature clusters are completely the same as those corresponding to the original multiple feature clusters, it indicates that the face feature cluster centers corresponding to the multiple feature clusters no longer change, and proceed to step D6.

[0098] D6. Cluster multiple features into clusters corresponding to the current identity identifier.

[0099] In steps D1-D6 above, by traversing real-time face features and calculating the feature cluster corresponding to the face feature cluster center with the smallest distance between the traversed real-time face features and the face feature cluster center, the face feature cluster center is updated until the face feature cluster center no longer changes, thereby obtaining multiple face feature clusters corresponding to the same identity identifier, which can realize adaptive division of face features.

[0100] S303, based on the multiple facial feature clusters corresponding to the current identity identifier, update the multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database.

[0101] Specifically, N real-time facial features are extracted from each facial feature cluster corresponding to the current identity identifier, and these N features are used as multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database. N is the number of sets of real-time facial features in the second facial identity database, and N > 1.

[0102] For example, if there are three facial feature clusters corresponding to the current identity, namely facial feature cluster 1, facial feature cluster 2, and facial feature cluster 3, and N=20, then 20 real-time facial features are extracted from facial feature cluster 1, 20 real-time facial features are extracted from facial feature cluster 2, and 20 real-time facial features are extracted from facial feature cluster 3, which are then used as multiple sets of real-time facial features corresponding to the current identity in the second facial identity database.

[0103] In one feasible implementation, N real-time facial features can be extracted from each of the multiple facial feature clusters corresponding to the current identity in the second facial identity database as follows: First, based on the feature scores corresponding to the real-time facial features in the target facial feature cluster, the real-time facial features in the target facial feature cluster are divided into N facial feature groups; the target facial feature cluster is any facial feature cluster among multiple facial feature clusters corresponding to the current identity identifier; different facial feature groups correspond to different feature score ranges.

[0104] Here, the feature score can be the similarity calculated in step B1 above or the face quality score determined in step C1 above.

[0105] Taking the face quality score determined by feature score C1 as an example, assuming the target cluster feature cluster is the face feature cluster 1 obtained in step C2 above, and assuming N=10, since the face feature cluster 1 corresponds to the quality score range [90, 100], the quality score range can be divided into 10 quality score ranges, namely [90, 91), [91, 92)...[99, 100]. According to the averaged quality score range, the real-time face features in face feature cluster 1 are divided into 10 face feature groups, assuming they are face feature group 1 to face feature group 10. The face quality score corresponding to the real-time face features in face feature group 1 is within the quality score range [90, 91), the face quality score corresponding to the real-time face features in face feature group 2 is within the quality score range [91, 92)... and the face quality score corresponding to the real-time face features in face feature group 10 is within the quality score range [99, 100].

[0106] Then, from each of the N face feature groups, a real-time face feature is extracted to obtain a set of real-time face features corresponding to the current identity in the second face identity database.

[0107] For example, if N face feature groups are respectively face feature group 1 to face feature group 10, then one real-time face feature is extracted from each of face feature group 1 to face feature group 10, forming a set of real-time face features.

[0108] By extracting real-time facial features from each facial feature cluster in the same way, multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database can be obtained.

[0109] The face features are grouped according to the feature scores corresponding to the real-time face features in the face feature clusters, and the real-time face features are extracted from the face feature groups as multiple sets of real-time face features corresponding to the same identity. This enables multiple sets of real-time face features for the same identity to cover different face poses.

[0110] Optionally, N real-time facial features can be randomly selected from each of the multiple facial feature clusters to serve as multiple sets of real-time facial features corresponding to the current identity in the second facial identity database. Alternatively, the N most recent real-time facial features can be selected from each of the multiple facial feature clusters to serve as multiple sets of real-time facial features corresponding to the current identity in the second facial identity database; the classroom image to which the head image corresponding to the N most recent real-time facial features belongs is the classroom image acquired at the latest time.

[0111] Multiple real-time facial features are extracted from each facial feature cluster to form multiple sets of real-time facial features, which can cover facial features under more facial poses as much as possible, thereby improving the probability of successful face matching.

[0112] In steps S301-S303 above, all facial features of the same identity identifier identified during the class period are obtained and clustered to form multiple facial feature clusters. Then, multiple sets of real-time facial features corresponding to the identity identifiers in the second facial identity database are updated according to the multiple facial feature clusters. This enables real-time updates to the second facial identity database, ensuring that the facial features in the second facial identity database can cover the facial features of students in the latest different facial states.

[0113] The method of this application has been described above; the apparatus of this application will be described below.

[0114] See Figure 4 , Figure 4 This is a schematic diagram of a classroom attendance device based on face recognition provided in an embodiment of this application, as shown below. Figure 4 As shown, the classroom attendance device 40 based on facial recognition includes: The face feature matching module 401 is used to match the face features of the current classroom with the preset face features in the first face identity database to obtain the first face matching result; the face features of the current classroom are obtained by extracting face features from classroom images during the current classroom period; the first face identity database is used to store the identity identifiers and preset face features of multiple pre-registered students. The face feature matching module 401 is further configured to, when the first face matching result indicates that no identity identifier corresponding to the face feature of the current classroom is matched, match the face feature of the current classroom with the real-time face features in the second face identity database to obtain a second face matching result; the second face identity database is used to store the identity identifiers and real-time face features of the multiple students, each identity identifier in the second face identity database corresponds to multiple sets of real-time face features, the multiple sets of real-time face features belong to the same student, and the real-time face features of different groups in the multiple sets of real-time face features belong to different face feature clusters, the face feature clusters are obtained by clustering the face features of historical classrooms, and the face features of historical classrooms are obtained by extracting face features from classroom images in the previous classroom period of the current classroom period; The attendance module 402 is used to generate the attendance result corresponding to the current class period based on the second face matching result when the second face matching result indicates that the identity identifier corresponding to the face feature of the current class is obtained.

[0115] It should be noted that the aforementioned face recognition-based classroom attendance device 40 can execute the face recognition-based classroom attendance method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments can be found in the face recognition-based classroom attendance method provided in the embodiments of this application.

[0116] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device 50 provided in an embodiment of this application. The computer device 50 includes a processor 501 and a memory 502. The memory 502 is connected to the processor 501, for example, via a bus.

[0117] Processor 501 is configured to support the computer device 50 in performing the corresponding functions in the methods described in the above method embodiments. Processor 501 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0118] Memory 502 is used to store program code, etc. Memory 502 may include volatile memory (VM), such as random access memory (RAM); memory 502 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 502 may also include combinations of the above types of memory.

[0119] The memory 502 is used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the face recognition-based classroom attendance method in the embodiments of this application. The processor executes various functional applications and data processing of the face recognition-based classroom attendance method by running the non-volatile software programs, instructions, and modules stored in the memory, thereby realizing the functions of the face recognition-based classroom attendance method provided in the above method embodiments.

[0120] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the face recognition-based classroom attendance device. In some embodiments, the memory may include memory remotely configured relative to the processor, which can be connected to the face recognition-based classroom attendance device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0121] The one or more modules are stored in the memory. When executed by the one or more processors, they perform the classroom attendance method based on face recognition in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0122] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0123] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing associated hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0124] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A classroom attendance method based on facial recognition, characterized in that, include: The facial features of the current classroom student are matched with the preset facial features in the first facial identity database to obtain the first facial matching result; The current classroom facial features are obtained by extracting facial features from classroom images during the current classroom period. The first facial identity database is used to store the identity identifiers and preset facial features of multiple pre-registered students. If the first face matching result indicates that no identity identifier corresponding to the current classroom face feature is matched, the current classroom face feature is matched with the real-time face feature in the second face identity database to obtain a second face matching result. The second face identity database is used to store the identity identifiers and real-time face features of the multiple students. Each identity identifier in the second face identity database corresponds to multiple sets of real-time face features. The multiple sets of real-time face features belong to the same student. The real-time face features of different groups in the multiple sets of real-time face features belong to different face feature clusters. The face feature clusters are obtained by clustering the face features of historical classrooms. The face features of historical classrooms are obtained by extracting face features from classroom images in the previous classroom period of the current classroom period. When the second face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, the attendance result corresponding to the current classroom time period is generated based on the second face matching result.

2. The method according to claim 1, characterized in that, The method further includes: When the first face matching result is used to indicate the identity identifier corresponding to the face features of the current classroom, the attendance result corresponding to the current classroom time period is generated based on the first face matching result.

3. The method according to claim 1 or 2, characterized in that, The method further includes: If the first face matching result or the second face matching result is used to indicate that the identity identifier corresponding to the face feature of the current classroom is obtained, all face features corresponding to the current identity identifier within the current classroom time period are obtained as multiple real-time face features corresponding to the current identity identifier; the current identity identifier is the identity identifier corresponding to the face feature of the current classroom. Cluster the multiple real-time facial features to obtain multiple facial feature clusters corresponding to the current identity identifier; Based on the multiple facial feature clusters, update the multiple sets of real-time facial features corresponding to the current identity in the second facial identity database.

4. The method according to claim 3, characterized in that, The process of clustering the multiple real-time facial features to obtain multiple facial feature clusters corresponding to the current identity includes: Calculate the similarity between each of the plurality of real-time face features and a current preset face feature; the current preset face feature is the preset face feature corresponding to the current identity identifier in the first face identity database; Real-time facial features with similarity within the same range are grouped into the same facial feature cluster, resulting in multiple facial feature clusters corresponding to the current identity identifier.

5. The method according to claim 3, characterized in that, The step of clustering the multiple real-time facial features to obtain multiple facial feature clusters corresponding to the current identity includes: Determine the face quality score corresponding to each of the plurality of real-time face features; Real-time facial features with facial quality scores belonging to the same quality score range are grouped into the same facial feature cluster, resulting in multiple facial feature clusters corresponding to the current identity identifier.

6. The method according to claim 3, characterized in that, The process of clustering the multiple real-time facial features to obtain multiple facial feature clusters corresponding to the current identity includes: Iterate through the multiple real-time facial features; Calculate the distance between the real-time face features traversed and the face feature cluster centers corresponding to multiple feature clusters, and divide the traversed real-time face features into the target feature clusters among the multiple feature clusters until all the multiple real-time face features have been traversed. The target feature cluster is the feature cluster corresponding to the face feature cluster center with the smallest distance. Based on the real-time face features in each of the multiple feature clusters, update the face feature cluster center corresponding to each feature cluster; If the facial feature cluster centers corresponding to the multiple feature clusters no longer change, the multiple feature clusters are determined as the multiple facial feature clusters corresponding to the current identity identifier.

7. The method according to claim 3, characterized in that, The step of updating multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database based on the multiple facial feature clusters includes: From each of the multiple facial feature clusters, N real-time facial features are extracted as multiple sets of real-time facial features corresponding to the current identity identifier in the second facial identity database, where N is the number of a set of real-time facial features in the second facial identity database, and N > 1.

8. The method according to claim 7, characterized in that, The step of extracting N real-time facial features from each of the multiple facial feature clusters as multiple sets of real-time facial features corresponding to the current identity in the second facial identity database includes: Based on the feature scores corresponding to the real-time face features in the target face feature cluster, the real-time face features in the target face feature cluster are divided into N face feature groups; the target face feature cluster is any face feature cluster among the multiple face feature clusters; different face feature groups correspond to different feature score ranges; From each of the N facial feature groups, extract a real-time facial feature to obtain a set of real-time facial features corresponding to the current identity identifier in the second facial identity database.

9. A computer device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, and the processor, when executing the one or more computer programs, causing the computer device to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-8.

Images