Automatic teacher face acquisition method and system, storage medium and program product

Abstract

The invention discloses a teacher face automatic acquisition method and system, a storage medium and a program product, and relates to the technical field of education. And performing similarity comparison on the clustered face features and the feature library to judge the teacher identity, extracting the course teacher information of the current classroom from the course schedule, and then judging whether the course teacher information exists in the teacher face feature library or not. And if not, subsequent related steps are executed. The class schedule is used as the guidance of teaching arrangement, it is ensured that the obtained teacher information is closely related to the current classroom and needs to be included in the face feature library, repeated adding and mistaken adding are prevented, the purity and accuracy of the teacher face feature library information are effectively ensured, a candidate teacher face set with the most face pictures is selected firstly, and then the candidate teacher face set with the most face pictures is selected; according to the method, the range can be quickly locked, the face features with high quality evaluation are selected, the data volume is reduced, in conclusion, the teacher face can be automatically collected, and the face collection modeling difficulty is reduced.

Description

Technical Field

[0001] The present application relates to the field of educational technology, and in particular to a method, system, storage medium and program product for automatically collecting teacher faces. Background Art

[0002] With the improvement of teaching management requirements, routine teaching inspection has become an important part of ensuring teaching quality. Among them, whether teachers teach on time according to the schedule, avoid being late, leaving early, missing classes, and changing classes without permission are important contents of routine teaching inspection. The core of this is the teacher's face recognition.

[0003] At present, the common face recognition technology mainly uses feature comparison to confirm identity, please refer to Figure 2 , Figure 2 This is a schematic diagram of the face recognition technology in related technologies. The premise of face recognition is to collect face photos, train them, and store the feature information templates in the database. In actual recognition, the system extracts the face to be recognized from the surveillance video, extracts the feature information of the face, and calculates and compares the similarity of these feature information with the feature templates stored in the database to determine the identity of the person being recognized.

[0004] In actual applications, the teacher face collection that teacher face attendance relies on faces great implementation difficulties. This problem is particularly prominent in primary and secondary school environments. Specifically, there are two main difficulties: one is the heavy teaching tasks, especially in high school, when teachers devote their main energy to preparing for the college entrance examination; the other is that the face collection process is relatively complicated. Every time the system is deployed or school teachers are transferred, it is necessary to collect teacher photos, feature training, and system verification. The process is cumbersome and time-consuming.

[0005] Therefore, related technologies have problems such as difficulty in facial collection and modeling due to heavy school teaching tasks and high teacher mobility, as well as cumbersome and time-consuming system deployment and update processes. Summary of the invention

[0006] The present application provides a teacher face automatic collection method, system, storage medium and program product, which are used to reduce the difficulty of face collection modeling and system update pressure caused by teacher mobility.

[0007] In a first aspect, the present application provides a method for automatically collecting teacher faces, comprising: obtaining a face set of to-be-identified persons from a classroom video stream; Compare the similarity between the facial features in the face set and the feature vectors in the teacher's facial feature library; In the case that the teacher identity of the lecturer cannot be identified through the teacher face feature database; the teacher information of the current class teacher is extracted from the class schedule; the lecturer is the teacher of the current class; Determine whether the teacher face feature database has the teacher information of the teacher in charge of the course; If there is no teacher information of the teacher, then a candidate teacher face is selected; the candidate teacher face is a face set with the most face images; Select the facial features of the candidate teacher faces whose quality assessment results are greater than a threshold; and create the teacher information of the lecturer in the teacher facial feature library.

[0008] By adopting the above technical solution, the identity of the teacher is determined by comparing the clustered facial features with the feature library for similarity, extracting the current class teacher information from the class schedule, and then judging whether the teacher information exists in the teacher face feature library. If not, the subsequent related steps are executed. The class schedule serves as a guide for teaching arrangements, ensuring that the teacher information obtained is closely related to the current class and needs to be included in the face feature library, preventing repeated additions and mis-additions, and effectively ensuring the purity and accuracy of the teacher face feature library information. First, select the candidate teacher face set with the most face images, which can quickly lock the range, and then select the face features with high quality assessment to reduce the amount of data. In summary, the teacher's face can be automatically collected to reduce the difficulty of face collection modeling and system update pressure caused by teacher mobility.

[0009] In conjunction with some embodiments of the first aspect, in some embodiments, the step of creating the teacher information of the lecturer in the teacher face feature library specifically includes: Determine whether the teacher information of the lecturer exists in the teacher face feature database to be confirmed; If the lecturer's teacher information does not exist, the lecturer's teacher information is created in the teacher face feature database to be confirmed based on the lecturer's facial features; If the teacher information of the lecturer exists, the facial features of the lecturer are compared with the facial features in the teacher information of the lecturer in the teacher facial feature database to be confirmed for similarity; If the similarity is less than the repetition threshold, the facial features of the lecturer are added to the teacher information of the lecturer in the teacher facial feature database to be confirmed; After the teacher information in the teacher facial feature database is confirmed, the confirmed teacher information is added to the teacher facial feature database.

[0010] By adopting the above technical solution, the facial feature database of teachers to be confirmed is processed conditionally to avoid the influx of invalid data when the teacher mobility is high, thereby ensuring the validity and manageability of the database data. After subsequent confirmation, it is added to the teacher facial feature database, which can ensure the accuracy and completeness of the database data, improve the recognition accuracy of the attendance system, reduce attendance errors, ensure system reliability, and reduce the difficulty of facial collection and modeling and system update pressure caused by teacher mobility.

[0011] In conjunction with some embodiments of the first aspect, in some embodiments, the step of obtaining a set of faces to be identified from a classroom video stream specifically includes: Intercepting pictures from the classroom video stream at preset time intervals to obtain a picture set; Extract facial features from a collection of images; Cluster the facial features to obtain a set of faces whose identities are to be determined.

[0012] By adopting the above technical solution, pictures are captured from the classroom video stream at preset time intervals to obtain a picture set, facial features are extracted from the picture set, individuals are accurately distinguished, and confusion is avoided; facial features are clustered to obtain a face set of individuals whose identities are to be determined, and personnel information is associated and integrated.

[0013] In conjunction with some embodiments of the first aspect, in some embodiments, the step of extracting facial features from a picture set specifically includes: Detect faces from a picture set to obtain multiple face pictures; Perform face quality assessment on multiple face images, including face angle assessment, lighting condition assessment, clarity assessment, and occlusion assessment; Filter out the face images whose quality assessment results are lower than the preset quality standard to obtain qualified face images; Convolutional neural networks are used to extract feature points from qualified face images and generate facial features.

[0014] By adopting the above technical solution, through multi-dimensional quality assessment such as face angle, lighting conditions, clarity and occlusion, low-quality face images are filtered out through preset quality standards, avoiding these images from participating in the subsequent feature extraction and comparison process. This not only saves computing resources, but also eliminates the interference that low-quality face images may cause, effectively improving the accuracy of the final recognition results.

[0015] In conjunction with some embodiments of the first aspect, in some embodiments, the step of clustering facial features to obtain a set of faces whose identities are to be determined specifically includes: Initialize the face set; For each newly acquired facial feature, the newly acquired facial feature is compared with all facial features in the face set for similarity; When there is a similarity greater than the clustering threshold, determine the face set to which the face feature with the greatest similarity belongs, and add the newly acquired face feature to the corresponding face set; When there is no similarity greater than the clustering threshold, a new face set is created, and the newly acquired facial features are added to the new face set.

[0016] By adopting the above technical solution, intelligent grouping of facial features is achieved through dynamic clustering. First, the system dynamically creates and updates face sets based on similarity, so that facial features of the same teacher from different angles and expressions can be classified into the same set. When the similarity between the newly acquired facial features and the existing set does not reach the clustering threshold, the system will create a new set. This adaptive clustering mechanism not only ensures the accuracy of classification, but also improves the flexibility of the system in processing facial features of different teachers.

[0017] In combination with some embodiments of the first aspect, in some embodiments, the facial features in the face set are compared with the feature vectors in the teacher face feature library for similarity; when the similarity is greater than a preset threshold, the step of confirming the teacher identity of the lecturer specifically includes: Sort the facial features in the face set from high to low according to the quality assessment results; According to the sorting results, the facial features are compared with the feature vectors in the teacher's facial feature library for similarity; When the similarity is greater than a preset threshold, the identity to be determined corresponding to the facial feature is determined as the corresponding teacher identity in the teacher facial feature library.

[0018] By adopting the above technical solutions, a hierarchical comparison mechanism based on quality assessment is established. The features in the face set are sorted according to the quality assessment results, and high-quality face features are used for identity comparison first. This strategy not only improves the efficiency of recognition, but also reduces the probability of recognition errors by giving priority to matching high-quality samples. At the same time, once high-quality face features are successfully matched, the system can confirm the identity in a timely manner, avoiding unnecessary low-quality feature comparisons and improving the accuracy and efficiency of overall recognition.

[0019] In combination with some embodiments of the first aspect, in some embodiments, before the step of acquiring a face set to be identified from a classroom video stream, the method further includes: Receive original photos and teacher identification information; Adjusting the size of the original photo to a preset size to obtain a first adjusted photo; Performing below-threshold adjustment on the face shape in the first adjusted photo to obtain a second adjusted photo; Extracting feature points from the original photo, the first adjusted photo, and the second adjusted photo using a convolutional neural network and generating a feature vector; The feature vector, teacher identity information, collection time, collection method, and hit count are stored in the teacher face feature library.

[0020] By adopting the above technical solution, a multi-dimensional feature vector containing original photos and adjusted photos is established for each teacher, expanding the expression range of facial features. Among them, the original photos are adjusted below the threshold, so that the feature vectors in the feature library are richer, and the photo features with different degrees of beauty effects are reversely simulated. By clustering the facial features extracted from these pictures, the facial features of the same teacher at different times are automatically classified into the same set, so that more facial features can be used for comparison during identity recognition, improving the reliability of recognition. By comparing the clustered facial features with the feature library for similarity and setting a reasonable preset threshold, while providing sufficient recognition fault tolerance space, the reliability of the recognition results is ensured, thereby solving the problem of recognition failure caused by beauty, filters and other effects, and improving the accuracy of face recognition.

[0021] In a second aspect, the present application provides a teacher's face automatic acquisition system, the teacher's face automatic acquisition system comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and one or more processors call the computer instructions to enable the teacher's face automatic acquisition system to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0022] In a third aspect, the present application provides a computer program product comprising instructions, characterized in that when the computer program product runs on an automatic teacher face acquisition system, the automatic teacher face acquisition system executes the method described in the first aspect and any possible implementation method of the first aspect.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, characterized in that when the instructions are run on an automatic teacher face acquisition system, the automatic teacher face acquisition system executes the method described in the first aspect and any possible implementation method of the first aspect.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The identity of the teacher is determined by comparing the clustered facial features with the feature library for similarity, extracting the current class teacher information from the class schedule, and then determining whether the teacher information exists in the teacher face feature library. If not, the subsequent related steps are executed. The class schedule serves as a guide for teaching arrangements, ensuring that the teacher information obtained is closely related to the current class and needs to be included in the face feature library, preventing repeated additions and mis-additions, and effectively ensuring the purity and accuracy of the teacher face feature library information. First, select the candidate teacher face set with the most face images to quickly lock the range, and then select the face features with high quality assessment to reduce the amount of data. In summary, the teacher's face can be automatically collected to reduce the difficulty of face collection modeling and system update pressure caused by teacher mobility.

[0025] 2. Process the facial feature database of teachers to be confirmed based on their conditions, and add them conditionally to avoid the influx of invalid data when teacher mobility is high, to ensure the validity and manageability of the database data. After subsequent confirmation, add them to the teacher facial feature database to ensure that the database data is accurate and complete, improve the recognition accuracy of the attendance system, reduce attendance errors, ensure system reliability, and reduce the difficulty of facial collection and modeling and system update pressure caused by teacher mobility.

[0026] 3. By establishing a multi-dimensional feature vector containing the original photo and the adjusted photo for each teacher, the expression range of facial features is expanded. Among them, the original photo is adjusted below the threshold to make the feature vector in the feature library richer, and the photo features with different degrees of beauty effects are reversely simulated. By clustering the facial features extracted from these pictures, the facial features of the same teacher at different times are automatically classified into the same set, so that more facial features can be used for comparison during identity recognition, which improves the reliability of recognition. By comparing the clustered facial features with the feature library for similarity and setting a reasonable preset threshold, while providing sufficient recognition fault tolerance space, the reliability of the recognition results is ensured, thereby solving the problem of recognition failure caused by beauty, filters and other effects, and improving the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the automatic teacher face collection method in the embodiment of the present application; Figure 2 It is a schematic diagram of the face recognition technology in the related technology; Figure 3 It is a structural diagram of the teacher face automatic collection method in the embodiment of the present application; Figure 4 This is a schematic diagram of part of the data format of the teacher's face feature library in the embodiment of the present application; Figure 5This is a schematic diagram of the data format of a face set in an embodiment of the present application; Figure 6 It is an exemplary hardware structure diagram of the automatic teacher face acquisition system in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0030] See also Figure 1 and Figure 3 , Figure 1 This is a flow chart of the automatic teacher face collection method in the embodiment of the present application; Figure 3 It is a structural diagram of the teacher face automatic collection method in the embodiment of the present application; S101, obtaining a set of faces to be identified from a classroom video stream; In some embodiments, step S101 specifically includes: S1011, capturing pictures from the classroom video stream at preset time intervals to obtain a picture set; The preset time interval refers to the fixed time period between two image acquisitions, which is used to control the frequency of image acquisition by the system. The classroom video stream refers to the continuous video data collected in real time by the camera equipment installed in the classroom. The picture set refers to a collection of multiple static images captured from the video stream within a certain period of time.

[0031] S1012, extracting facial features from the picture set; Among them, facial features refer to the digital expressions extracted from images to describe facial features, including information such as the location of facial key points, texture features, and geometric features.

[0032] In some embodiments, the system first performs face detection on each image in the image set to locate the face area in the image. Then, a pre-trained deep neural network model or a neural network model in related technologies is used to extract a high-dimensional feature vector from the pre-processed face image to obtain face features.

[0033] In some specific embodiments, S1012 specifically includes: S10121, detecting faces from a picture set to obtain multiple face pictures; The face image refers to a sub-image cropped from the original image that only contains the face area.

[0034] In some embodiments, the system uses the YOLO (You Only Look Once) target detection model to detect faces in input images. The model can quickly and accurately locate all face areas in the image and crop each detected face area into an independent small image. This preprocessing method can decompose an original image containing multiple faces into multiple small images containing only a single face. Each small image will be used as an independent sample for subsequent feature extraction processing, thereby improving the system's ability to handle complex scenes.

[0035] S10122. Performing face quality assessment on multiple face images, where the face quality assessment includes face angle assessment, lighting condition assessment, clarity assessment, and occlusion assessment; Among them, face quality assessment refers to the process of quantitatively analyzing multiple key indicators of face images. Face angle assessment refers to measuring the degree to which the face posture deviates from the front. Lighting condition assessment is used to measure the light and dark distribution and contrast of the image. Clarity assessment refers to analyzing the degree of retention of image details. Occlusion assessment refers to detecting the degree to which the face area is blocked by other objects.

[0036] In some specific embodiments, Face angle assessment: Calculate the face orientation by detecting several key points of the face; evaluate the rotation angle in three dimensions: left and right deflection angle (yaw angle), up and down nodding angle (pitch angle), and head tilt angle (roll angle); the closer the angle is to the front, the higher the score.

[0037] Lighting condition assessment: Analyze the brightness histogram distribution of the image and calculate the overall brightness mean and variance; detect the local contrast of key facial areas (eyes, nose, mouth); evaluate whether there are severe shadows or overexposure; calculate the lighting uniformity and contrast scores of the facial area.

[0038] Clarity assessment: Calculate the gradient amplitude of the image and assess the edge clarity Occlusion assessment: Calculate the visibility of key facial areas; assess the impact of occlusions on facial feature extraction; ensure that important facial features (such as eyes, nose, etc.) are not blocked, without limitation here.

[0039] S10123, filtering out face images whose quality assessment results are lower than a preset quality standard to obtain qualified face images; In some embodiments, the system first loads the preset quality standard parameters, including the threshold requirements for each indicator. Then the quality assessment results of each face image are compared with the standard. If an indicator is lower than the threshold, the image will be marked as unqualified. The system only retains the images that meet the requirements for all indicators to form a qualified sample set. S10124. Use a convolutional neural network to extract feature points from qualified face images and generate facial features.

[0040] It should be noted that the principle and process of this step are similar to those of step S1012. For the related principles and processes, please refer to step S1012 and are not limited here.

[0041] It can be seen that through multi-dimensional quality assessment such as face angle, lighting conditions, clarity and occlusion, low-quality face images are filtered out through preset quality standards, avoiding these images from participating in the subsequent feature extraction and comparison process. This not only saves computing resources, but also eliminates the interference that low-quality face images may bring, effectively improving the accuracy of the final recognition results.

[0042] See also Figure 5 , Figure 5 This is a schematic diagram of the data format of a face set in an embodiment of the present application; S1013, clustering facial features to obtain a set of faces whose identities are to be determined; Clustering refers to the process of grouping facial features with similar characteristics into the same group. A facial feature set refers to a group of facial features with similar characteristics obtained after clustering. An identity to be determined refers to a target object whose specific identity information needs to be confirmed through subsequent steps.

[0043] In some embodiments, the system first calculates the similarity matrix between all extracted facial features to evaluate the similarity between different facial features. Then, based on preset clustering parameters (such as clustering distance threshold, minimum number of samples, etc.), facial features with higher similarity are classified into the same category.

[0044] It should be noted that, in some embodiments, during the clustering process, the system will consider time continuity, that is, similar facial features in adjacent time periods are more likely to belong to the same person. For example: the system will reflect time continuity through time weights. Specifically, for two facial features to be matched, the system not only considers their feature similarity scores, but also calculates the time weight based on their acquisition time difference. The time weight decreases as the time difference increases, for example, the weight value is obtained by "1 divided by (1 plus the time difference divided by the class time)". The system multiplies the feature similarity score by the time weight to obtain the final matching score, so that similar facial features in adjacent time periods are more easily clustered into the same category.

[0045] It should be noted that, considering the particularity of classroom teaching scenarios, teachers usually teach students and are always within the camera's shooting range. Therefore, in the collected face images, the frequency of teacher's face appearance is significantly higher than that of students. Based on this feature, after completing face clustering, the system can regard the cluster with the largest sample size as the teacher's face set. In the subsequent face recognition process, the system can prioritize or only compare features with the largest cluster. This strategy not only conforms to the characteristics of the actual scene, but also significantly improves recognition efficiency.

[0046] It can be seen that by intercepting pictures from the classroom video stream at preset time intervals to obtain a picture set, facial features in the picture set are extracted to accurately distinguish individuals and avoid confusion; facial features are clustered to obtain a face set with an identity to be determined, and personnel information is associated and integrated.

[0047] In some embodiments, step S1013 specifically includes: S10131, initializing a face set; In some embodiments, the system first creates an empty face set list for subsequent storage of facial feature sets of different persons.

[0048] S10132. For each newly acquired facial feature, perform a similarity comparison between the newly acquired facial feature and all facial features in the face set; This step is performed after acquiring new facial features, and is used to find the best matching face set. In some embodiments, the system compares the newly acquired facial features with the features in all existing face sets one by one. When comparing, a metric such as cosine similarity or Euclidean distance is used to calculate the similarity between feature vectors.

[0049] S10133. When there is a similarity greater than the clustering threshold, determine the face set to which the face feature with the greatest similarity belongs, and add the newly acquired face feature to the corresponding face set; In some embodiments, the system first checks whether the maximum similarity exceeds a preset clustering threshold. If it exceeds the threshold, the system adds the new feature to the set of features with the highest similarity. At the same time, the system updates the statistical information of the set, such as the number of features, feature centers, etc.

[0050] In some embodiments, to ensure the quality of the collection, the system may reorder or filter the features in the collection.

[0051] S10134. When there is no similarity greater than the clustering threshold, create a new face set and add the newly acquired facial features to the new face set.

[0052] In some embodiments, the system creates a separate feature set for the new identity, then adds the newly acquired facial features as the first sample in the set and records the relevant timestamp and feature information.

[0053] It can be seen that intelligent grouping of facial features is achieved through dynamic clustering. First, the system dynamically creates and updates face sets based on similarity, so that facial features of the same teacher from different angles and expressions can be classified into the same set. When the similarity between the newly acquired facial features and the existing set does not reach the clustering threshold, the system will create a new set. This adaptive clustering mechanism not only ensures the accuracy of classification, but also improves the flexibility of the system in processing facial features of different teachers.

[0054] S102, performing a similarity comparison between the facial features in the face set and the feature vectors in the teacher's facial feature library; In some embodiments, the system first calculates the mean vector of all feature vectors in each face set obtained by clustering as the representative feature of the category, and then calculates the similarity between the representative feature and all feature vectors of each teacher stored in the teacher face feature library.

[0055] In some specific embodiments, a cosine similarity calculation method is used, which is not limited here.

[0056] In some specific embodiments, the Euclidean distance calculation method is adopted, which is not limited here.

[0057] When the similarity is greater than the preset threshold, the lecturer’s identity as a teacher is confirmed.

[0058] When the similarity is not greater than the preset threshold, the teacher identity of the lecturer cannot be confirmed.

[0059] In some embodiments, step S102 specifically includes: S1021, sorting the facial features in the face set from high to low according to the quality assessment results; It should be noted that the face quality assessment has already been performed in step S1032.

[0060] S1022, according to the sorting results, the facial features are sequentially compared with the feature vectors in the teacher's facial feature library for similarity; It should be noted that the similarity comparison may refer to step S102, which will not be described in detail here.

[0061] It should be noted that, in some embodiments, the system adopts an optimized feature matching strategy in consideration of the characteristics of classroom teaching scenarios. When performing similarity comparison, the system only selects the largest face set (usually the teacher's face set) for comparison. In the teacher feature library, the system records the historical hit counts of each feature vector. Feature vectors with higher hit counts have higher matching priority. This priority strategy based on frequency of use not only conforms to the high frequency of teachers in actual scenarios, but also improves recognition efficiency.

[0062] It should be noted that the system uses a hierarchical comparison strategy for identity recognition. Specifically, the system first uses the highest quality features in the face set to compare with the teacher feature library for the first round. If the first round does not produce satisfactory results, the second round of comparison is performed using the second highest quality features, and so on. This progressive comparison method not only ensures a high recognition accuracy rate, but also stops in time after obtaining satisfactory results, avoiding unnecessary computational overhead.

[0063] S1023. When the similarity is greater than a preset threshold, the identity to be determined corresponding to the facial feature is determined as the corresponding teacher identity in the teacher facial feature library.

[0064] It can be seen that a hierarchical comparison mechanism based on quality assessment has been established. The features in the face set are sorted according to the quality assessment results, and high-quality face features are used first for identity comparison. This strategy not only improves the efficiency of recognition, but also reduces the probability of recognition errors by giving priority to matching high-quality samples. At the same time, once high-quality face features are successfully matched, the system can confirm the identity in a timely manner, avoiding unnecessary low-quality feature comparisons and improving the accuracy and efficiency of overall recognition.

[0065] In some embodiments, the system first checks whether the maximum similarity exceeds a preset recognition threshold. If it exceeds the threshold, the system will associate the successfully matched teacher information (including name, work number, department, etc.) in the teacher feature library with the corresponding facial feature set. At the same time, the system will update the relevant database records and may trigger corresponding business processes, such as attendance records or behavior analysis.

[0066] S103, when the teacher identity of the lecturer cannot be identified through the teacher face feature database; extract the teacher information of the teacher of the current class from the class schedule; the lecturer is the teacher of the current class; Among them, the timetable refers to a document or spreadsheet that records information related to the school's teaching arrangements. It refers to information that lists in detail key content such as the teachers, teaching time, and teaching location of each class according to dimensions such as courses, classes, and time. It is used to indicate the planning and arrangements of the entire teaching activities.

[0067] In some embodiments, the system first locates the class schedule record corresponding to the current class, and then extracts the relevant teacher information of the marked teacher from it, thereby obtaining the name of the current teacher, the possible associated identity number, and other pre-registered information about the teacher.

[0068] S104, judging whether the teacher face feature database has the teacher information of the teacher in charge of the course; In some embodiments, the system will take the teacher information extracted from the class schedule, such as the teacher's identification number or name and other key identifiers, and perform data retrieval and matching in the teacher's facial feature library to see if there is a corresponding complete facial feature data record, so as to determine whether the teacher's facial features have been included in the library.

[0069] S105. If there is no teacher information of the teacher, then select candidate teacher faces; the candidate teacher faces are the face set with the most face images; In some embodiments, the system first determines whether the similarity of face recognition is lower than a preset threshold. If it is lower than the threshold, it means that the current teacher may not be recorded in the teacher face feature library. Then the system selects the set with the largest number of pictures from all the face sets to be processed as the candidate teacher face. Because a large number of pictures usually means that the teacher appears in the picture for a longer time and is more likely to be the actual teacher.

[0070] S106, selecting facial features of candidate teacher faces whose quality assessment results are greater than a threshold; and creating teacher information of the lecturer in the teacher facial feature database.

[0071] It should be noted that the principle and process of this step are similar to those of step S10122. Please refer to step S10122 and will not be repeated here.

[0072] It can be seen that the identity of the teacher is determined by comparing the clustered facial features with the feature library for similarity, extracting the current class teacher information from the class schedule, and then judging whether the teacher information exists in the teacher face feature library. If it does not exist, the subsequent related steps are executed. The class schedule serves as a guide for teaching arrangements, ensuring that the teacher information obtained is closely related to the current class and needs to be included in the face feature library, preventing repeated additions and mis-additions, and effectively ensuring the purity and accuracy of the teacher face feature library information. First, select the candidate teacher face set with the most face images to quickly lock the range, and then select the face features with high quality assessment to reduce the amount of data. In summary, the teacher's face can be automatically collected to reduce the difficulty of face collection modeling and system update pressure caused by teacher mobility.

[0073] In some embodiments, step S106 specifically includes: S1061, determining whether the teacher information of the lecturer exists in the teacher face feature database to be confirmed; In some embodiments, the system will take the teacher information extracted from the class schedule, such as the teacher's identification number or name and other key identifiers, and perform data retrieval and matching in the teacher's facial feature library to see if there is a corresponding complete facial feature data record, so as to determine whether the teacher's facial features have been included in the library.

[0074] S1062. If the teacher information of the lecturer does not exist, the teacher information of the lecturer is created in the teacher face feature database to be confirmed according to the facial features of the lecturer; Among them, the teacher face feature library represents a data set used to store teacher facial feature information, including the identity information of multiple teachers and their corresponding facial feature data. Teacher identity information refers to data used to uniquely identify the teacher's identity, such as ID, work number, name and other basic information. The feature vector represents the digital representation form into which the face image is converted, which is used for subsequent face recognition comparison. The original photo refers to the teacher's facial image without any beauty or filter processing. The adjusted photo refers to the image after the original photo has been beautified or filtered to a certain extent, and the processing degree must be controlled within the adjustment threshold range. The adjustment threshold is used to indicate the maximum allowable image processing degree to ensure that the processed image still maintains a high recognition value.

[0075] In some specific embodiments, multiple photos of all teachers are collected uniformly; all photos are standardized, including resizing and lighting normalization; face regions are extracted using a face detection algorithm; multiple adjusted versions of each original photo are generated to varying degrees; feature vectors of all photos are extracted in batches; and all feature vectors and corresponding identity information are imported into a database at one time, without limitation here.

[0076] S1063. If the teacher information of the lecturer exists, the facial features of the lecturer are compared with the facial features in the teacher information of the lecturer in the teacher facial feature database to be confirmed for similarity; In some embodiments, the system first calculates the mean vector of all feature vectors in each face set obtained by clustering as the representative feature of the category, and then calculates the similarity between the representative feature and all feature vectors of each teacher stored in the teacher face feature library.

[0077] In some specific embodiments, a cosine similarity calculation method is used, which is not limited here.

[0078] In some specific embodiments, the Euclidean distance calculation method is adopted, which is not limited here.

[0079] S1064, if the similarity is less than the repetition threshold, the facial features of the lecturer are added to the teacher information of the lecturer in the teacher facial feature database to be confirmed; In some embodiments, when it is found that the information of the existing speaker in the library is incomplete or has deviations, it is supplemented and corrected. The facial feature data of the newly acquired speaker will be added to the teacher information record of the existing speaker in the teacher facial feature library to be confirmed according to the established data format and storage rules, so as to improve the accuracy and reliability of the entire teacher identification system in identifying the speaker.

[0080] S1065. After the teacher information in the teacher facial feature database is confirmed, the confirmed teacher information is added to the teacher facial feature database.

[0081] This step is performed after manual confirmation is completed. In some embodiments, the system will wait for the administrator to review the newly added features in the pending confirmation feature library. Once confirmed, the system will transfer these features from the pending confirmation feature library to the formal teacher face feature library, so that they can officially participate in the subsequent face recognition process.

[0082] It can be seen that by processing the facial feature database of teachers to be confirmed, their conditional addition can avoid the influx of invalid data when the teacher mobility is high, ensure the validity and manageability of the database data, and add them to the teacher facial feature database after subsequent confirmation, which can ensure the accuracy and completeness of the database data, improve the recognition accuracy of the attendance system, reduce attendance errors, ensure system reliability, and reduce the difficulty of facial collection and modeling and system update pressure caused by teacher mobility.

[0083] In actual applications, in the education scenario, the photos that the facial feature library relies on are often distorted, such as photos taken with mobile phones, "artistic photos", and photos that have been retouched. These photos are generally beautified by beautification processing, filter effects, etc., resulting in a large difference between the stored feature templates and the facial features of teachers in actual classroom teaching environments. When the system uses the feature templates established by these beautified photos for identity recognition, recognition failure or recognition errors are likely to occur.

[0084] Therefore, the face recognition effect of the related technology is poor, and recognition failure or recognition error is prone to occur.

[0085] In some embodiments, before step S101, the method further includes: S201, establishing a teacher face feature database, the teacher face feature database includes teacher information of each teacher, the teacher information includes teacher identity information and feature vectors of multiple photos, the multiple photos include original photos and adjusted photos obtained by adjusting the original photos below an adjustment threshold; Among them, the teacher face feature library represents a data set used to store teacher facial feature information, including the identity information of multiple teachers and their corresponding facial feature data. Teacher identity information refers to data used to uniquely identify the teacher's identity, such as ID, work number, name and other basic information. The feature vector represents the digital representation form into which the face image is converted, which is used for subsequent face recognition comparison. The original photo refers to the teacher's facial image without any beauty or filter processing. The adjusted photo refers to the image after the original photo has been beautified or filtered to a certain extent, and the processing degree must be controlled within the adjustment threshold range. The adjustment threshold is used to indicate the maximum allowable image processing degree to ensure that the processed image still maintains a high recognition value.

[0086] It should be noted that this step needs to be performed when the current teacher's face is recognized.

[0087] In some specific embodiments, multiple photos of all teachers are collected uniformly; all photos are standardized, including resizing and lighting normalization; face regions are extracted using a face detection algorithm; multiple adjusted versions of each original photo are generated to varying degrees; feature vectors of all photos are extracted in batches; and all feature vectors and corresponding identity information are imported into a database at one time, without limitation here.

[0088] See also Figure 4 , Figure 4 This is a schematic diagram of part of the data format of the teacher's face feature library in the embodiment of the present application; In some other specific embodiments, step S201 specifically includes: S2011, receiving original photos and teacher identity information; It should be noted that a teacher can submit multiple original photos to increase the sample size and diversity of feature collection. The system will process each original photo submitted by the teacher separately, including subsequent steps such as resizing and face shape adjustment. Each original photo will generate a corresponding feature vector, which together constitute the feature set of the teacher. The processing process of each original photo is independent of each other, but the final feature vector will be associated with the same teacher identity.

[0089] S2012, adjusting the size of the original photo to a preset size to obtain a first adjusted photo; The preset size indicates the standard image size specified by the system, which is used to ensure the uniformity of all processed images.

[0090] It should be noted that, because the original photos from different devices often have different resolutions and aspect ratios, these original photos of inconsistent sizes will cause deviations in subsequent feature vectors, so the sizes of all processed images need to be unified.

[0091] S2013, performing below-threshold adjustment on the face shape in the first adjusted photo to obtain a second adjusted photo; In some specific embodiments, the system first determines the position and shape of the facial contour through facial key point detection, and then analyzes the difference between the current face shape and the standard face shape template. According to the preset adjustment threshold, the system will perform appropriate deformation processing on the face shape, which is not limited here.

[0092] In some other specific embodiments, the positions of several key points of the face are detected; a deformation field is generated according to a threshold constraint; and the deformation field is applied to perform image deformation to smooth the deformed edges, which is not limited here.

[0093] S2014, extracting feature points from the original photo, the first adjusted photo, and the second adjusted photo using a convolutional neural network and generating a feature vector; S2015. Store the feature vector, teacher identity information, collection time, collection method, and number of hits in the teacher face feature library.

[0094] Among them, the collection method represents the specific way and method of obtaining the face image, including the collection method of step S201 (manual) and the collection method of steps S101 to S106 (classroom), and the number of hits is used to record the cumulative number of times the feature is successfully matched during the recognition process.

[0095] It should be noted that, in some embodiments, the number of hits of a feature vector reflects the frequency and reliability of use of the feature in the recognition process. In some embodiments, the higher the cumulative number of hits, the higher the recognition accuracy of the feature vector, and the system will increase its priority accordingly.

[0096] It can be seen that by establishing a multi-dimensional feature vector containing original photos and adjusted photos for each teacher, the expression range of facial features has been expanded. Among them, the original photos are adjusted below the threshold to make the feature vectors in the feature library richer, and the photo features with different degrees of beauty effects are reversely simulated. By clustering the facial features extracted from these pictures, the facial features of the same teacher at different times are automatically classified into the same set, so that more facial features can be used for comparison during identity recognition, which improves the reliability of recognition. By comparing the clustered facial features with the feature library for similarity and setting a reasonable preset threshold, while providing sufficient recognition fault tolerance space, the reliability of the recognition results is ensured, thereby solving the problem of recognition failure caused by beauty, filters and other effects, and improving the accuracy of face recognition.

[0097] In actual use, the photos that the facial feature library relies on are often distorted, such as photos taken with mobile phones, "artistic photos", and photos that have been retouched. These photos are generally beautified by beautification processing, filter effects, etc., resulting in a large difference between the stored feature templates and the facial features of teachers in actual classroom teaching environments. This fundamental problem stemming from the construction of the feature library is a technical difficulty that the above technical solutions have not yet effectively solved.

[0098] After step S102, the method further includes: S107, in the case of corresponding to the teacher identity of the lecturer, selecting a confirmed teacher face set, the confirmed teacher face set is a face feature set and a feature vector in a teacher face feature library are compared for similarity, and a face data set corresponding to the face feature set when the similarity is greater than a preset threshold; the lecturer is the teacher of the current class; It should be noted that the teacher face set has been confirmed to be a face set corresponding to facial features when the similarity is greater than a preset threshold.

[0099] The application scenario is mainly when it has been preliminarily determined that the current teacher's identity is in line with the teaching arrangement and to further screen out high-quality, high-matching facial data from the identified face set for subsequent operations, such as enriching the teacher's facial feature library or performing more accurate identity confirmation.

[0100] In some embodiments, the system first selects high-quality facial features equal to the number of existing features of the teacher in the teacher feature library from the identified face set in the order of quality scores from high to low. This step can ensure that the selected features have a good quality basis and match the existing features in quantity, which is convenient for subsequent fair comparison. Then, the system compares these selected features with all the features of the teacher in the feature library one by one, and calculates the similarity value. Through this detailed comparison, the similarity between each feature and the features in the library can be accurately measured. Finally, the arithmetic mean of all similarity values ​​is calculated as the overall similarity index of this comparison. Based on this index, it is determined whether to include the corresponding face data in the confirmed teacher face set. If the overall similarity index is greater than the preset threshold, the corresponding face data will be included in the set, providing accurate data support for subsequent steps.

[0101] S108, selecting several facial features with the highest quality assessment results from the confirmed teacher face set; and performing similarity comparison with facial features of corresponding teachers in the teacher face feature database; In some embodiments, the system first selects high-quality facial features from the identified face set, in descending order of quality scores, that are equal to the number of features of the teacher in the teacher feature library. Then, the system compares these selected features with all the features of the teacher in the feature library one by one, calculates the similarity value, and finally obtains the arithmetic mean of all similarity values ​​as the overall similarity index of this comparison.

[0102] S109, calculating the average similarity after comparison; S110. When the mean similarity is less than the learning threshold, the facial features with the highest quality assessment result are selected from the confirmed teacher face set and added to the teacher information of the teacher in the teacher face feature library.

[0103] It can be seen that the feature library has achieved self-learning and self-improvement. The system compares the similarity mean of the successfully recognized facial features with the existing feature library. When the mean is low, it means that the currently successfully recognized facial features contain new feature information. By adding these new high-quality facial features to the feature library, the system continuously enriches and updates the feature templates while maintaining recognition accuracy, so that the feature library can dynamically adapt to subtle changes in the teacher's facial features and continuously improve the robustness of recognition.

[0104] In actual use, the above embodiment realizes self-learning and self-improvement of the feature library. However, as time goes by, the data in the teacher's face feature library becomes more and more, resulting in lower efficiency of face recognition and increased computing overhead.

[0105] Therefore, in some embodiments, the system needs to regularly prune the feature vector library. The pruning strategy is mainly based on two core goals: one is to maintain the quality of the feature library to avoid quality degradation and error propagation caused by dynamic expansion (i.e., an erroneous feature vector may cause more erroneous features to be added by mistake); the other is to control the size of the feature library, reduce the amount of calculation during feature matching, and improve system performance.

[0106] Specifically, when the number of feature vectors of a teacher exceeds the preset upper limit (such as 4), the system will screen and clean up the feature vectors according to the following conditions: First, clean up the features that are too similar to other vectors in the library, because such features have low discrimination and are not meaningful to retain; second, clean up those feature vectors that are less similar to other features and have a longer storage time and a lower number of hits, because such features may be outdated or unrepresentative. This pruning mechanism based on similarity, timeliness, and frequency of use can effectively maintain the quality and scale of the feature library.

[0107] In some embodiments, after step S110, the method further includes: S111, determining whether the number of facial features in the teacher information exceeds a preset upper limit threshold; S112. If the preset upper limit threshold is exceeded, facial features in the teacher information whose similarity with other facial features is greater than the discrimination threshold are cleaned up; or cleaning is performed based on the storage time and the number of hits, where the storage time is positively correlated with the cleaning probability, and the number of hits is negatively correlated with the cleaning probability.

[0108] It can be seen that by setting the similarity mean and learning threshold for screening, and cleaning up facial features whose similarity with other facial features is greater than the discrimination threshold, the quality of the library can be maintained, and features that are easy to confuse and have low discrimination can be eliminated to avoid error propagation, thereby ensuring that subsequent face recognition and identity authentication are accurate and reliable. On the other hand, by judging and cleaning up facial features that exceed the quantity based on the preset upper limit threshold, and comprehensively considering factors such as storage time and number of hits, the size of the library can be controlled, the burden of feature matching calculations can be reduced, and the system operation efficiency can be improved.

[0109] The following introduces an exemplary teacher face automatic acquisition system 600 provided in an embodiment of the present application. Figure 6 It is a schematic diagram of an exemplary hardware structure of the teacher face automatic acquisition system 600 provided in an embodiment of the present application.

[0110] In some embodiments, the teacher face automatic collection system 600 is a computer device or the teacher face automatic collection system 600 includes a computer device. The computer device includes a processor, a memory and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with other external terminals or servers through a network connection. In some embodiments, the network interface can be a wired network interface, and in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, the method in the embodiment of the present application is implemented.

[0111] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0113] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0114] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0115] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for automatically collecting teacher faces, characterized in that: include: Obtain a set of faces whose identities are to be determined from a classroom video stream; Compare the facial features in the face set with the feature vectors in the teacher's facial feature library for similarity; In the case that the teacher identity of the lecturer cannot be identified through the teacher face feature database; the teacher information of the teacher of the current class is extracted from the class schedule; the lecturer is the teacher of the current class; Determine whether the teacher face feature database has the teacher information of the teacher; If the teacher information of the teacher is not available, a candidate teacher face is selected; the candidate teacher face is a face set with the most face images; Selecting facial features of the candidate teacher faces whose quality assessment results are greater than a threshold; and creating the teacher information of the lecturer in the teacher facial feature library.

2. The method according to claim 1, characterized in that The step of creating the teacher information of the lecturer in the teacher face feature library specifically includes: Determine whether the teacher information of the lecturer exists in the teacher face feature database to be confirmed; If the teacher information of the lecturer does not exist, creating the teacher information of the lecturer in the teacher face feature database to be confirmed according to the facial features of the lecturer; If the teacher information of the presenter exists, the facial features of the presenter are compared with the facial features in the teacher information of the presenter in the face feature library of teachers to be confirmed for similarity; If the similarity is less than the repetition threshold, the facial features of the lecturer are added to the teacher information of the lecturer in the to-be-confirmed teacher facial feature database; After the teacher information in the teacher facial feature library to be confirmed is confirmed, the confirmed teacher information is added to the teacher facial feature library.

3. The method according to claim 1, characterized in that The step of obtaining a set of faces to be identified from the classroom video stream specifically includes: Intercepting pictures from the classroom video stream at preset time intervals to obtain a picture set; Extracting the facial features in the picture set; The facial features are clustered to obtain a set of faces whose identities are to be determined.

4. The method according to claim 3, characterized in that The step of extracting facial features from the picture set specifically includes: Detecting faces from the picture set to obtain a plurality of face pictures; Performing face quality assessment on the plurality of face images, wherein the face quality assessment includes face angle assessment, lighting condition assessment, clarity assessment, and occlusion assessment; Filter out the face images whose quality assessment results are lower than the preset quality standard to obtain qualified face images; A convolutional neural network is used to extract feature points from the qualified face image and generate face features.

5. The method according to claim 4, characterized in that The step of clustering the facial features to obtain a set of faces whose identities are to be determined specifically includes: Initializing the face set; For each newly acquired facial feature, performing a similarity comparison between the newly acquired facial feature and all the facial features in the face set; In the case where there is a similarity greater than a clustering threshold, determining the face set to which the face feature with the greatest similarity belongs, and adding the newly acquired face feature to the corresponding face set; When there is no similarity greater than the clustering threshold, a new face set is created, and the newly acquired face features are added to the new face set.

6. The method according to claim 5, characterized in that The steps of comparing the facial features in the face set with the feature vectors in the teacher facial feature library for similarity; and confirming the teacher identity of the lecturer when the similarity is greater than a preset threshold, specifically include: Sorting the facial features in the face set from high to low according to quality evaluation results; According to the sorting results, the facial features are compared with the feature vectors in the teacher's facial feature library in turn for similarity; When the similarity is greater than a preset threshold, the identity to be determined corresponding to the facial feature is determined as the corresponding teacher identity in the teacher facial feature library.

7. The method according to claim 1, characterized in that Before the step of acquiring a set of faces to be identified from the classroom video stream, the method further comprises: Receive original photos and teacher identification information; Adjusting the size of the original photo to a preset size to obtain a first adjusted photo; Performing below-threshold adjustment on the face shape in the first adjusted photo to obtain a second adjusted photo; Extracting feature points from the original photo, the first adjusted photo, and the second adjusted photo using a convolutional neural network and generating the feature vector; The feature vector, the teacher identity information, the collection time, the collection method, and the number of hits are stored in the teacher face feature library.

8. A teacher face automatic collection system, characterized in that: The automatic teacher face acquisition system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the automatic teacher face acquisition system to execute the method described in any one of claims 1-7.

9. A computer program product comprising instructions, characterized in that When the computer program product runs on the teacher face automatic acquisition system, the teacher face automatic acquisition system executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the teacher face automatic acquisition system, the teacher face automatic acquisition system executes the method as described in any one of claims 1-7.

Images