A classroom attendance management method and system
By using the method based on neural networks and edge computing in the classroom attendance management system, the face recognition of MTCNN, SSH and FACENET algorithms is used to solve the problem of low face recognition accuracy and poor efficiency in existing technologies, and achieve high precision high accuracy Monitoring and management of class attendance rate.
Patent Information
- Application Number
- CN202210231229.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-09
- Publication Date
- 2025-01-24
- Estimated Expiration
- 2042-03-09
AI Technical Summary
The prior art has problems with low facial recognition accuracy, poor efficiency and transparency in classroom attendance management, especially in the aspects of large number and small size facial recognition.
Adopt attendance management methods and systems based on neural networks and edge computing, use deep convolutional neural network models for face target recognition, and use MTCNN and SSH algorithms for cascade detection, combined with FaceNet models for feature extraction, to achieve high-precision facial recognition. Face recognition and matching.
The accuracy of the number of people in the classroom and small size in the classroom has been improved, and the accurate online attendance rate data collection is realized in real -time online.
Smart Images

Figure CN114581994B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a classroom attendance management method and system. Background Art
[0002] First of all, teaching management standards are the basic requirements for college students, and obtaining classroom information is also of great significance for colleges and universities to improve teaching quality. The above-mentioned traditional teaching management is mainly based on roll call and sign-in, but rolling call in every class is undoubtedly a huge waste of manpower and material resources. Not only is the workload large, but the efficiency and transparency are poor. With the development and widespread application of database technology and computer vision technology, using computers to manage teaching work in colleges and universities has become a general trend.
[0003] At present, the prior art has disclosed technical solutions for classroom attendance management through contactless smart devices, such as by arranging cameras in the classroom and using computer vision technology to realize contactless extraction of teacher and student information, and realizing multi-target detection through image information obtained by the camera, thereby realizing attendance management activities for the entire classroom. For example, the patent with publication number CN111612930A provides an attendance system and method, which arranges a smart camera matrix in the classroom, obtains image information through the camera, transmits it to the edge computing smart box for processing, and uploads the processing results to the school's attendance management platform.
[0004] The current classroom attendance system can already perform simple facial recognition, but feedback on student attendance and facial recognition accuracy still need to be improved. Summary of the invention
[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an attendance management method and system based on neural networks and edge computing. The method uses face recognition technology to perform contactless sign-in for students in the classroom. The teacher only needs to adjust the fixed recognition camera to obtain the class situation. The present invention solves the problem of low recognition accuracy of large numbers of small-sized faces, and can provide real-time online accurate data for university teaching managers to grasp the current course attendance rate.
[0006] The technical solution adopted by the present invention is:
[0007] In a first aspect, the present invention provides a classroom attendance management method for a cloud server, the method comprising:
[0008] Receive a recognition request sent by a client, the recognition request includes a course number, a class number and a classroom number, obtain a corresponding class student information table L from a database according to the course number and the class number in the recognition request; mobilize a corresponding image acquisition device in the classroom according to the classroom number to start image acquisition;
[0009] Receive information M to be identified sent by the edge computing device, where the information M to be identified includes: facial feature information F, where the facial feature information F is a feature vector P3 formed by an original facial image P1 acquired by a corresponding image acquisition device in the classroom through two operations of detection alignment and feature extraction in the edge computing device;
[0010] According to the student information table L, retrieve the pre-collected information M0 of all students in the information table, the pre-collected student information M0 includes: name, class and pre-collected facial features F0; match the pre-collected facial features F0 with the facial feature information F obtained on-site, and display the list of students who have attended and the list of students who have not attended according to the matching results.
[0011] Among them, the MTCNN algorithm and SSH model are used to detect and align the original face image P1 to output the preliminary processed image set P2; the FaceNet model is used to extract features from the image set P2 to form a feature vector P3;
[0012] This method proposes a classroom attendance method, the key is to realize the function of face monitoring and recognition. In the past, the underlying features and intermediate features that required manual parameter setting had a small amount of computation, but their accuracy was far from meeting the recognition requirements when the number of targets was large and the size was small. Therefore, we introduced a neural network algorithm to improve the recognition accuracy. In the present invention, a deep convolutional neural network model was used for face target recognition. Convolutional neural networks (CNN) are a type of feedforward neural network that includes convolution calculations and has a deep structure, and are one of the representative algorithms for deep learning. Convolutional neural networks are constructed by imitating the visual perception mechanism of organisms, and can perform supervised learning and unsupervised learning. The convolution kernel parameter sharing in the hidden layer and the sparsity of the inter-layer connection enable the convolutional neural network to learn grid-like topology features such as pixels and audio, with a stable effect and no additional feature engineering requirements for the data. However, the high accuracy of convolutional neural networks also requires a corresponding price, the most prominent of which is that its computational workload is very large, requiring the use of a large amount of computing resources. In the real-time classroom attendance monitoring scenario faced by the present invention, the conventional simple C / S service architecture is prone to a sharp drop in response speed or even crash due to insufficient resources when facing high concurrency. For this reason, edge computing technology is introduced.
[0013] Edge computing refers to processing operations at one end close to an object or data source, and providing the nearest service nearby. Initiating an application at the edge end can generate network service responses more quickly and efficiently, meeting the basic needs of the industry in terms of real-time business, security and privacy protection, and data detection. Edge computing technology has also had a huge impact in the field of the Internet of Things. The application of edge computing technology means that a large amount of control and processing will be realized through terminal devices without being handed over to the server background, and the processing process will be completed by the edge computing layer at the terminal. This processing technology will undoubtedly improve processing efficiency very quickly and reduce the processing burden of the background. And because the control module is closer to the user, it can provide users with faster responses and try to solve the needs directly at the edge end. Preferably, the edge computing device used in the present invention is a JetsonNano development board, which plays the role of an edge computing node, and the face recognition algorithm runs directly on the development board, thereby extracting facial features in real time. The use of edge computing technology not only disperses the huge computing power required by the deep convolutional neural network algorithm, but also avoids consuming a large amount of network resources by directly transmitting image data in the network, and at the same time, the security of face data as private data is also guaranteed.
[0014] Furthermore, the image acquisition device uses a high-definition camera to capture high-resolution images of the area to be identified in the classroom; preferably, the resolution of the image acquisition device used is at least 3280x2464, which is 3.9 times the mainstream 1080p (1920x1080) and 8.7 times the 720p (1280x720).
[0015] Furthermore, the MTCNN algorithm and SSH model are used to detect and align the original face image P1, as shown in FIG. Figure 2 As shown, the specific steps include:
[0016] The original face image P1 is preprocessed, and its size is continuously changed according to the resize_factor until a 12-pixel image is obtained after the last change. At this time, several changed images, namely image pyramids, are generated, and the image pyramids are input into P-Net one by one;
[0017] A large number of preliminary detection windows are quickly generated through a shallow CNN network P-Net. Here, P-Net is preferably used as a fully convolutional neural network, which detects the face area and regresses the bounding box in the picture obtained by the camera. The obtained face area window will be corrected by the result of BB regression, and then the non-maximum suppression function (NMS) is used to remove the windows with the overlap rate lower than the threshold;
[0018] Use R-Net to filter out most of the non-face candidate windows. The network model at this stage is called R-Net, which is an optimized network. Its purpose is to filter out most of the non-face region candidate windows, further adjust the regression results, and finally use NMS to merge them.
[0019] Reuse the network with stronger capabilities to find the five marked positions on the face candidate window. The network model at this stage is referred to as O-Net. After the second stage data input, after network processing and recognition optimization, the five positions of face calibration are finally obtained;
[0020] Unlike MTCNN, SSH is scale-invariant. When predicting, MTCNN needs to input pictures of different sizes for prediction and then perform NMS processing, while SSH only needs one forward propagation to solve the problem. The implementation method is to make three branches (M1, M2, M3) for different convolutional output layers of VGG. Each branch uses a similar process to predict detection and classification results.
[0021] The SSH model uses high-resolution feature maps (shallow features) to detect small faces; uses intermediate-resolution feature maps (middle-layer features) to detect medium-sized faces; and finally uses low-resolution feature maps (deep features) to detect large faces. The specific method is to perform face detection for the first time on the output of a shallow convolutional layer in the backbone network to detect small faces; perform a second test on the output of the convolutional layer in the middle layer to detect medium faces; and use the network's final pooled output to detect large faces. Because the receptive field of each pixel in the shallow convolutional layer is smaller than that of the deeper convolutional layer. By testing convolutional layers of different depths separately, faces of different scales can be detected, and the final set of detected and segmented face parts is output;
[0022] The processing results of the above two models are concat-joined.
[0023] Furthermore, the FaceNet model is used to extract features from the image P2 to form an image feature vector P3, and the specific steps include: concat splicing the images detected and aligned by the MTCNN algorithm and the SSH model to obtain P2, inputting P2 into FaceNet, and mapping the image P2 to the Euclidean space through normalization of the convolutional neural network to output the image feature vector P3; calculating the Euclidean space example to obtain its similarity detection result. The FaceNet model used abandons the common solution of learning features from key points of the face, and directly learns the mapping relationship between the image and the points in the Euclidean space. The distance between the mapping points of the features of the two images in the Euclidean space directly corresponds to the similarity of the two images, thereby greatly improving the robustness of the method of the present invention.
[0024] In order to save the storage space of the development board and reduce the working energy consumption, the server will transmit the pre-recorded portraits of the students in the class to the development board. Since it is meaningless to carry out face recognition detection all the time, it will waste the computing power and energy consumption of the development board, and will also occupy the storage space of the image, so it is set to start image detection upon request.
[0025] As a preferred solution of the classroom attendance management method of the present invention, the corresponding image acquisition device in the classroom is mobilized according to the classroom number to start image acquisition. Preferably, the number of image acquisitions is twice. After the cloud server receives the request, the image acquisition device is immediately mobilized to start graphic acquisition and the matching result is output as the first attendance result according to the above steps. It is set to automatically start the second image acquisition after an interval of 1-15 minutes, and the matching demerit is output as the second attendance result. The first attendance result is compared with the second attendance result. The list of students who are extra in the second attendance result is latecomers, and those who are still not matched in the second attendance result are absentees;
[0026] As another preferred embodiment of the classroom attendance management method of the present invention, in order to increase the evaluation dimension of the attendance method and facilitate different punishment management for students who are late at different times, the image acquisition here can be increased to at least three times. By comparing the attendance results between different times, not only the list of absentees can be obtained, but also the list of people who are late at different time periods can be obtained, thereby increasing the evaluation dimension of attendance management.
[0027] In a second aspect, the present invention provides a classroom attendance management system, such as Figure 1 As shown, the system includes: an image acquisition device, an edge computing device, a cloud server, and a client device;
[0028] The image acquisition device is used to receive the recognition request sent by the client, start and obtain the original face image P1 of all people in the current classroom, and transmit it to the edge computing device for processing;
[0029] The edge computing device is used to process the original face image P1 sent by the image acquisition device, and perform detection, alignment and feature extraction on the image P1;
[0030] The cloud server device is used to receive an identification request sent by a client, where the identification request includes a course number, a class number and a classroom number; obtain a corresponding class student information table L from a database according to the course number and the class number in the identification request; mobilize a corresponding image acquisition device in the classroom according to the classroom number to start image acquisition; receive information to be identified M sent by an edge computing device, where the information to be identified M includes facial feature information F; retrieve pre-collected information M0 of all students in the information table according to the student information table L, where the pre-collected student information M0 includes: name, class and pre-collected facial features F0; match the pre-collected facial features F0 with the facial feature information F obtained on site, and display a list of students who have attended and a list of students who have not attended according to the matching results.
[0031] The client device is used to send an identification request and input identification request information.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] The attendance management method and system of the present invention respectively establishes cascade detection based on convolutional neural networks using MTCNN and SSH, and inputs the feature detection results obtained from the output of the two models into FaceNet after concat layer processing, and selects the value with higher confidence as the final recognition result. Real-time face detection and point calibration are achieved by only using a lightweight CNN model, which has a simple and highly efficient structure, realizes online training to improve performance of difficult sample mining, and can complete multiple target recognition tasks at one time.
[0034] The method and system of the present invention adopts a method based on deep convolutional neural network, using edge computing and cloud computing to efficiently complete face recognition and matching, and builds a real-time classroom situation monitoring system with a user-friendly interface, which can be deployed without additional network and server performance requirements. A detailed description is given below.
[0035] Since the distance between the front and back of the classroom is long and there are many people, and due to the position and angle of the camera, the sizes of faces in the captured images are not consistent, and accurate recognition is not an easy task. Based on cost, complexity, and resource utilization considerations, the method and system described in the present invention choose to increase the resolution of a single image, and use MTCNN with less computational effort to first detect and segment the high-resolution image, and the input resolution obtained by the feature extraction algorithm will meet the recognition requirements. At the same time, in order to improve the ability to detect small targets, the method of downsampling the input image to form a pyramid and then performing network forward operation detection on each layer of the image is not adopted. The reason is that this method has a slow processing speed. By introducing the SSH module, targets of different sizes are detected at multiple scales, which improves the detection accuracy and solves the problem of recognizing a large number of small-sized faces in classroom attendance.
[0036] Due to the improvement of accuracy, the storage space and working energy consumption increase. The existing face recognition detection system transmits large files, slow speed, high requirements for network conditions, and the risk of personal information leakage. After the improvement, the development board only needs to transmit a small number of information, and the server does not directly use the original image, but extracts the features and stores them in the database. The development board is no longer responsible for matching, but only needs to return the extracted features, which greatly reduces the amount of data, improves security, and solves the problem of data transmission and storage.
[0037] The attendance system in the existing technology generally collects images through traditional cameras and then sends them to the background processor to complete facial feature extraction and matching. First of all, traditional cameras require separate data cables to be installed, which is costly and complex. At the same time, direct image transmission will cause bandwidth resource constraints, and all computing tasks will be handed over to the server, which will cause performance constraints. Through edge computing technology, the computing load is effectively dispersed, the amount of data transmission is reduced, and it only needs the existing campus network system to access.
[0038] The attendance management method and system provided by the present invention have broad application prospects and can certainly effectively regulate the classroom discipline of colleges and universities, while helping managers to grasp information to better improve teaching plans. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 The overall architecture diagram of the attendance management system described in the present invention;
[0040] Figure 2 It is the algorithm framework for processing the original image to be tested according to the present invention;
[0041] Figure 3 This is the face recognition classification process of the present invention;
[0042] Figure 4 yes Figure 3 The process of obtaining the facial feature vector step;
[0043] Figure 5 It is a mobile terminal APP interface diagram output by an embodiment of the present invention;
[0044] Figure 6 It is the configuration table of the JetsonNano development board described in the embodiment of the present invention. DETAILED DESCRIPTION
[0045] The present invention is further described below in conjunction with the accompanying drawings.
[0046] The system consists of two parts: hardware and software. The hardware part is an integrated image acquisition and recognition device deployed in the classroom, and the software part includes face recognition algorithms, backend logic and database, and a mobile phone APP directly facing users.
[0047] 1. Hardware
[0048] The image acquisition and recognition integrated device is divided into two parts. This device is an integrated device for image acquisition and edge processing. The image acquisition part is a high-definition camera, and the edge processing part is an Nvidia JetsonNano development board. The camera uses a Sony IMX219 CMOS sensor with up to 8 million pixels and a maximum image resolution of 5632x4224.
[0049] In order to realize edge computing, that is, extract features immediately after acquiring image information and only send the extracted feature vectors to the server, the JetsonNano development board is preferred because it has sufficient computing power to run lightweight neural networks and has the advantages of low price, rich interfaces, strong scalability, and extremely low power consumption (about 5W).
[0050] 2. Software
[0051] 1. Face recognition algorithm
[0052] The face recognition algorithm is divided into two steps, such as Figure 2 As shown in the figure, detection and alignment are performed first, and then features are extracted, using MTCNN+SSH and FaceNet algorithms respectively.
[0053] Among them, MTCNN is a model that implements face detection and five-point calibration in one network, mainly through the cascade of CNN models to realize a multi-task learning network. The entire model is divided into three stages. The first stage uses a shallow CNN network P-Net to quickly generate a large number of preliminary detection windows; the second stage uses R-Net to filter out most of the non-face candidate windows; the third stage reuses a more powerful network to find the five marker positions on the face candidate window.
[0054] Phase 1: The selected fully convolutional neural network is a recommended network, referred to as P-Net, which detects the face area and bounding box regression in the picture obtained by the camera. The obtained face area window will be corrected by the result of BB regression, and then the non-maximum suppression function (NMS) will be used to remove the windows with overlap rate lower than the threshold.
[0055] The second stage: The network model of this stage is called R-Net, which is an optimized network that aims to filter most non-face area candidate windows, further adjust the regression results, and finally merge them using NMS.
[0056] The third stage: The network model of this stage is referred to as O-Net. The data input from the second stage is processed and recognized by the network, and finally the five positions of face calibration are obtained.
[0057] SSH: Since the traditional pyramid structure is too slow to process, SSH abandons the traditional FPN structure and adopts the method of multiple detection in each level of convolutional layer. The specific method is to perform face detection for the first time on the output of a shallow convolutional layer in the backbone network to detect small faces; perform a second detection on the output of the convolutional layer in the middle layer to detect medium faces; and use the output after the final pooling of the network to detect large faces. Because the receptive field of each pixel in the shallow convolutional layer is smaller than that of the deeper convolutional layer. By detecting convolutional layers of different depths separately, faces of different scales can be detected.
[0058] FaceNet: The results of the above two models are concat-processed as the input of this network. The input samples are passed through the existing general convolutional neural network framework, and the results are normalized and mapped to the hyperplane. The spatial distance is calculated to obtain the similarity, and finally the recognition result is obtained. In this system, we used the ResNet50 framework, which can effectively avoid overfitting, improve the accuracy of face recognition, and ensure that the performance overhead is not too large. Abandoning the common solution of learning features from facial key points, directly learn the mapping relationship between images and points in Euclidean space. The distance between the mapping points of the features of the two images in the Euclidean space directly corresponds to the similarity of the two images, thereby greatly improving the robustness of the method of the present invention.
[0059] The overall process of face recognition is as follows: Figure 3 As shown in Figure 1, the real-time image captured by the high-definition camera is processed by the above model to obtain a feature vector, which is then compared and matched with the labeled features extracted from the local image to obtain the final similarity classification result. The feature extraction process of the local image and the real-time image is as follows: Figure 4 As shown in the figure, the picture is imported into the model, resized and then input into the model. The obtained feature vector is optimized by the deep residual network to output a 128-dimensional face feature vector.
[0060] 2. Backend logic and database
[0061] The backend of the entire APP is written in Python, and uses tornado, a Python-based Web service framework and asynchronous network library, to build the entire backend and front-end data transmission. We store the transmitted data in the MongoDB database in JSON format.
[0062] The implementation of the specific framework is mainly divided into the following parts: Tornado's basic Web framework tornado.web is responsible for global configuration, including the routing table that maps requests to handlers, RequestHandler and its subclasses implement Web applications, HttpServer implements HTTP clients and servers, and the asynchronous network library IOLoop provides building blocks for HTTP implementation. At runtime, HttpServer opens the HTTP service and starts IOLoop to listen to the port in a loop, then Application parses the HTTP request sent by the front end and maps it to an instantiation of the RequestHandler class object. RequestHandler encapsulates all the information and processing methods for request processing to process the corresponding request method.
[0063] In order to realize basic student sign-in and add, delete, modify and query functions of student information, five subclasses are defined. The POST method is defined under UserFeatureHandler, which is used to receive the facial features posted by the development board to the server, and then judge the student's attendance status through Euclidean distance comparison and store it in the database. UserInfoHandler and UserDetailHandler define GET requests, and return student information after receiving the request from the mobile phone APP. UserupdateHandler and UserSearchHandler both use regular expressions. Tornado stores the string that matches the expression in the brackets and passes the string as a parameter to the RequestHandler class. The POST method is defined under UserupdateHandler, and the mobile phone terminal passes in the corrected student information, which is then modified in the database. UserSearchHandler receives the search string passed in by the mobile phone terminal, performs string matching on the student information in the database, and returns the required student data.
[0064] The tornado framework used in the backend is a lightweight web framework, similar to another Python web framework Web.py, which includes asynchronous non-blocking IO. Tornado has excellent load resistance. The official conducted an experimental comparison of its capabilities, using nginx reverse proxy to deploy Tornado and other Python web application frameworks. The experimental results show that Tornado's maximum page views exceed the second place by nearly 40%. In the formal sign-in situation, our mobile APP will be visited in large quantities, which requires our backend to run normally without crashing under a large amount of load. At the same time, Tornado has excellent processing performance, does not rely on multi-process / multi-threading, and performs well in processing a large amount of facial feature data at the same time.
[0065] 3. User APP
[0066] User applications are implemented using the Ionic framework, which uses Web development technology and is based on HTML5. It is a development framework similar to native applications on mobile platforms. The framework is also bound to AngularJS and Sass. The Ionic framework was originally invented to develop mobile applications from a Web perspective. It can be compiled into applications for various platforms such as Android and iOS based on the PhoneGap compilation platform. When it is necessary to interact with the underlying mobile terminal, Cordova technology is used to achieve cross-platform compatibility. In interface design, Ionic uses many CSS components and JS UI libraries. A large number of interface optimizations facilitate adaptation to mobile terminals with different resolutions, achieving one-time development and multi-platform use.
[0067] The management software is displayed on the mobile phone in the form of a web page. The data is sent to the server via HTTP and loaded into the mobile phone interface. Each user will be assigned a different account for login and management. The app is divided into three homepages: the first page is the message notification interface, which is convenient for users to quickly view system notifications; the second page is the app settings interface, which is convenient for different users to customize. The third page is the real-time query information page, such as Figure 5 As shown, this page displays the course records of the current user, including the number of absentees in each class and the class time of the course. If you need to view detailed absentee information, click the "More" button on each class to view it. The "More" page displays the information of students who did not attend this class. The "More" interface provides a query function to facilitate users to query a specific student's information. At the same time, when you click on each student record, the detailed personal information of the student will be displayed. If you need to manually change the student status, the user can also manually change the student's attendance status through the "Delete" or "Add" button in front of each record. The background of this modification will automatically synchronize to the database to ensure data consistency.
Claims
1. A classroom attendance management method, used in a cloud server, characterized in that: The method comprises: Step S1: receiving a recognition request sent by a client, the recognition request including a course number, a class number and a classroom number, obtaining a corresponding class student information table L from a database according to the course number and the class number in the recognition request; mobilizing a corresponding image acquisition device in the classroom according to the classroom number to start image acquisition; Step S2: receiving the information to be identified M sent by the edge computing device, wherein the information to be identified M includes: facial feature information F, wherein the facial feature information F is a feature vector P3 formed by the original facial image P1 acquired by the corresponding image acquisition device in the classroom through two operation processes of detection alignment and feature extraction in the edge computing device; Step S3: according to the student information table L, retrieve the pre-collected information M0 of all students in the information table, the pre-collected student information M0 includes: name, class and pre-collected facial features F0; match the pre-collected facial features F0 with the facial feature information F obtained on site, and display the list of students who have attended and the list of students who have not attended according to the matching results; The edge computing device used in the present invention is the JetsonNano development board; the MTCNN model and the SSH model are used to detect and align the original face image P1 to output the preliminary processed image set P2; the FaceNet model is used to extract features from the image set P2 to form a feature vector P3; The MTCNN model and the SSH model are used to detect and align the original face image P1 to output a preliminary processed image set P2, and the specific steps include the MTCNN model processing process and the SSH model processing process; The MTCNN model processing process includes: image preprocessing of the original face image P1, reducing the size of the original image until a 12-pixel image is obtained, that is, generating an image pyramid, and inputting the image pyramid into P-Net one by one; quickly generating a large number of preliminary detection windows through P-Net and correcting the results, and then using the non-maximum suppression function (NMS) to remove windows with an overlap rate lower than the threshold; using R-Net to filter out most of the non-face candidate windows, and finally using NMS to merge; reusing a more powerful network to find the five marked positions on the face candidate window. The network model at this stage is referred to as O-Net. The data input of the second stage is processed by the network and recognized and optimized, and finally the 5 positions of face calibration are obtained; The SSH model processing process includes: making three branches for different convolutional output layers of VGG, including: branch M1, using high-resolution feature maps to detect small faces; branch M2, using medium-resolution feature maps to detect medium-sized faces; branch M3 using low-resolution feature maps to detect large faces. Each branch uses a similar process to predict detection and classification results; The processing results of the MTCNN model and the SSH model are concatenated to output image P2; The FaceNet model is used to extract features from the image P2 to form an image feature vector P3. The specific steps include: concat splicing the images detected and aligned by the MTCNN algorithm and the SSH model to obtain P2, inputting P2 into FaceNet, normalizing the image P2 through a convolutional neural network and mapping it to the Euclidean space, and outputting the image feature vector P3; calculating the Euclidean space example to obtain its similarity detection result.
2. A classroom attendance management method according to claim 1, characterized in that: The image acquisition times are two times. After receiving the request, the cloud server immediately mobilizes the image acquisition device to start graphic acquisition and outputs the matching result as the first attendance result according to the above steps. It is set to automatically start the second image acquisition after an interval of 1-15 minutes, and output the matching demerit as the second attendance result. The first attendance result is compared with the second attendance result. The students on the list of the second attendance result are latecomers, and those who are still not matched in the second attendance result are absentees.
3. A classroom attendance management method according to claim 1, characterized in that: The image acquisition can be increased to at least three times here, and by comparing the attendance results of different times, the list of absentees and the list of latecomers in different time periods can be obtained.
4. A classroom attendance management method according to claim 1, characterized in that: The edge computing device is the JetsonNano development board.
5. A classroom attendance management system, the system comprising: Image acquisition equipment, edge computing equipment, cloud server and client equipment as described in any one of claims 1 to 4; The image acquisition device is used to receive the recognition request sent by the client, start and obtain the original face image P1 of all people in the current classroom, and transmit it to the edge computing device for processing; The edge computing device is used to process the original face image P1 sent by the image acquisition device, and perform detection, alignment and feature extraction on the image P1; The cloud server device is used to receive an identification request sent by a client, the identification request includes a course number, a class number and a classroom number, and obtain a corresponding class student information table L from a database according to the course number and the class number in the identification request; mobilize a corresponding image acquisition device in the classroom to start image acquisition according to the classroom number; receive information to be identified M sent by an edge computing device, the information to be identified M includes facial feature information F; According to the student information table L, retrieve the pre-collected information M0 of all students in the information table, the pre-collected student information M0 including: name, class and pre-collected facial features F0; match the pre-collected facial features F0 with the facial feature information F obtained on site, and display the list of students who have attended and the list of students who have not attended according to the matching results; The client device is used to send an identification request and input identification request information.
Citation Information
Patent Citations
Attendance system and method
CN111612930A
Class roll call system based on multi-task cascade convolutional neural network
CN110163567A
Ai - based multimode random interval attendance management system (RIAMS) for virtual classrooms
IN202041034477A