Automatic attendance checking, abnormal behavior detection and classroom condition analysis device based on computer vision

Abstract

The invention discloses an automatic attendance checking, abnormal behavior detection and classroom condition analysis device based on computer vision, and relates to the technical field of intelligent education, the device comprises a front-end user interface, a rear-end server and a database; the system integrates attendance checking, abnormal behavior monitoring and classroom condition analysis functions, the classroom attendance checking module is provided with a dynamic window, a comparison algorithm is optimized, caching and early warning are combined, accurate non-inductive attendance checking and abnormity reminding are achieved, the abnormal behavior detection module fuses multiple algorithms and a dynamic threshold value, behaviors are recognized, instantaneous actions are filtered, continuous analysis is achieved, and the classroom attendance checking system is suitable for being popularized and applied. The method fills the blank of abnormal behavior analysis, realizes visual display of attendance and abnormal behavior detection results by superposing state labels through a front-end real-time monitoring panel and supporting historical data calling, automatically generates a classroom condition analysis report containing analysis data according to the detection results, provides real-time behavior feedback, assists teachers in mastering classroom dynamic conditions in real time, and improves the teaching efficiency. And support is provided for teaching decision making.

Description

Technical Field

[0001] The present invention relates to the field of intelligent education technology, and in particular to a device for automatic attendance, abnormal behavior detection and classroom situation analysis based on computer vision. Background Art

[0002] In the current education sector, computer vision-based facial recognition devices have been initially applied in some schools. Computer vision, a core branch of artificial intelligence, simulates human visual mechanisms and uses cameras, sensors, and other devices to collect image or video data. Combined with deep learning and image processing algorithms, it can detect, identify, track, and analyze targets. It has been widely used in security monitoring, medical diagnosis, smart education, intelligent transportation, and other fields. In educational settings, computer vision technology, through digital perception of classroom dynamics, provides key technical support for the intelligent upgrade of teaching management. Traditional manual roll call is time-consuming and inefficient, and is subject to vulnerabilities such as signed-in by others and missed signs. Automatic attendance uses computer vision technology to identify students in real time, enabling "no-touch sign-in." It can accurately record attendance status (such as attendance, tardiness, and absence), significantly improving attendance efficiency and accuracy, and providing basic data support for teaching management.

[0003] According to the patent application number 202310969637.0, an intelligent classroom attendance system based on face recognition is provided, including: a student face image acquisition module, which is used to establish a face feature database, establish a face feature database based on pictures uploaded by students, and detect and identify faces in the environment; a framework training and matching module, which constructs a deep learning training network framework for extracting student face feature information based on the characteristics of classroom attendance; a student attendance result detection and output module, which is used to save attendance records and perform statistics and analysis on attendance data; a user function module, which is used for different users to register and use, and at the same time completes the reception of attendance information.

[0004] The above scheme is used to apply facial recognition technology to school classroom attendance. The classroom's inherent equipment is used to collect student facial images for automatic facial detection and recognition to achieve classroom sign-in. This contactless, non-perceptual, quick and convenient sign-in method can greatly improve the efficiency of school statistics on student attendance. However, the schemes in the existing technology still have certain defects when used. The existing facial recognition attendance device has a single function and relies solely on facial attendance to reflect the true status of students in the classroom. It lacks systematic integration and practical classroom deployment. At the same time, the continuous analysis and statistics of abnormal classroom behavior, visual display, and automatic generation of classroom situation analysis reports are almost blank, which makes it difficult for teachers to grasp the classroom dynamics in real time and provide timely support for teaching decisions. Therefore, it is of great significance to develop automatic attendance, abnormal behavior detection and classroom situation analysis devices based on computer vision. Summary of the Invention

[0005] The purpose of the present invention is to make up for the shortcomings of the existing technology and provide an automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision. It can obtain classroom video data in real time and detect students' abnormal behavior in the classroom. At the same time, the device can accurately complete automatic attendance, abnormal behavior detection and result visualization, and automatically generate a classroom situation analysis report, which can help teachers grasp the classroom dynamics in real time and adjust teaching strategies in time.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision, the device comprising: a front-end user interface, a back-end server and a database; The front-end user interface is used for teacher interaction and data visualization; The backend server is developed based on Flask application and includes face acquisition module, course creation module, classroom attendance module, abnormal behavior detection module and classroom situation analysis report generation module; The database includes relational database, time series database and object storage; The front-end user interface interacts with the back-end server via HTTP / HTTPS protocol, and each module of the back-end server realizes data transmission via gRPC protocol. The database stores structured data, time series behavior data and unstructured files respectively; The face acquisition module uses a multi-task convolutional neural network (MTCNN) model to detect faces in the video stream, extract key points and feature vectors, and transmit them to the classroom attendance module; The course creation module is used to manage course information and generate a course-camera-student association table; The classroom attendance module generates attendance records based on the association table and facial feature comparison results and stores them in a relational database; The abnormal behavior detection module outputs the behavior judgment result through YOLOv5 target detection and OpenPose posture analysis algorithm, combined with LSTM time series tracking and dynamic threshold mechanism. The output of the abnormal behavior detection module is stored in the time series database; The classroom situation analysis report generation module integrates multi-source data, generates an analysis report through an association rule algorithm, and stores the report in an object storage.

[0007] Furthermore, the front-end user interface integrates a real-time monitoring panel, displays multiple video streams and superimposes student face frames and status labels. The face frames use color coding to distinguish attendance status and abnormal behavior types, and support clicking on the face frames to retrieve students' historical attendance and behavior data. The interface supports course schedule creation and detection strategy configuration, including setting attendance validity time, abnormal behavior detection type and warning trigger conditions. The configuration information is transmitted to the back-end module through the gRPC protocol.

[0008] Furthermore, the course creation module supports associating classroom camera equipment and student lists. The generated association table contains course ID, camera IP address, student ID and teaching time period fields. The fields are mapped to the course table, equipment table and student table in the relational database through foreign keys. The gateway receives the front-end course configuration request and generates course-camera-student association data.

[0009] Furthermore, the classroom attendance module sets an attendance validity time window, caches real-time attendance status through Redis, including student ID, attendance status and timestamp information, and pushes abnormal attendance warnings to the front end through the WebSocket protocol. The warnings include red highlight marks and sound reminders. The classroom attendance module uses the cosine distance similarity formula combined with dynamic weights to optimize face comparison. The formula is: ,in, is the similarity score, and is the eigenvector, is the time decay factor, where is the attenuation coefficient, which is determined by the face database update frequency statistics. It is a dynamic adjustment coefficient obtained by optimizing the historical attendance error rate.

[0010] Furthermore, the dynamic threshold mechanism of the abnormal behavior detection module is adjusted across time periods through a behavior confidence fusion formula, which is: ,in, The dynamic threshold is used to determine the detection threshold of abnormal behavior after dynamic adjustment, reflecting the strictness of the system's judgment on "a certain action is abnormal". It is the basic threshold, which is determined by balancing detection accuracy and coverage during target detection model training. For time domain adjustment threshold, according to the different time periods such as "teaching" or "break" in the classroom, the corresponding threshold is pre-set, so that the system can adapt to the detection needs in different scenarios. is the weight coefficient, which is used to balance the influence of the basic threshold and the time threshold. Its value is obtained by analyzing the behavioral data of the history class and combining it with the statistical optimization of the actual teaching scene. Calculate, where For the The activity index of the class, is the class duration, which is obtained by statistics and calculation of the history class analysis report data. The abnormal behavior detection module filters instantaneous actions through the LSTM time series tracking module and adopts the behavior continuity judgment formula: Identify real abnormal behavior, where The validity score is a score that reaches the set standard (e.g. close to 1), which is considered as a valid abnormal behavior. is the behavioral feature vector, which contains information reflecting the action state such as target detection confidence and human posture angle, and has been normalized. The time window is the number of frames of video that are continuously analyzed to determine the continuity of the behavior, which is related to the camera acquisition frame rate. is the inter-frame attenuation coefficient, which makes the recent video frames have a greater impact on the results and the impact of the distant frames gradually decreases. Its value is determined by cross-validation of multiple sets of video data. The Sigmoid activation function maps the score result to the range of 0-1, which makes it easier to judge whether the behavior is effective.

[0011] Furthermore, the relational database uses MySQL to store student files, course information and attendance records. The student file table contains student ID, name, class and facial feature vector fields. The facial feature vector is stored as text data in a serialized manner. The time series database uses InfluxDB to store behavior logs, which contain student ID, timestamp, behavior type and duration fields. The object storage uses MinIO to store video clips and analysis reports, and supports distributed encrypted transmission.

[0012] Furthermore, the classroom situation analysis report generation module integrates the Apriori association rule algorithm to mine the correlation between attendance data and abnormal behavior. The module calculates the classroom activity index through a multi-dimensional weighted sum formula: ,in, The activity index reflects the overall student participation, interaction and other "activity levels" of the class. The higher the value, the more positive the classroom atmosphere. is the number of dimensions, including attendance rate, frequency of abnormal behavior, number of interactions, attention span, etc. is the original data of each dimension, is the dimension weight, You can combine the experience of teaching experts and their understanding of the classroom, or analyze the data of many classes to see which dimension is more closely related to the final classroom effect, and slowly adjust it. The report supports custom templates, including statistical charts, trend charts and abnormal video links.

[0013] Furthermore, the back-end server adopts a microservice architecture, and each module is routed through the Flask application as an API gateway. The edge computing node is deployed at the classroom end to pre-process the camera video stream, including resolution compression and face ROI extraction, to reduce the back-end computing pressure. The edge node and the back-end server transmit the pre-processed facial feature data and video slices through the gRPC protocol.

[0014] Furthermore, the abnormal behavior detection module supports multiple types of behavior detection, including sleeping, playing with mobile phones, whispering, leaving seats, eating, raising hands, etc. The output results include behavior type, student coordinates and timestamp. The classroom attendance module supports a dynamic registration mechanism. After receiving the new student face data entered by the front end, it triggers Incremental model training to update the feature vector of the face library.

[0015] Compared with existing technologies, this computer vision-based automatic attendance, abnormal behavior detection, and classroom situation analysis device has the following beneficial effects: The present invention solves the problem of single function of existing devices by integrating attendance, abnormal behavior monitoring and classroom situation analysis functions. The classroom attendance module is equipped with a dynamic window, an optimized comparison algorithm, and combines caching and early warning to achieve accurate and seamless attendance and abnormal reminders. The abnormal behavior detection module integrates multiple algorithms and dynamic thresholds to identify behaviors and filter instantaneous actions, realize continuous analysis, and fill the gap in abnormal behavior analysis. Through the front-end real-time monitoring panel, status labels are superimposed and historical data retrieval is supported to achieve visual display of attendance and abnormal behavior detection results, and automatically generate a classroom situation analysis report containing analysis data based on the detection results, providing real-time behavior feedback, helping teachers to grasp classroom dynamics in real time, and providing support for teaching decisions.

[0016] Other advantages, objects and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be learned from the practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0018] Figure 1 This is a schematic diagram of the overall framework of the automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision; Figure 2 This is a flowchart of the face acquisition module in the automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision; Figure 3 This is a flow chart of the classroom attendance module in the automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision; Figure 4 This is a flow chart of the abnormal behavior detection module in the automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision; Figure 5 This is a flowchart of the classroom situation analysis report generation module in the computer vision-based automatic attendance, abnormal behavior detection and classroom situation analysis device. DETAILED DESCRIPTION

[0019] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0020] See also Figure 1 The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision mainly consists of three parts: front-end user interface, back-end server and database: Front-end user interface: used for teacher interaction and data visualization, with an integrated real-time monitoring panel that can display multiple video streams, overlay student face frames and status labels (using color coding to distinguish attendance status and abnormal behavior types), support clicking on the face frame to retrieve student historical attendance and behavior data, create course schedules, and configure detection strategies (such as attendance validity period, abnormal behavior detection type, warning trigger conditions, etc.). Configuration information is transmitted to the back-end module via the gRPC protocol.

[0021] Back-end server: Developed based on Flask application, it includes multiple functional modules. The face acquisition module uses the MTCNN model to detect faces in the video stream, extracts key points and feature vectors, and transmits them to the classroom attendance module. The course creation module manages course information and generates a course-camera-student association table. The classroom attendance module generates attendance records based on the association table and face feature comparison results, sets the attendance validity time window, caches real-time attendance status through Redis, and uses the cosine distance similarity formula combined with dynamic weights to optimize face comparison. It can also push abnormal attendance warnings through the WebSocket protocol. The abnormal behavior detection module uses YOLOv5 target detection and OpenPose posture analysis algorithm, combined with LSTM time series tracking and dynamic threshold mechanism, to output behavior judgment results and support 6 types of behavior detection. The classroom situation analysis report generation module integrates multi-source data and generates analysis reports through association rule algorithms. The back-end server adopts a microservice architecture, and the edge computing node is deployed at the classroom end to preprocess the video stream and reduce the back-end computing pressure.

[0022] Database: including relational database (MySQL is used to store structured data such as student files, course information and attendance records), time series database (InfluxDB is used to store time series behavior data such as behavior logs) and object storage (MinIO is used to store unstructured files such as video clips and analysis reports). Example

[0023] In a smart classroom scenario at a certain university, multiple network cameras are deployed in the classroom, covering the student seating area and the podium. Teachers need to use these devices to implement automated classroom management, including course attendance management, abnormal student behavior monitoring (such as sleeping, playing with mobile phones, etc.), and classroom activity analysis. Before the system was deployed, student facial data collection, course information entry, and camera device association were completed.

[0024] Teachers create a class schedule through the front-end user interface, set the class time to 8:30-10:00 every Tuesday morning, and associate the classroom camera IP address and student list (see Figure 1 ), in the detection strategy configuration, set the attendance validity window to 15 minutes before the class starts to 30 minutes after the class starts. The abnormal behavior detection types include six categories, such as sleeping, playing with mobile phones, and leaving the seat. The sound reminder function of abnormal attendance warning is also enabled. The configuration information is transmitted to the course creation module of the backend server through the gRPC protocol.

[0025] Face collection and feature extraction. Before the class begins, the edge computing node (deployed in the classroom) pre-processes the camera video stream, reduces data transmission pressure through resolution compression and face ROI (region of interest) extraction. The face collection module uses the MTCNN model to detect faces in the video stream and extract key points and feature vectors ( ) and transmitted to the classroom attendance module via the gRPC protocol (see Figure 2 ).

[0026] The course creation module receives the front-end configuration request and generates a relationship table containing the course ID, camera IP address, student ID and teaching time period. It maps the course table, device table and student table in the MySQL database through foreign keys to ensure that attendance and behavior data can be traced (see Figure 1 ).

[0027] The classroom attendance module uses the association table to extract the real-time facial feature vectors and the feature vector of the student files in the database ( For comparison, the cosine distance similarity formula is combined with dynamic weight optimization comparison: Among them, the time decay factor Dynamically adjust according to the face database update frequency, The value is obtained by optimizing the historical attendance error rate. The comparison result generates attendance records (such as attendance, lateness), which are stored in the MySQL database and cached in real time through Redis (including student ID, attendance status and timestamp). If an unregistered face is detected or an attendance anomaly (such as not signing in within the valid time), an alert is pushed to the front end through the WebSocket protocol, the interface highlights the face frame in red and triggers a sound reminder (see Figure 3 ).

[0028] The abnormal behavior detection module uses the YOLOv5 target detection algorithm to identify student actions (such as holding a mobile phone in hand, lowering the head), and combines it with the OpenPose posture analysis algorithm to obtain human joint angle data and generate behavior feature vectors. , filter instantaneous actions through the LSTM time series tracking module, and use the behavior continuity judgment formula: Among them, the time window τ matches the camera frame rate, the inter-frame attenuation coefficient λ is determined by cross-validation of multiple sets of video data, and the Sigmoid function Map the score to the range of 0-1. When the score exceeds the dynamic threshold When it is determined to be a valid abnormal behavior, the dynamic threshold is calculated by the confidence fusion formula: , weight coefficient According to the history classroom activity index Duration Dynamic adjustments, such as teaching time Set a higher threshold to reduce false positives and a lower threshold to capture abnormal activities during breaks (see Figure 4 ).

[0029] After the course ends, the classroom situation analysis report generation module integrates attendance data, abnormal behavior logs, and video slices. It uses the Apriori association rule algorithm to mine data correlations (such as the correlation between high absenteeism and mobile phone use) and uses a multi-dimensional weighted sum formula to calculate the classroom activity index: , dimensions include attendance rate, frequency of abnormal behavior, etc., weight Based on the teaching experience, the report automatically generates a visual chart including bar charts and trend lines, and embeds a link to the abnormal behavior video, which is stored in MinIO object storage for teachers to access (see Figure 5 ).

[0030] The database's hierarchical storage mechanism is as follows: relational data (student files, attendance records) is stored in MySQL, facial feature vectors are stored as text fields using serialization technology, time series behavior data (such as abnormal behavior timestamps and duration) are stored in InfluxDB, supporting fast queries by time range, video clips and analysis reports are stored in MinIO, and data security is ensured through distributed encrypted transmission.

[0031] In summary, through the deployment of this embodiment, teachers can view students' attendance status (such as green face frame indicates attendance, red indicates absence) and abnormal behavior (such as yellow frame indicates playing with mobile phone) in real time on the front-end interface. Clicking on the face frame can trace the student's attendance and behavior records for the past month. The classroom analysis report automatically generated by the system shows that teachers can adjust the teaching rhythm in a targeted manner, such as adding interactive links to improve students' attention. Compared with traditional manual attendance, attendance efficiency is improved and the missed detection rate of abnormal behavior is reduced, providing efficient data support for smart teaching management. Example

[0032] In a smart classroom scenario at a certain middle school, regular teaching management is carried out for a class in the second grade of junior high school. A single wide-angle camera is deployed in the classroom to cover the seating area of ​​all 40 students in the class. Teachers need to use this device to achieve efficient attendance and classroom concentration monitoring (such as identifying behaviors such as whispering and eating), and generate classroom participation analysis reports to assist in optimizing teaching strategies. Before the system deployment, the batch import of student facial data, the setting of course time periods and the calibration of camera parameters have been completed.

[0033] Teachers create courses through the front-end user interface, set the teaching time to every Wednesday afternoon 14:00-14:45, associate the classroom camera IP address, and import the student list and class information (see Figure 1), in the detection strategy configuration, set the attendance validity time window from 5 minutes before the class starts to 10 minutes after the class starts, adjust the abnormal behavior detection types to 4 categories: whispering, eating, leaving the seat, and raising hands, and turn off the sound warning to avoid disrupting the class. The configuration information is transmitted to the course creation module of the backend server through the gRPC protocol.

[0034] Face collection and preprocessing, 5 minutes before class, the edge computing node compresses the camera video stream resolution (such as from 1080P to 720P) and extracts the face ROI to reduce the amount of data transmission. The face collection module calls the MTCNN model to detect faces in the video stream, extracting 68 key points and 128-dimensional feature vectors ( ), transmitted to the classroom attendance module in real time via the gRPC protocol (see Figure 2 ).

[0035] The course creation module receives the front-end request and generates a relationship table containing the course ID (such as MATH-2025-03), camera IP, student ID and teaching time period. It maps the course table and device table in the MySQL database through foreign keys to ensure that the attendance data corresponds to the course information (see Figure 1 ).

[0036] The classroom attendance module uses the association table to extract the feature vectors ( Characteristic vector of student files in MySQL ( ) to calculate the cosine distance similarity: Among them, the time decay factor Dynamic adjustment based on the frequency of weekly updates of the school's face database, The value is obtained by optimizing the historical attendance error rate of the school (such as the false positive rate of proxy signing). If a student is not recognized within the valid time (such as being late), the attendance status is marked as "abnormal". The status data is cached in Redis and a silent warning is pushed to the front end via the WebSocket protocol. Only the face frame on the interface is highlighted in red, and no sound is heard (see Figure 3 ).

[0037] The abnormal behavior detection module uses the YOLOv5 algorithm to detect students' head posture (e.g., lowering the head at an angle greater than 45° indicates possible sleeping) and hand movements (e.g., holding an object close to the mouth indicates eating). It also uses the joint coordinates extracted by OpenPose (e.g., the relative positions of shoulders and elbows) to determine whispering behavior. The behavior feature vector Contains normalized data such as target detection confidence and joint angle deviation, and filters instantaneous actions (such as briefly raising a hand to turn a page) through the LSTM timing tracking module: Time window Set to analyze 20 frames continuously (matching the camera's 25fps frame rate), the inter-frame attenuation coefficient The dynamic threshold is determined by cross-validation of the school's classroom videos. Automatically adjust according to the teaching period: When the class enters the exercise explanation period (activity index higher), The value automatically decreases, making (low threshold) ratio is increased to capture frequent group discussion behaviors and avoid misjudging them as whispering (see Figure 4 ).

[0038] After the course ends, the classroom situation analysis report generation module obtains multi-source data from MySQL (attendance data), InfluxDB (behavior logs), and MinIO (video slices). It uses the Apriori algorithm to mine association rules (for example, the support rate of "whispering" and "leaving seat" is greater than 60%). The activity index is calculated as follows: , dimensions include attendance rate , frequency of abnormal behavior , number of hands raised The raw data is normalized and weighted, and the report automatically generates a line graph (showing the fluctuation of activity throughout the class), a heat map (marking the areas with high incidence of abnormal behavior), and embeds a 10-second video clip of typical abnormal behavior (stored in MinlO) for teachers to analyze after class (see Figure 5 ).

[0039] The database's hierarchical storage mechanism is as follows: student files (including serialized facial feature vectors) and attendance records are stored in MySQL, supporting quick queries by class and course. Behavior logs (such as student ID, timestamp, and 30-second duration of "whispering") are stored in InfluxDB, facilitating time-series analysis of behavior patterns. Analysis reports and video clips are distributedly stored in MinIO, supporting encrypted access by teachers through the front-end interface.

[0040] In summary, through the application of this embodiment, teachers can intuitively view the status of students through the front-end real-time monitoring panel during the teaching process. The green face frame indicates normal attendance, the orange frame marks "whispering", and the blue frame marks "raising hands". Combined with the analysis report, the teacher adjusts the seating layout and increases the frequency of classroom questions. In subsequent courses, the "whispering" behavior decreases and the number of raising hands increases. The device realizes the automation of the entire process from attendance to behavior analysis, improves the efficiency of teachers' classroom management, and provides data-driven decision support for precise teaching.

[0041] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A computer vision-based automatic attendance, abnormal behavior detection and classroom situation analysis device, characterized in that: The device includes: a front-end user interface, a back-end server and a database; The front-end user interface is used for teacher interaction and data visualization; The backend server is developed based on Flask application and includes face acquisition module, course creation module, classroom attendance module, abnormal behavior detection module and classroom situation analysis report generation module; The database includes relational database, time series database and object storage; The front-end user interface interacts with the back-end server via HTTP / HTTPS protocol, and each module of the back-end server realizes data transmission via gRPC protocol. The database stores structured data, time series behavior data and unstructured files respectively; The face acquisition module uses a multi-task convolutional neural network (MTCNN) model to detect faces in the video stream, extract key points and feature vectors, and transmit them to the classroom attendance module; The course creation module is used to manage course information and generate a course-camera-student association table; The classroom attendance module generates attendance records based on the association table and facial feature comparison results and stores them in a relational database; The abnormal behavior detection module outputs the behavior judgment result through YOLOv5 target detection and OpenPose posture analysis algorithm, combined with LSTM time series tracking and dynamic threshold mechanism. The output of the abnormal behavior detection module is stored in the time series database; The classroom situation analysis report generation module integrates multi-source data, generates an analysis report through an association rule algorithm, and stores the report in an object storage.

2. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The front-end user interface integrates a real-time monitoring panel, displays multiple video streams and overlays student face frames and status labels. The face frames use color coding to distinguish attendance status and abnormal behavior types, and support clicking on the face frames to retrieve students' historical attendance and behavior data. The interface supports course schedule creation and detection strategy configuration, including setting attendance validity time, abnormal behavior detection type and warning trigger conditions. The configuration information is transmitted to the back-end module via the gRPC protocol.

3. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The course creation module supports associating classroom camera equipment and student lists. The generated association table contains the course ID, camera IP address, student ID and teaching time period fields. The fields are mapped to the course table, equipment table and student table in the relational database through foreign keys. The gateway receives the front-end course configuration request and generates course-camera-student association data.

4. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The classroom attendance module sets an attendance valid time window, caches real-time attendance status through Redis, including student ID, attendance status and timestamp information, and pushes abnormal attendance warnings to the front end through the WebSocket protocol. The warning includes a red highlight mark and a sound reminder. The classroom attendance module uses the cosine distance similarity formula combined with dynamic weights to optimize face comparison. The formula is: ,in, is the similarity score, and is the eigenvector, is the time decay factor, where is the attenuation coefficient, which is determined by the face database update frequency statistics. It is a dynamic adjustment coefficient obtained by optimizing the historical attendance error rate.

5. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The dynamic threshold mechanism of the abnormal behavior detection module is adjusted across time periods through the behavior confidence fusion formula, which is: ,in, is the dynamic threshold, is the basic threshold, is the time domain adjustment threshold, is the weight coefficient, through Calculate, where For the The activity index of the class, The abnormal behavior detection module filters instantaneous actions through the LSTM time series tracking module and adopts the behavior continuity determination formula: Identify real abnormal behavior, where For the effectiveness score, is the behavioral feature vector, is the time window, is the inter-frame attenuation coefficient, is the Sigmoid activation function.

6. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The relational database uses MySQL to store student files, course information and attendance records. The student file table contains student ID, name, class and facial feature vector fields. The facial feature vector is stored as text data in a serialized manner. The time series database uses InfluxDB to store behavior logs, which contain student ID, timestamp, behavior type and duration fields. The object storage uses MinIO to store video clips and analysis reports, and supports distributed encrypted transmission.

7. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The classroom situation analysis report generation module integrates the Apriori association rule algorithm to mine the correlation between attendance data and abnormal behavior. The module calculates the classroom activity index through a multi-dimensional weighted sum formula: ,in, is the activity index, is the number of dimensions, is the original data of each dimension, For dimension weights, the report supports custom templates, including statistical charts, trend charts and abnormal video links.

8. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The back-end server adopts a microservice architecture, and each module is routed through the Flask application as an API gateway. The edge computing node is deployed at the classroom end to pre-process the camera video stream, including resolution compression and face ROI extraction, to reduce the back-end computing pressure. The edge node and the back-end server transmit the pre-processed facial feature data and video slices through the gRPC protocol.

9. The automatic attendance, abnormal behavior detection and classroom situation analysis device based on computer vision according to claim 1 is characterized in that: The abnormal behavior detection module supports multiple types of behavior detection, and the output results include behavior type, student coordinates and timestamp. The classroom attendance module supports a dynamic registration mechanism, which triggers the registration after receiving the new student face data entered by the front end. Incremental model training to update the feature vector of the face library.

Images