Classroom attendance checking method based on multi-source data fusion

Abstract

The embodiment of the invention provides a classroom attendance checking method based on multi-source data fusion, and the method comprises the steps: obtaining a face confidence coefficient, a seat occupation confidence coefficient and a clock-in confidence coefficient corresponding to each student identifier in a preset time window, the face confidence is used for representing the matching degree of feature data obtained based on the face image and pre-stored feature data; the seat occupancy confidence is used for representing an overlapping degree between a target space area obtained based on the face image and a pre-stored target seat area; the clock-in confidence is used for representing the effective probability of clock-in; performing weighted summation on the face recognition confidence coefficient, the seat occupation confidence coefficient and the card punching confidence coefficient of the target student identifier to obtain the attendance confidence coefficient of the target student; and the real-time attendance state of the target student is output based on the attendance confidence of the target student, and the real-time attendance state at least comprises class arrival, non-class arrival and abnormal recheck.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a classroom attendance method that integrates multi-source data. Background Technology

[0002] With the continuous advancement of smart campus and digital teaching construction, classroom attendance needs are gradually evolving from traditional manual roll call to automation and intelligence. Existing attendance methods primarily rely on a single data source, such as recording entry and exit times via card readers or using facial recognition via cameras. However, these single methods often face multiple limitations in actual teaching scenarios. On the one hand, card readers can only record entry and exit behavior, failing to reflect whether students are actually seated or have left midway, and are also insufficient to meet the real-time requirements of the classroom. On the other hand, relying solely on facial recognition is easily affected by factors such as changes in lighting, occlusion, camera angle, and student movement, leading to unstable recognition or missed detections. Furthermore, camera recognition struggles to directly confirm the correspondence between students and their assigned seats, easily resulting in the identification of a student but inability to confirm their seating position.

[0003] In actual teaching management, schools generally require students to sit in fixed seats. Therefore, relying solely on one type of information cannot accurately reflect whether students are actually present in class or seated in the correct location. Especially in large classes, with free entry and exit, complex lighting conditions, or when students briefly leave their seats, existing systems cannot provide stable and reliable real-time attendance results. Although some systems have attempted to combine facial recognition with simple sign-in records, they lack an effective data fusion mechanism and cannot guarantee attendance accuracy and consistency in complex scenarios. This problem becomes increasingly prominent as the number of students and their dynamic classroom behavior increases.

[0004] Therefore, there is an urgent need for an attendance method that can improve the accuracy of attendance tracking. Summary of the Invention

[0005] The purpose of this application is to provide a classroom attendance method that integrates multi-source data, and the specific technical solution adopted is as follows: In a first aspect, embodiments of this application provide a classroom attendance method based on multi-source data fusion, the method comprising: Within a preset time window, the face confidence score, seat occupancy confidence score, and attendance confidence score corresponding to each student identifier are obtained. The face confidence score is used to characterize the matching degree between the feature data obtained based on the face image and the pre-stored feature data; the seat occupancy confidence score is used to characterize the overlap between the target spatial region obtained based on the face image and the pre-stored target seat region; and the attendance confidence score is used to characterize the probability of an effective attendance check-in. The attendance confidence of the target student is obtained by weighted summing of the confidence scores of facial recognition, seat occupancy, and attendance check-in. The real-time attendance status of the target student is determined based on the attendance confidence level of the target student, wherein the real-time attendance status includes at least attendance, absence, and abnormality review.

[0006] Secondly, a classroom attendance device integrating multi-source data is provided, the device comprising: The acquisition module is used to acquire the face confidence score, seat occupancy confidence score, and attendance confidence score corresponding to each student identifier within a preset time window. The face confidence score characterizes the matching degree between feature data obtained from the face image and pre-stored feature data; the seat occupancy confidence score characterizes the overlap between the target spatial region obtained from the face image and the pre-stored target seat region; and the attendance confidence score characterizes the probability of a valid attendance check-in. The weighted summation module is used to perform a weighted summation of the face recognition confidence, seat occupancy confidence, and attendance confidence of the target student to obtain the attendance confidence of the target student. The determination module is used to determine the real-time attendance status of the target student based on the attendance confidence level of the target student, wherein the real-time attendance status includes at least attending class, absent from class, and abnormal review.

[0007] Thirdly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described above.

[0008] Fourthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described above.

[0009] This application has the following beneficial effects: Anti-interference capability: Multi-dimensional data complement each other, reducing misjudgments caused by single data anomalies (such as face recognition failure but the person is actually present); Dynamics: Continuous data collection within a preset time window, supporting real-time updates of attendance status; Explainability: The weighted summation and threshold determination logic is transparent, facilitating auditing and optimization.

[0010] It can be widely used in smart campuses, teaching management systems and automated attendance scenarios, and can provide accurate, real-time and stable student attendance status without human intervention, providing an efficient solution for the digitalization and intelligentization of teaching management. Attached Figure Description

[0011] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the multi-source data fusion-based classroom attendance method provided in this application embodiment; Figure 2 A flowchart illustrating a multi-source data fusion algorithm provided in an embodiment of this application; Figure 3 A schematic diagram of a classroom attendance device with multi-source data fusion provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer block device provided in an embodiment of this application. Detailed Implementation

[0013] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a multi-source data fusion classroom attendance method proposed according to this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined from any suitable form.

[0014] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0015] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0017] This application provides a classroom attendance method that integrates multi-source data, such as... Figure 1 As shown, this can be achieved through the following steps: Step S110: Within a preset time window, obtain the face confidence score, seat occupancy confidence score, and attendance confidence score corresponding to each student identifier. The face confidence score characterizes the matching degree between feature data obtained from the face image and pre-stored feature data; the seat occupancy confidence score characterizes the overlap between the target spatial region obtained from the face image and the pre-stored target seat region; and the attendance confidence score characterizes the probability of a valid attendance check-in. Here, within a preset time window (such as 5 minutes after the start of each class, every 15 minutes, etc.), the system synchronously collects three types of confidence data for each student: Face confidence score s_face: The matching probability output by the face recognition algorithm after capturing the student's facial features through the camera and comparing them with the database. For example, 95% means "it is very likely that the student is student number 001".

[0018] Seat occupancy confidence s_seat: Detects whether a student is actually sitting in a designated seat using classroom seat sensors (such as pressure-sensitive pads, infrared sensors) or cameras, and outputs the confidence probability of "occupancy". For example, the camera can be used to detect whether the student with student ID 001 is actually sitting in seat number 001.

[0019] The attendance tracking confidence score s_punch: This score calculates the probability that a student successfully completed their attendance tracking via card swiping, QR code scanning, or app check-in. For example, it determines whether student ID 001 successfully checked in via card swiping.

[0020] The core of this step is multi-source data collection to ensure that student attendance evidence is obtained from three dimensions: visual, physical location, and behavioral operation.

[0021] Step S120: The confidence scores of the face recognition, seat occupancy, and attendance of the target student are weighted and summed to obtain the attendance confidence score of the target student. During implementation, for a single target student (such as student ID 001), the system weights and sums the three confidence levels according to preset weights to generate a total attendance confidence score. For example: The weighting might be: facial recognition (40%), seat occupancy (30%), and check-in (30%). If the confidence level of student No. 001's face is 90%, the confidence level of seat occupancy is 80%, and the confidence level of attendance is 70%, then the attendance confidence level = 90% × 0.4 + 80% × 0.3 + 70% × 0.3 = 81%.

[0022] The core of this step is data fusion, which integrates multi-dimensional evidence through weighted averaging to avoid misjudgments or omissions from a single data source (such as relying solely on attendance records).

[0023] During implementation, weight allocation and threshold settings can be adjusted based on specific scenarios (such as classroom environment and student behavior habits) through training with historical data to improve the accuracy of judgment.

[0024] Step S130: Output the real-time attendance status of the target student based on the attendance confidence level of the target student, wherein the real-time attendance status includes at least attendance, absence, and abnormality review.

[0025] During implementation, the system can output real-time attendance status based on the target students' total attendance confidence score, typically including three categories: Attendance: A total confidence score ≥ the threshold (e.g., 85%) is considered "normal attendance". Missing class: If the total confidence score is less than or equal to the threshold (e.g., 30%), it is considered "missing class". Anomaly review: If the total confidence score is in the middle range (e.g., 30% to 85%), manual review or secondary verification is triggered (e.g., retrieving classroom recordings or contacting the counselor for confirmation).

[0026] The core of this step is the decision output, which achieves automated judgment through threshold division, while retaining the "anomaly review" mechanism to handle boundary cases, balancing efficiency and accuracy.

[0027] In this embodiment, the attendance process is optimized collaboratively from three dimensions: multi-source data fusion, identity binding, and spatial location association. Students' faces are identified in real-time via cameras, and their spatial coordinates in the classroom are obtained. Positional constraints are established using students' fixed seat numbers to ensure a one-to-one correspondence between the identified student's identity and their seat location. Simultaneously, a complete data link is constructed by combining the time and identity verification information provided by the attendance records, enhancing the reliability of the attendance results. Through the three stages of "collection-fusion-determination," closed-loop management from multi-source data to attendance status is achieved. The fusion of facial camera data, fixed seat number information, and student attendance records enables highly reliable and robust real-time attendance determination. This real-time and accurate determination of student attendance status aims to improve the automation, reliability, and real-time performance of attendance processing in teaching scenarios, thereby enhancing the overall efficiency of smart classrooms and teaching management systems.

[0028] It has the following beneficial technical effects: Anti-interference capability: Multi-dimensional data complement each other, reducing misjudgments caused by single data anomalies (such as face recognition failure but the person is actually present); Dynamics: Continuous data collection within a preset time window, supporting real-time updates of attendance status; Explainability: The weighted summation and threshold determination logic is transparent, facilitating auditing and optimization.

[0029] It can be widely used in smart campuses, teaching management systems and automated attendance scenarios, and can provide accurate, real-time and stable student attendance status without human intervention, providing an efficient solution for the digitalization and intelligentization of teaching management.

[0030] In some embodiments, the step S110 above, "obtaining the face recognition confidence level corresponding to each student identifier", can be achieved through the following steps: Step 11: Use a face recognition model to identify the collected multi-frame face images to obtain the first multi-frame identity sequence; During implementation, the classroom environment is monitored in real time via camera video streams, including face detection, face alignment, face feature extraction, and cross-frame tracking. The system employs a lightweight, high-precision face recognition model (such as deep learning models like FaceNet and ArcFace) to recognize each frame of continuously acquired video images, outputting the corresponding identity ID (such as student ID or employee ID) for each frame, forming the first multi-frame identity sequence ordered by time.

[0031] By leveraging the temporal characteristics of video, random errors caused by pose changes and blurring in single-frame recognition are avoided, and serialized data provides context for subsequent occlusion detection.

[0032] Step 12: Perform occlusion probability detection on the first multi-frame identity sequence; During implementation, occlusion probability analysis is performed on the first multi-frame identity sequence. Common methods include: Image level: Calculate the occlusion area ratio of each frame through facial key point detection (such as eye and nose occlusion) or occlusion region segmentation model; At the sequence level: Analyze the abnormal frequency of identity switching (such as frequent identity switching in a short period of time may indicate occlusion leading to misidentification), or detect the "jumping" of identities in the sequence through a temporal consistency model.

[0033] Thus, due to uncontrollable factors in actual classrooms such as student occlusion, brief gatherings of multiple people, light reflection, and changes in camera angle, the results of face recognition and seat matching can be unstable between consecutive frames. During implementation, low-quality recognition frames that may be caused by occlusion can be located first, providing target regions (target subsequences) for subsequent repair.

[0034] Step 13: If the occlusion probability of the target subsequence in the first multi-frame identity sequence is greater than the probability threshold, the target subsequence is reconstructed using a reconstruction model to obtain the second multi-frame identity sequence. The reconstruction model is trained using a temporal prediction and neighboring frame compensation mechanism. During implementation, mask transformations of varying intensities are applied to detected abnormal locations (target sub-sequences), and a reconstruction model is used to recover the disturbed segments. The reconstruction model, by jointly optimizing the sanitization loss and temporal consistency loss, ensures that the output sequence maintains consistency with the natural classroom behavior trajectory while recovering the occluded content.

[0035] When the occlusion probability of the target subsequence (e.g., 3 consecutive frames) exceeds a preset threshold (e.g., occlusion area > 30%), the reconstruction process is initiated.

[0036] Using time series models such as LSTM and Transformer, the identity information of the occluded frame is predicted based on the normal frames before and after the subsequence (e.g., the first two frames are student 001, the last two frames are student 001, and the predicted middle occluded frame is still student 001). Neighboring frame compensation: By estimating the optical flow or interpolating the features of adjacent frames, visual features (such as facial texture and key point positions) of the occluded area are filled in, and a second multi-frame identity sequence after restoration is generated (e.g., [student001, student001 (restored), student001]).

[0037] By fusing multimodal information, the system corrects recognition gaps caused by occlusion, improving the continuity and accuracy of sequences. Through this mechanism, the system can effectively repair problems such as identity mutations and recognition interruptions in single frames or short sequences, avoiding misjudgments of absence or seat changes.

[0038] In some embodiments, during model training, the parameters of the face recognition model are frozen, and the parameters of the reconstruction model are adjusted using posture features, facing direction features, and swing range features, so that the reconstruction model can reconstruct the identity sequence using temporal prediction and neighbor frame compensation mechanisms.

[0039] Here, during model training, the system freezes the main face recognition network and only fine-tunes the temporal parameters of the cleanup module (reconstruction model), enabling it to automatically learn "normal behavior patterns" in a classroom setting, including students' approximate sitting posture, facing direction, and short-term swaying range. During real-time operation, when the system detects transient disturbances such as single-frame abnormal recognition, brief occlusion, rapid head turning, or slight leaving of the seat, the cleanup module automatically fills in the abnormal frames using temporal prediction and neighboring frame compensation mechanisms, outputting a continuous and stable identity sequence. This mechanism proactively suppresses unstable factors such as "false jumps," short-term recognition gaps, and abnormally low confidence levels, ensuring that attendance results are not affected by short-term noise.

[0040] Step 14: Use the second multi-frame identity sequence and the pre-stored identity sequence to perform similarity matching to obtain the face recognition confidence level.

[0041] During implementation, the second multi-frame identity sequence is compared with a pre-stored identity sequence (such as the standard facial feature vector sequence of student number 001 in the database) based on similarity. Common methods include: Sequence similarity: Calculate the cosine similarity and Euclidean distance between the repaired sequence and the standard sequence; Probabilistic fusion: The recognition confidence of each frame (such as the probability value output by the model) is combined and weighted to calculate the final comprehensive confidence (such as 95%).

[0042] In this way, the reliability of the quantitative identification results provides key input for attendance determination (such as step S120).

[0043] In this embodiment, a closed loop is achieved from single frame to sequence and from raw data to confidence level through four steps: "identification-detection-repair-matching". Its core advantage lies in: Occlusion resistance: Effectively handles recognition problems in occluded scenarios through temporal prediction and neighboring frame compensation; Dynamic adaptability: Multi-frame sequence analysis can adapt to dynamic scenarios such as changes in student posture and lighting conditions; Confidence quantification: Provide interpretable confidence indicators through similarity matching to support subsequent attendance status determination (such as step S130).

[0044] This process can be integrated into smart classrooms and attendance systems to improve recognition accuracy in complex scenarios and reduce manual verification costs. Through the aforementioned state purification and interference filtering mechanisms, it can automatically correct short-term recognition errors in consecutive frames, avoiding erroneous attendance records caused by occlusion, momentary false detections, or camera shake, ensuring stable output even in complex classroom scenarios. It significantly enhances the overall stability of the three data sources (face, seat, and card reader) in complex classroom environments, resulting in higher reliability, consistency, and robustness in the final merged attendance status.

[0045] In some embodiments, the face recognition model adopts a bidirectional multi-scale fusion structure; step 11 above, "using the face recognition model to identify multiple frames of face images and obtain the first multi-frame identity sequence," can be implemented through the following process: Using the aforementioned bidirectional multi-scale fusion structure, the face recognition model obtains the first multi-frame identity sequence with cross-scale semantic enhancement and spatial detail compensation through bidirectional information flow from top to bottom and bottom to top during the recognition of the multi-frame face images.

[0046] Here, the face recognition model first uses a convolutional neural network (such as ResNet, HRNet, etc.) to extract features at multiple scales from each frame of the face image, generating feature maps at different levels (such as low-level high-resolution spatial details, mid-level semantic information, and high-level global semantics). For example: Underlying features: Preserve spatial details such as face edges and textures (e.g., 128×128 resolution, 64 channels). High-level features: Extract semantic information of the overall facial structure and key regions (such as eyes and nose) (e.g., 32×32 resolution, 512 channels).

[0047] Two-way information flow paths include the following two types: Bottom-up approach: Starting from the low-level detailed features, spatial detail information is passed to higher levels through layer-by-layer upsampling and feature fusion (such as the skip connections of U-Net), thus avoiding the loss of semantic information; Top-down approach: Starting from high-level semantic features, global semantic information is fed back to lower levels through downsampling and attention mechanisms (such as SE module, CBAM) to enhance the semantic consistency of detailed features.

[0048] At each interaction node in the bidirectional path, the model integrates features at different scales through feature concatenation, weighted summation, or attention fusion (such as PANet's Adaptive Feature Pooling). For example, low-level features provide details such as the precise location of the face and changes in expression; high-level features provide semantic information such as the face's category label and pose estimation; the fused features contain both high-resolution spatial details and strong semantic expressive power.

[0049] After bidirectional multi-scale fusion and temporal modeling, the features of each frame image are mapped to identity vectors (such as 128-dimensional embedding vectors) and arranged in chronological order to form the first multi-frame identity sequence.

[0050] High-level semantic information is passed down to the bottom level through a top-down path, making the recognition of low-level detailed features (such as eyes and nose) more consistent with global semantics; detail compensation: low-level spatial details are passed down to the high level through a bottom-up path, making high-level semantic features more accurately locate specific facial regions and reducing misjudgments (such as misjudging occluded parts as other people).

[0051] In complex scenarios such as occlusion and blurring, the bidirectional fusion structure can automatically repair locally missing details or semantics through cross-scale information complementarity, thereby improving the robustness of identity sequences.

[0052] In actual classrooms, student distribution exhibits significant scale differences: students in the front rows are closer to the camera, resulting in higher image resolution; students in the back rows are farther away, their faces appear very small in the image, and there may even be overlapping of multiple students or distorted corners. Therefore, single-scale feature extraction methods are insufficient for uniform recognition of all students in the class. To address this issue, a bidirectional multi-scale feature fusion structure is adopted to effectively model the faces and seating areas of students at different sizes and distances. A top-down path enhances semantic features, while a bottom-up path supplements detailed features. Dynamically learnable weights are introduced to enable the system to adapt to different classroom layouts and student distributions, improving recognition performance. The contribution of cross-scale features is optimized to improve recognition accuracy in scenarios with dense student distribution and significant changes in scale.

[0053] In this embodiment, a face recognition model employing a bidirectional multi-scale fusion structure avoids the problem of lost spatial details in high-level semantics in traditional single-scale models, while overcoming the limitation of low-level details lacking semantic guidance. In scenarios such as intelligent attendance and video surveillance, it can significantly improve the accuracy and anti-occlusion capability of multi-frame face recognition, providing more reliable basic data for subsequent attendance confidence calculation.

[0054] In some embodiments, obtaining the seat occupancy confidence level in step S110 above can be achieved through the following steps: Step 21: Obtain the seating area template generated from the classroom layout information. The seating area template represents the mapping relationship between each student identifier and the seating area. During implementation, a seating area template can be generated based on the actual physical layout of the classroom (such as floor plan, CAD drawings, or on-site measurement data). This template is a spatial mapping table that records each student's unique identifier (such as student ID, name) and the geometric attributes of their corresponding seating area (such as a rectangular area with the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) or a circular area with the center point plus the radius).

[0055] The location, size, and arrangement of classroom seats are obtained through manual annotation or automatic extraction (such as image segmentation algorithms); a two-way mapping between student identifiers and seat areas is established (e.g., student ID 001 corresponds to seat 5 in row 3), forming queryable template data (such as JSON, database tables).

[0056] This provides a benchmark for subsequent spatial matching, ensuring the accuracy of the "student-seat" association.

[0057] Step 22: Perform perspective correction and spatial mapping on the acquired classroom images to obtain the spatial area for each student's identification. Here, the classroom images captured by the camera may have perspective distortion (such as objects appearing larger when closer and smaller when farther away, or tilted angles), causing the seating area to appear non-rectangular or distorted in size in the image.

[0058] Image correction is achieved through perspective transformation. For example, four actual corner points of the classroom (such as the endpoints of the intersection of the wall and the floor) are selected as control points, the transformation matrix is ​​calculated, and the seating area in the image is mapped to a top view (orthographic projection), restoring its true shape and proportions.

[0059] Spatial mapping includes coordinate transformation and student positioning as follows: Coordinate transformation: Converting the corrected image pixel coordinates (u, v) to actual physical coordinates (x, y) (e.g., metric units based on the classroom floor). This requires camera calibration (obtaining camera intrinsic and extrinsic parameters) and coordinate system transformation (image coordinate system → camera coordinate system → world coordinate system).

[0060] Student localization: Identify students in images using object detection algorithms (such as YOLO and Face Detection) and obtain the bounding boxes of their heads or bodies, then calculate the physical coordinates of their center points.

[0061] In this way, the student positions in the image can be mapped to the actual classroom space, providing a spatial reference for matching with the seating area template.

[0062] Step 23: Determine the seat occupancy confidence level for seat matching based on the spatial region of each student and the seat region model.

[0063] During implementation, the student space area (such as the coordinates of the head center point) obtained in step 22 is subjected to geometric overlap analysis with the seat area template in step 21 to quantify whether the student is in the correct seat.

[0064] The confidence level calculation method includes at least one of the following: Region overlap: Calculate the percentage of overlap between the student's positioning area (e.g., a circular area) and the template seating area. For example, if the overlap between the student's head area and the seating area accounts for 80% of the seating area, the confidence level is 0.8.

[0065] Distance threshold: Calculate the Euclidean distance between the student's center point and the center point of the template seat. If the distance is less than the preset threshold (e.g., 0.5 meters), it is judged as "occupied" with a confidence level of 1; if the distance exceeds the threshold, the confidence level decreases linearly (e.g., the confidence level decreases by 0.1 for every 0.1 meter increase in distance).

[0066] Multi-dimensional fusion: Combining multi-dimensional features such as regional overlap, distance, and student posture (e.g., whether facing the podium), the comprehensive confidence score is output through weighted summation or machine learning models (e.g., SVM, random forest).

[0067] Finally, the seat occupancy confidence score for each student is obtained (e.g., 0.9 means "most likely to be sitting in the correct seat"). This value will be used as one of the inputs for the weighted summation in step S120 to determine the attendance status.

[0068] To address the challenges of fixed seating, large student numbers, and image obstruction in real-world classroom applications, this application establishes a two-way constraint mechanism for "fixed seating – student identity." The system first generates a seat coordinate template based on the classroom layout and performs perspective correction and spatial mapping on the camera image to obtain the spatial correspondence between seats and the image. Subsequently, the overlap between the identified student positions and the seat template is calculated, serving as the confidence score s_seat for seat matching.

[0069] In this embodiment, perspective correction restores the true geometric shape of the seats, avoiding misjudgments caused by camera angles; camera calibration and coordinate system transformation enable precise mapping of student positions from images to actual space; and geometric indicators such as area overlap and distance quantify seat occupancy, providing interpretable and quantifiable input for the attendance system. This can be widely applied in scenarios such as smart classrooms, meeting check-in, and exam room monitoring, improving the accuracy and efficiency of attendance management through precise seat occupancy determination.

[0070] In some embodiments, step 23 above, "determining the seat occupancy confidence based on the spatial region of each student and the seat region model," can be achieved through the following steps: Step 231: Obtain the target spatial region of the target student identifier; During implementation, based on the classroom image obtained through perspective correction and spatial mapping in step 22, target detection algorithms (such as YOLO and Mask R-CNN) are used to identify the position of the target student (such as student ID 001) in the image, and the actual spatial region is obtained through coordinate transformation (image coordinates → physical coordinates).

[0071] The object detection outputs the student's bounding box or segmentation mask to determine the outline of their head / body.

[0072] By using camera calibration parameters (intrinsic and extrinsic parameters), the pixel coordinates in the image are converted into physical coordinates in the classroom floor coordinate system, forming the spatial region R of the target student. face (e.g., rectangular or circular areas).

[0073] This provides the actual space occupied by students for subsequent matching with the seating area template.

[0074] Step 232: Based on the target student identifier and the seat area template, determine the target seat area for the target student; During implementation, based on the seating area template (a mapping table between student identifiers and seating areas) generated in step 21, the corresponding standard seating area R of the target student is queried using the unique identifier of the target student (such as student ID or name). seat .

[0075] Seating area templates are typically stored in the form of database tables, JSON files, or spatial indexes (such as R-Tree) to support fast querying; For example, the seating area corresponding to student ID 0001 may be "seat 5 in row 3", and its geometric properties are a rectangular area (x1=2.0m, y1=3.0m, x2=2.5m, y2=3.5m) or a circular area (center point (2.25m, 3.25m), radius 0.3m).

[0076] Define the baseline seating area where the target student "should" sit, providing a benchmark for subsequent intersection calculations.

[0077] Step 233: Determine the intersection region of the target spatial region and the target coordinate region; During implementation, geometric calculations can be used to calculate the overlap between the target student's actual spatial area and the standard seating area.

[0078] Intersection of rectangular regions: If both are rectangles, the intersection region can be determined by calculating the coordinate range of the overlapping rectangles; Intersection of circular regions: If the region is circular, the area of ​​the intersection can be determined by calculating the relationship between the distance between the centers of the two circles and their radii (e.g., using the formula for the area of ​​a circle or numerical integration). Mixed regions: If the region has a complex shape (such as a polygon), spatial operation libraries (such as Shapely, OpenCV's contourArea) can be used to calculate the intersection area.

[0079] In this way, the degree of spatial overlap between the student's actual location and the standard seat is quantified, providing core data for confidence calculation.

[0080] Step 234: Determine the ratio of the intersection region to the target coordinate region as the seat occupancy confidence level.

[0081] Here, the area (or number of pixels) of the intersection region obtained in step 233 is divided by the total area of ​​the target seat region to obtain the seat occupancy confidence.

[0082] The confidence level is calculated using the following formula (1): s_seat=R face ∩R seat / R seat (1); Where s_seat represents the seat occupancy confidence score, R face R represents the target spatial region. face ∩R seat R represents the area of ​​the intersection region. sea Indicates the target seating area.

[0083] Normalization: The ratio range is [0, 1], where 0 indicates no overlap and 1 indicates complete overlap; Threshold determination: Confidence thresholds can be set according to the application scenario (e.g., ≥0.8 is "occupied", ≤0.3 is "unoccupied", and the middle value is "abnormal review").

[0084] By quantifying the geometric overlap, we can intuitively reflect whether students are "sitting in the correct seats," providing key input for attendance status determination (such as step S130).

[0085] In this embodiment, a complete computational chain of "location-template-overlap-confidence" is formed by acquiring student space areas, querying standard seat areas, calculating intersections, and finally quantifying confidence. This approach calculates confidence based on the geometric overlap of actual spatial areas, aligning with human perception of seat occupancy. Perspective correction and spatial mapping eliminate the influence of factors such as camera angle and distance on location judgment. The confidence level is presented as a ratio, making it easy to understand and verify. This method can be widely applied to scenarios requiring precise seat management, such as smart classrooms, meeting attendance tracking, and exam room monitoring, improving the accuracy and reliability of attendance determination through quantified confidence.

[0086] In some embodiments, obtaining the attendance confidence level in step 110 above can be achieved through the following steps: Step 31: Obtain attendance record data, wherein the attendance record data includes at least student identifier, attendance timestamp, and location information; During implementation, the system obtains raw records from attendance devices (such as card readers, barcode scanners, and APP location services), which include the following three key fields: Student identification: A unique identity ID (such as student ID or employee ID); Check-in timestamp: Check-in time accurate to the second (e.g., 2025-12-09 09:05:30); Location information: Geographic coordinates (such as GPS latitude and longitude) or area code (such as classroom number "A101") at the time of check-in.

[0087] This provides the original data source for subsequent time matching and location verification, ensuring that confidence calculations are based on real and complete attendance records.

[0088] Step 32: Determine the first sub-confidence level based on the matching degree between the check-in timestamp and the preset time period corresponding to the student identifier; Here, the matching degree between the check-in timestamp and the preset time period is calculated. The preset time period is usually set based on the course schedule or attendance rules (such as 10 minutes before the start of the class to 5 minutes after the start of the class is a valid check-in period).

[0089] The calculation method is illustrated below: Absolute time difference: Calculate the difference between the check-in time and the preset time period boundary (e.g., if the course starts at 9:00 and the check-in time is 8:58, then the time difference is 2 minutes). Matching function: The time difference is mapped to the first sub-confidence using a piecewise function or Gaussian distribution (e.g., the first sub-confidence is 1 when the time difference is ≤5 minutes, linearly decays to 0.5 from 5 to 10 minutes, and is 0 when the time difference is >10 minutes). Example: If student number 001 corresponds to the preset time period from 8:50 to 9:10, and student number 001 checks in at 8:55, the time difference is 5 minutes (within the allowable range), the first sub-confidence level = 1; if student number 001 checks in at 9:15, the time difference is 5 minutes (outside the time period), the first sub-confidence level = 0.

[0090] This allows us to verify whether the check-in behavior occurred within a reasonable time window, eliminating erroneous check-ins such as being late, leaving early, or outside of class time.

[0091] Step 33: Determine the second sub-confidence level based on the matching degree between the location information and the preset location corresponding to the student identifier; Here, the consistency between the check-in location information and the preset location is verified. The preset location is usually an attendance area such as a classroom or laboratory, which can be defined by geofencing (such as a polygonal area) or area coding (such as a classroom number).

[0092] The calculation method is illustrated below: Geofence matching: Determine whether the check-in coordinates fall within the polygonal area of ​​the preset location corresponding to student number 001 (such as the fence area of ​​classroom A101). Distance threshold: Calculate the distance between the check-in location and the center point of the preset location. If it is less than the threshold (e.g., 50 meters), the confidence level is high. Area code matching: directly compare the area code in the attendance record with the preset location code to see if they match; Example: If the preset location is "Classroom A101" (center point coordinates (116.3, 39.9)), and the student's check-in coordinates are (116.301, 39.902), which is about 15 meters away from the center point (<50-meter threshold), then the second sub-confidence = 1; if the check-in coordinates are outside the campus and the distance is >500 meters, then the second sub-confidence = 0.

[0093] This ensures that the check-in process occurs in the correct physical space, preventing misjudgments caused by proxy check-ins across regions or location drift.

[0094] Step 34: Perform a weighted summation of the first sub-confidence and the second sub-confidence to obtain the attendance confidence.

[0095] During implementation, the confidence scores of the time dimension (first sub-confidence score) and the confidence scores of the spatial dimension (second sub-confidence score) are weighted and averaged according to preset weights to generate the final check-in confidence score.

[0096] The calculation method is illustrated below: Weighting: Set weights according to scenario requirements (e.g., time weight 60%, space weight 40%), and the confidence calculation formula (2) is as follows: Confidence of attendance = w1 × first sub-confidence + w2 × second sub-confidence (2); Where w1 is the weight parameter of the first sub-confidence and w2 is the weight parameter of the second sub-confidence.

[0097] Example: If the first sub-confidence score is 0.9 (good time match), the second sub-confidence score is 0.8 (good location match), and the weights are 0.6 and 0.4 respectively, then the overall confidence score is 0.9 × 0.6 + 0.8 × 0.4 = 0.86; Threshold determination: The overall confidence level can be further used to determine attendance status (e.g., ≥0.8 is "valid attendance", ≤0.4 is "invalid attendance", and intermediate values ​​need to be verified).

[0098] In this way, the robustness of confidence is improved by multi-dimensional fusion, avoiding misjudgment based on a single dimension (such as only time or only location), while supporting weight adjustment to adapt to different scenarios (such as classroom attendance with more emphasis on location, and library reservation with more emphasis on time).

[0099] In this embodiment, a complete "data-verification-fusion-output" chain is formed, from data collection, time matching, and spatial matching to comprehensive confidence level. This dual verification of time and space effectively prevents abnormal behaviors such as proxy attendance and location-based cheating; the confidence level is presented in numerical form, supporting threshold setting and automated judgment; parameters such as weights and thresholds can be dynamically adjusted according to actual scenarios (e.g., increasing location weight during exams); it is widely applicable to scenarios requiring accurate attendance verification, such as school attendance, corporate access control, and meeting check-in, improving system reliability through multi-dimensional confidence level calculation.

[0100] In some embodiments, the above step S120, "weighted summing of the face recognition confidence, seat occupancy confidence, and attendance confidence of the target student to obtain the attendance confidence of the target student," can be achieved through the following steps: Step 121: Determine the first weighting parameter of the face recognition confidence based on the recognition confidence, trajectory stability and environmental conditions; Here, face recognition confidence, such as a similarity score of 0.9; trajectory stability, such as the range of coordinate fluctuations in continuous multi-frame detection; and environmental conditions, such as light intensity and degree of occlusion.

[0101] Identify confidence weights and directly map them to face similarity scores (e.g., 0.8-1.0 corresponds to a weight of 0.6-0.9, and the higher the score, the greater the weight). The trajectory stability weight is calculated by taking the variance of the student's head / face coordinates in consecutive frames (e.g., weight +0.1 when variance < 0.5 pixels, weight -0.1 when variance > 2 pixels), which reflects the detection stability. Environmental condition weights are calculated based on environmental sensor data (e.g., when the light sensor value is >500 lux, the weight is increased by 0.05; when the occlusion detection algorithm outputs an occlusion ratio <10%, the weight is increased by 0.05). The comprehensive weighting is achieved by summing the weights of the three factors mentioned above (e.g., 60% for recognition confidence, 30% for trajectory stability, and 10% for environmental conditions) to obtain the first weight parameter. Example: If the face similarity is 0.98 (weight 0.85), the trajectory variance is 0.3 pixels (weight +0.1), and the illumination is 600 lux (weight +0.05), then... First weighting parameter W face =0.85×0.6+0.1×0.3+0.05×0.1=0.54.

[0102] In this way, by dynamically adjusting the weights, the impact of environmental interference (such as backlighting or occlusion) or detection jitter on the confidence of the face can be reduced, thereby improving reliability.

[0103] Step 122: Determine the second weighting parameter of the seat occupancy confidence level based on the fixed probability of seat occupancy; Here, the fixed probability of seat occupancy is such as the "prior probability of a student sitting in the correct seat" based on historical data statistics.

[0104] If historical data shows that students sit in the correct seats 80% of the time, then the second weighting parameter W is directly set. seat It is 0.8; In some embodiments, the fixed probability can be adjusted based on classroom type (e.g., fixed-seat classrooms have a higher weight, while free-seat classrooms have a lower weight) or course nature (e.g., exam classes have a higher weight, while discussion classes have a lower weight).

[0105] Example: In a classroom with fixed seating, if 90% of students correctly identify their seats in history, then the second weight parameter is 0.9.

[0106] In this way, the basic reliability of seat occupancy is reflected by prior probability, providing a stable benchmark for subsequent fusion.

[0107] Step 123: Determine the third weighting parameter of the attendance confidence based on the abnormal markers and time offset of the attendance; Here, abnormal attendance records are marked, such as "attendance by proxy", "device malfunction", and time offset (the difference between the attendance time and the course start time).

[0108] Anomaly marking weight: If a attendance record is marked as abnormal (such as proxy attendance), the weight is set to 0; if it is normal, the weight is 1. Time offset weight: Calculate the ratio of the time offset to the preset threshold (e.g., if you check in within 10 minutes of the start of the class, the offset is 5 minutes and the threshold is 10 minutes, then the weight is 1 - 5 / 10 = 0.5). Multiply the anomaly label weight by the time offset weight (e.g., anomaly label weight 1 × time offset weight 0.5 = 0.5) to obtain the third weight parameter Wpunch.

[0109] Example: If the check-in is normal (abnormal marker weight 1) and the time offset is 3 minutes (threshold 10 minutes, weight 0.7), then the third weight parameter = 1 × 0.7 = 0.7.

[0110] In this way, by dynamically adjusting the weights, the interference of abnormal attendance (such as proxy attendance or lateness) on attendance confidence can be reduced.

[0111] Step 124: Based on the first weight parameter, the second weight parameter, and the third weight parameter, the face recognition confidence, seat occupancy confidence, and attendance confidence of the target student are weighted and calculated to obtain the attendance confidence of the target student.

[0112] Core logic: Assign confidence scores (s_face, s_seat, s_punch) and corresponding weights (W) to the face recognition confidence score (s_face), seat occupancy confidence score (s_seat), and attendance tracking confidence score (s_punch). face W seat The Wpunch weighted summation formula (3) is as follows: Attendance confidence=W face ×s_face+W seat ×s_seat+Wpunch×s_punch (3); Example: If s_face = 0.9 (face confidence), s_seat = 0.8 (seat occupancy confidence), and s_punch = 0.7 (check-in confidence), the weight W face =0.5, W seat =0.3, Wpunch=0.2, then the attendance confidence level = 0.5×0.9+0.3×0.8+0.2×0.7=0.83.

[0113] Threshold determination: Attendance thresholds can be set according to the scenario (e.g., ≥0.8 is "attendance", ≤0.5 is "absence", and intermediate values ​​need to be reviewed).

[0114] In this way, by integrating multi-dimensional data, students' attendance status can be comprehensively evaluated, avoiding misjudgments from a single data source (such as false detections of facial recognition, incorrect seat occupancy, and abnormal check-in).

[0115] In this embodiment, a complete chain of "data quantification - weight allocation - comprehensive judgment" is formed, from weight calculation to weighted fusion. In this way, the weights for face recognition and attendance tracking can be dynamically adjusted according to the environment and time, while the seat weight provides a stable benchmark; interference from proxy attendance and lateness is reduced through anomaly marking and time offset; the weight allocation logic is transparent, facilitating manual review and system optimization; it is widely applicable to scenarios requiring high-precision attendance judgment, such as smart classrooms, corporate attendance systems, and meeting check-in, improving system reliability through multi-dimensional data fusion. By using a structured and quantitative approach, it solves the problem of "single data source dependence" in traditional attendance systems, providing a highly reliable comprehensive basis for attendance status judgment, which is the core decision-making logic of the intelligent attendance system.

[0116] In some embodiments, the step S130 above, "determining the real-time attendance status of the target student based on the target student's attendance confidence level," can be achieved through the following steps: Step 131: Obtain multiple attendance confidence scores of the target students at different time points within the preset time window; Within a preset time window (e.g., 10 minutes before the start of class to 30 minutes after the start of class), collect the attendance confidence Ct of the target students at fixed time intervals (e.g., every 30 seconds) or event triggers (e.g., students entering the classroom, clocking in, or checking seats).

[0117] Set time windows according to the needs of the scenario, for example: classroom attendance: 5 minutes before the start of the class to 15 minutes after the start of the class (covering the students' entry and stable seating). Meeting check-in: 10 minutes before the meeting starts to 5 minutes after the meeting starts (to avoid being disturbed too early or too late); Data acquisition frequency: High-frequency acquisition (e.g., every 10 seconds) can capture sudden changes in state (e.g., students leaving midway), while low-frequency acquisition (e.g., every 2 minutes) can reduce computational overhead. A trade-off must be made between real-time performance and resource consumption.

[0118] Example: During the class time window from 9:00 to 9:30, attendance confidence was collected every 30 seconds, for a total of 61 data points (including the times at 9:00 and 9:30).

[0119] By covering the dynamic changes in student attendance status (such as lateness, early departure, and leaving midway) through multi-time point data, the random errors of a single detection (such as brief false detections in facial recognition) can be avoided.

[0120] Step 132: Calculate a weighted average of the attendance confidence scores of multiple target students to obtain the real-time attendance status of the target students.

[0121] During implementation, a weighted average of multiple attendance confidence levels can be used to obtain the real-time attendance status. Examples of weight values ​​are as follows: Time decay weight: Recent data has a higher weight, reflecting the student's current state (e.g., the closer to the current moment, the greater the weight), which can reduce the interference of historical data on the real-time state (e.g., after a student leaves early, the weight of early high-confidence data decreases).

[0122] Event trigger weighting: Data at the moment of a critical event (such as clocking in, seat detection, successful facial recognition) has higher weight; if a student clocks in at 9:10, the weight for that moment is 0.5. Other moments have a weight of 0.1. This highlights the decisive impact of key behaviors on attendance status.

[0123] Uniform weighting: The weights are the same at all time points, suitable for scenarios with slow state changes (such as long-term stable sitting). It simplifies calculations but has weaker anti-interference capabilities.

[0124] In this embodiment, the problem of "unreliable single detection" in traditional attendance systems is solved by using a structured and quantitative approach, providing a highly reliable comprehensive basis for real-time attendance status determination, and serving as the core of status decision-making for intelligent attendance systems.

[0125] In some embodiments, obtaining the face confidence score in step S110 above can be achieved through the following process: Based on the valid check-in information, it is determined that the target student has made a valid check-in. At the same time, it is determined that the visual trajectory of the target student is missing more than the missing threshold. Multiple frames of facial images are acquired within a preset observation time window to determine the facial confidence based on the multiple frames of facial images.

[0126] During implementation, the attendance records of the target students are obtained from the attendance system (such as card swiping, QR code, NFC, etc.) and the legality of the attendance data is verified (such as the attendance time being within the preset time window, the attendance device ID being bound to the classroom, and the attendance student ID matching the target student). If the check-in data passes verification, it is marked as "valid check-in" and the subsequent face confidence calculation process is triggered. If the check-in data is invalid (such as duplicate check-in, check-in outside the classroom, or abnormal check-in time), it will be directly judged as "invalid check-in" and no face confidence calculation will be performed.

[0127] This avoids interference from false attendance records caused by equipment malfunctions, misoperation, or cheating (such as proxy attendance) with subsequent facial recognition; facial confidence assessment is only performed on students who actually enter the classroom, reducing the waste of computing resources.

[0128] Example: Student 001 used their campus card to swipe at the card reader in classroom 1 at 9:05. The system verified that the swipe time (9:00-9:30), device ID (classroom 1), and student ID (001) were all valid and marked it as "valid swipe". Student 002 checked in at 8:50 in classroom 2. Because the check-in time was outside the current course time window, it was marked as "invalid check-in" and no subsequent process was triggered.

[0129] The location and movement trajectory of target students are continuously tracked using cameras in the classroom (such as panoramic cameras and seat cameras). The percentage of time within a preset observation window (e.g., 5 minutes after attendance) that the target student is not captured by the camera is calculated (i.e., the "visual trajectory missing rate"). If the missing rate is greater than a missing threshold (e.g., 30%), the student is considered to be in an abnormal state such as "occlusion, leaving their seat, or leaving the room," requiring further verification through facial confidence analysis. If the missing rate is less than or equal to the missing threshold, the student is directly considered present (no facial confidence analysis is needed, as the visual trajectory sufficiently proves their presence).

[0130] In this way, if a student's visual trajectory is complete (such as always sitting in their seat), attendance can be confirmed without additional facial recognition; facial confidence calculation is only performed on students with missing visual trajectories, improving system efficiency; and by using the missing rate threshold, "temporary occlusion" (such as looking down while writing) and "long-term absence" (such as leaving early) can be distinguished.

[0131] Starting from the effective check-in time, an observation time window (e.g., 3 minutes after check-in) is set to ensure coverage of dynamic processes such as students sitting down and adjusting their posture. Within the preset observation time window, multiple frames of facial images are collected through classroom cameras (e.g., seat cameras, door cameras) at a fixed frequency (e.g., 1 frame per second) or triggered by events (e.g., a face is detected). Blurry, obstructed, or abnormally angled (e.g., side profile) images are removed, and clear, frontal, and complete facial frames are retained.

[0132] By covering students' different postures and expressions with multiple frames of images, the risk of misrecognition in a single frame (such as false rejection caused by blinking or changes in expression) is reduced; it also adapts to the brief occlusion during students' sitting process (such as organizing their bags or adjusting their seats) or movement (such as walking from the door to their seats).

[0133] For each valid face image frame, feature vectors are extracted using a face recognition model (such as FaceNet or ArcFace) and compared with the face feature database registered by the target student. The single-frame confidence score is output (typically ranging from 0 to 1, with higher values ​​indicating a higher matching degree). The arithmetic mean of all single-frame confidence scores is taken to obtain the overall face confidence score.

[0134] In some embodiments, the confidence scores of a single frame can be weighted according to image quality (such as sharpness, illumination) or time weight (such as recent frames having higher weight), and then summed to obtain the face confidence score.

[0135] In this embodiment, dual verification of attendance validity and visual trajectory loss eliminates interference from non-target students and abnormal states; multi-frame image fusion adapts to students' dynamic behavior, reducing the risk of misidentification in a single frame; and face calculation is performed only in abnormal scenarios (missing visual trajectories), reducing resource consumption. Through structured, multi-layered verification logic, the problems of traditional face attendance systems being "susceptible to occlusion interference and having a high single-frame misidentification rate" are solved, providing highly reliable biometric evidence for attendance status determination.

[0136] This application provides a multi-source fusion algorithm, such as... Figure 2 As shown, this can be achieved through the following steps: Step S210: Write the three types of inputs—visual recognition, seat area detection, and attendance record—into the visual recognition buffer, seat status buffer, and attendance event buffer, respectively, and synchronize their timestamps using a unified time base. During implementation, multimodal evidence from visual recognition, seat area detection, and attendance records is integrated on a unified time scale, and student attendance judgments are output in real time from a continuous data stream. The system first writes the three types of inputs into a visual recognition buffer, a seat status buffer, and an attendance event buffer, respectively, and synchronizes their timestamps using a unified time base. This allows data from different sources to form a structured fusion input set in time slices. This design ensures that multi-source data has a unified, alignable, and correlateable representation before entering the core algorithm.

[0137] Among these, attendance data serves as crucial temporal information for students entering the classroom. First, the system receives raw attendance events from various attendance terminals. Each event carries a student's identity identifier, attendance timestamp, terminal device number, and optional location information. Upon receiving the event, the system immediately writes it into a unified data access module, standardizes the fields, and enters it as a structured record into the "attendance event cache queue," awaiting subsequent verification and synchronization.

[0138] After data enters the cache queue, the system performs validity filtering. First, it generates a "valid check-in time window" for the current class based on the course schedule to determine if the check-in time is within the allowed range; events outside the window are directly discarded. Then, the system uses a "device-classroom mapping table" to verify whether the check-in location matches the classroom for the course, thus eliminating invalid behaviors such as cross-classroom, cross-floor, or remote proxy check-ins. For devices carrying location information, the system further verifies whether their spatial coordinates fall within a reasonable area to enhance data reliability. Records that pass the verification are marked as "valid check-in events" and written to the "valid check-in record pool."

[0139] To ensure that attendance events can be correctly fused with camera video data, the system performs crucial time synchronization after validity verification. The times of all attendance devices, cameras, and edge servers are periodically aligned via NTP service, ensuring a unified reference scale between the "attendance timestamp" and the "video frame timestamp." The synchronized attendance event generates attendance evidence s_punch, which is written to the "fusion input buffer," awaiting integration with visual information for attendance determination.

[0140] Step S220: Using student identity as an index, construct a "fusion state unit" for each student; After the data is aligned temporally, the system will construct a "fusion state unit" for each student, indexed by their identity. This unit serves as a temporally sequenced data container, accumulating elements such as visual confidence, seat matching, and valid attendance evidence generated by the student within the current time slice, and organizing them into a continuous sequence of evidence in chronological order. Through this structured storage method, the system can form a continuous and traceable fusion view of students' behavioral trajectories during class.

[0141] Step S230: Weight the "fusion state unit" according to the reliability of the data source; After the evidence is written into the state unit, the system weights the "fusion state unit" based on the reliability of the data source. The weight of visual evidence is dynamically adjusted by recognition confidence, trajectory stability, and environmental conditions; seating evidence maintains a high baseline confidence due to its fixed nature; and attendance evidence is corrected based on factors such as anomaly markers and time offsets. The weighted multimodal evidence is then comprehensively calculated within the current time slice to form a fusion confidence distribution that reflects the probability of a student's actual presence.

[0142] Building upon single-time-slice fusion, the system further enhances the stability of its judgments through temporal accumulation. For students who consistently appear within consecutive time slices, the system rapidly increases their attendance confidence. For students who have valid attendance records but whose visual trajectories are briefly missing, the system continues to match visual evidence within a set observation time window. If a conflict arises between visual evidence and seat matching, the system activates a conflict resolution mechanism, adjusting or re-evaluating the corresponding weights to avoid misjudgments caused by abnormal behaviors such as proxy attendance or seat switching.

[0143] Step S240: When the accumulated evidence in the fusion state unit meets the corresponding decision threshold, the system outputs the student's real-time attendance status.

[0144] During implementation, once the accumulated evidence in the fusion state unit meets the corresponding decision threshold, the system outputs the student's real-time attendance status, including categories such as "present," "absent," or "abnormal and awaiting review." All judgment results are continuously written to the global attendance result table and updated with each time slice, forming a complete class attendance record.

[0145] This application's embodiments construct a multi-source data fusion framework for real-time attendance scenarios in smart classrooms. It systematically integrates facial video streams, fixed seat number information, and student attendance records acquired in the classroom environment from three dimensions: identity recognition, spatial location constraints, and temporal verification. This improves the accuracy and robustness of the attendance system in complex teaching scenarios. The framework combines real-time recognition, location verification, and identity matching, enabling the system to maintain stable judgments under conditions of varying lighting, complex student postures, and partial occlusion. Furthermore, for common boundary situations such as large-class teaching, multi-angle cameras, and students briefly leaving their seats, this invention introduces a multi-frame fusion and spatial consistency verification mechanism. This elevates attendance recognition from single-frame decision-making to a comprehensive judgment based on multiple sources, multiple temporal sequences, and spatial binding, fundamentally enhancing the system's ability to handle complex situations.

[0146] By integrating facial image features, fixed seat number information, and student attendance records, the system achieves deep fusion and highly reliable identity matching of multi-source heterogeneous data. This effectively solves the problem of inaccurate identification caused by factors such as occlusion, changes in lighting, and students briefly leaving their seats, which are common in traditional single-data-source attendance methods. Through a feature-level multimodal matching mechanism, the system can simultaneously verify identity consistency, location consistency, and attendance validity, thus forming a highly redundant joint verification system that significantly improves the accuracy and robustness of attendance determination.

[0147] This application provides a classroom attendance device that integrates multi-source data. Please refer to [link / reference]. Figure 3 The device includes: The acquisition module 310 is used to acquire the face confidence score, seat occupancy confidence score, and attendance confidence score corresponding to each student identifier within a preset time window. The face confidence score characterizes the matching degree between feature data obtained from the face image and pre-stored feature data; the seat occupancy confidence score characterizes the overlap between the target spatial region obtained from the face image and the pre-stored target seat region; and the attendance confidence score characterizes the probability of a valid attendance check-in. The weighted summation module 320 is used to perform weighted summation on the face recognition confidence, seat occupancy confidence, and attendance confidence of the target student to obtain the attendance confidence of the target student. The determination module 330 is used to determine the real-time attendance status of the target student based on the attendance confidence level of the target student, wherein the real-time attendance status includes at least attending class, absent from class, and abnormal review.

[0148] In some embodiments, the acquisition module includes an identification submodule, a detection submodule, a reconstruction submodule, and a first matching submodule. The identification submodule is used to identify multiple frames of acquired face images using a face recognition model to obtain a first multi-frame identity sequence. The detection submodule is used to perform occlusion probability detection on the first multi-frame identity sequence. The reconstruction submodule is used to reconstruct the target subsequence using a reconstruction model when the occlusion probability of the target subsequence in the first multi-frame identity sequence is greater than a probability threshold, to obtain a second multi-frame identity sequence. The reconstruction model is trained using temporal prediction and neighbor frame compensation mechanisms. The first matching module is used to perform similarity matching between the second multi-frame identity sequence and a pre-stored identity sequence to obtain the face recognition confidence score.

[0149] In some embodiments, the apparatus further includes an adjustment module for freezing the parameters of the face recognition model during model training and adjusting the parameters of the reconstruction model using posture features, facing direction features, and swing range features, so that the reconstruction model can reconstruct the identity sequence using temporal prediction and neighbor frame compensation mechanisms.

[0150] In some embodiments, the face recognition model adopts a bidirectional multi-scale fusion structure; the recognition submodule is further used to utilize the face recognition model with the bidirectional multi-scale fusion structure to obtain a first multi-frame identity sequence with cross-scale semantic enhancement and spatial detail compensation through bidirectional information flow from top to bottom and from bottom to top during the recognition of the multi-frame face images.

[0151] In some embodiments, the acquisition module includes a first acquisition submodule, a mapping submodule, and a first determination submodule, wherein the first acquisition submodule is used to acquire a seat area template generated from classroom layout information, the seat area template representing the mapping relationship between each student identifier and a seat area; the mapping submodule is used to perform perspective correction and spatial mapping on the acquired classroom image to obtain the spatial area of ​​each student identifier; the first determination submodule is used to determine the seat occupancy confidence of seat matching based on the spatial area of ​​each student and the seat area model.

[0152] In some embodiments, the first determining submodule includes an acquisition unit, a first determining unit, a second determining unit, and a third determining unit, wherein the acquisition unit is used to acquire the target spatial region of the target student identifier; the first determining unit is used to determine the target seat region of the target student based on the target student identifier and the seat region template; the second determining unit is used to determine the intersection region of the target spatial region and the target coordinate region; and the third determining unit is used to determine the ratio of the intersection region to the target coordinate region as the seat occupancy confidence level.

[0153] In some embodiments, the acquisition module includes a second acquisition submodule, a second determination submodule, a third determination submodule, and a fourth determination submodule. The second acquisition submodule is used to acquire attendance record data, wherein the attendance record data includes at least a student identifier, an attendance timestamp, and location information. The second determination submodule is used to determine a first sub-confidence level based on the matching degree between the attendance timestamp and the student identifier corresponding to a preset time period. The third determination submodule is used to determine a second sub-confidence level based on the matching degree between the location information and the student identifier corresponding to a preset location. The fourth determination submodule is used to perform a weighted summation of the first sub-confidence level and the second sub-confidence level to obtain the attendance confidence level.

[0154] In some embodiments, the weighted summation module includes a fourth determining submodule, a fifth determining submodule, a sixth determining submodule, and a weighted summation submodule, wherein the fourth determining submodule is used for a first weight parameter; the fifth determining submodule is used for determining a second weight parameter of the seat occupancy confidence based on a fixed probability of seat occupancy; the sixth determining submodule is used for determining a third weight parameter of the attendance confidence based on the abnormal marker and time offset of the attendance; and the weighted summation submodule is used for weighting the face recognition confidence, seat occupancy confidence, and attendance confidence of the target student identifier based on the first weight parameter, the second weight parameter, and the third weight parameter.

[0155] In some embodiments, the determining module includes a third acquisition submodule and a weighted average submodule, wherein the third acquisition submodule is used to acquire multiple attendance confidence scores of the target students at different time points within the preset time window; the weighted average submodule is used to perform a weighted average of the multiple attendance confidence scores of the target students to obtain the real-time attendance status of the target students.

[0156] In some embodiments, the acquisition module is further configured to determine that the target student has made a valid check-in based on the valid check-in information, and at the same time determine that the visual trajectory of the target student is missing more than a missing threshold, and acquire multiple frames of face images within a preset observation time window, so as to determine the face confidence based on the multiple frames of face images.

[0157] Figure 4 This is a schematic diagram of the structure of a computer block device provided in an embodiment of this application. For example, as shown... Figure 4As shown, the computer block device 400 includes: a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program 403, the computer block device can execute any of the multi-source data fusion classroom attendance methods described above.

[0158] Furthermore, this application also protects a control block device, which may include a memory and a processor. The memory stores executable program code, and the processor is used to call and execute the executable program code to perform a multi-source data fusion classroom attendance method provided in this application. This application can divide the control block device into functional modules based on the above method examples. For example, each module can correspond to a specific function, or two or more functions can be integrated into a processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this application is illustrative and only represents a logical functional division; other division methods may exist in actual implementation. It should also be noted that all relevant content of each step involved in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here. It should be understood that the control block device provided in this application is used to execute the above-mentioned multi-source data fusion classroom attendance method, and therefore can achieve the same effect as the above-mentioned implementation method. When using integrated units, the control block device may include a processing module and a storage module. When the control block device is applied to a block device, the processing module can be used to control and manage the actions of the block device. The storage module can be used to support the block device in executing mutual program code, etc. The processing module can be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of Digital Signal Processing (DSP) and a microprocessor, etc., and the storage module can be a memory.

[0159] Furthermore, the control block device provided in the embodiments of this application may specifically be a chip, component, or module. The chip may include a connected processor and a memory. The memory stores instructions, and when the processor calls and executes the instructions, the chip can execute the multi-source data fusion classroom attendance method provided in the above embodiments. The embodiments of this application also provide a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, it causes the computer to execute the aforementioned method steps to implement the multi-source data fusion classroom attendance method provided in the above embodiments.

[0160] This application also provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the aforementioned related steps to implement the multi-source data fusion classroom attendance method provided in the above embodiments. The control block device, computer-readable storage medium, computer program product, or chip provided in this application embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they achieve can be referred to in the beneficial effects of the corresponding methods provided above, and will not be repeated here. Through the description of the above embodiments, those skilled in the art can understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the control block device can be divided into different functional modules to complete all or part of the functions described above. In the embodiments provided in this application, it should be understood that the disclosed control block device and method can be implemented in other ways. For example, the control block device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another control block device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between control block devices or units may be electrical, mechanical, or other forms.

[0161] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multiple task processing and parallel processing are possible or may be advantageous. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. The above content is only a specific implementation of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A classroom attendance method based on multi-source data fusion, characterized in that, The method includes: Within a preset time window, the face confidence score, seat occupancy confidence score, and attendance confidence score corresponding to each student identifier are obtained. The face confidence score is used to characterize the matching degree between the feature data obtained based on the face image and the pre-stored feature data; the seat occupancy confidence score is used to characterize the overlap between the target spatial region obtained based on the face image and the pre-stored target seat region; and the attendance confidence score is used to characterize the probability of an effective attendance check-in. The attendance confidence of the target student is obtained by weighted summing of the confidence scores of facial recognition, seat occupancy, and attendance check-in. The real-time attendance status of the target student is determined based on the attendance confidence level of the target student, wherein the real-time attendance status includes at least attendance, absence, and abnormality review.

2. The method as described in claim 1, characterized in that, The step of obtaining the facial recognition confidence score corresponding to each student identifier includes: The first multi-frame identity sequence is obtained by using a face recognition model to identify multiple frames of face images collected. Perform occlusion probability detection on the first multi-frame identity sequence; If the occlusion probability of the target subsequence in the first multi-frame identity sequence is determined to be greater than the probability threshold, the target subsequence is reconstructed using a reconstruction model to obtain the second multi-frame identity sequence. The reconstruction model is trained using a temporal prediction and neighboring frame compensation mechanism. The face recognition confidence score is obtained by performing similarity matching between the second multi-frame identity sequence and the pre-stored identity sequence.

3. The method as described in claim 2, characterized in that, The method further includes: During model training, the parameters of the face recognition model are frozen, and the parameters of the reconstruction model are adjusted using posture features, facing direction features, and swing range features, so that the reconstruction model can reconstruct the identity sequence using temporal prediction and neighbor frame compensation mechanisms.

4. The method as described in claim 2, characterized in that, The face recognition model adopts a two-way multi-scale fusion structure; The step of using a face recognition model to identify multiple frames of face images to obtain a first multi-frame identity sequence includes: Using the aforementioned bidirectional multi-scale fusion structure, the face recognition model obtains the first multi-frame identity sequence with cross-scale semantic enhancement and spatial detail compensation through bidirectional information flow from top to bottom and bottom to top during the recognition of the multi-frame face images.

5. The method as described in claim 1, characterized in that, Obtaining the seat occupancy confidence includes: A seating area template is generated by obtaining classroom layout information, wherein the seating area template represents the mapping relationship between each student identifier and the seating area; Perspective correction and spatial mapping are performed on the collected classroom images to obtain the spatial area for each student's identification. The seat occupancy confidence level for seat matching is determined based on the spatial region of each student and the seat region model.

6. The method as described in claim 5, characterized in that, The determination of the seat occupancy confidence score based on the spatial region of each student and the seat region model includes: Obtain the target spatial region of the target student identifier; Based on the target student identifier and the seat area template, the target seat area for the target student is determined; Determine the intersection region of the target spatial region and the target coordinate region; The ratio of the intersection region to the target coordinate region is determined as the seat occupancy confidence level.

7. The method as described in claim 1, characterized in that, The process of obtaining the attendance confidence level includes: Obtain attendance record data, wherein the attendance record data includes at least student identifier, attendance timestamp, and location information; The first sub-confidence level is determined based on the matching degree between the check-in timestamp and the student identifier corresponding to the preset time period; The second sub-confidence level is determined based on the matching degree between the location information and the preset location corresponding to the student identifier; The first sub-confidence and the second sub-confidence are weighted and summed to obtain the attendance confidence.

8. The method according to any one of claims 1 to 7, characterized in that, The attendance confidence score of the target student is obtained by weighted summation of the facial recognition confidence score, seat occupancy confidence score, and attendance check-in confidence score, including: The first weighting parameter for the face recognition confidence is determined based on the recognition confidence, trajectory stability, and environmental conditions. A second weighting parameter is determined based on a fixed probability of seat occupancy to determine the confidence level of seat occupancy; The third weighting parameter for the confidence level of the check-in is determined based on the anomaly marker and time offset of the check-in. The confidence scores of the target student's face recognition, seat occupancy, and attendance are weighted based on the first weight parameter, the second weight parameter, and the third weight parameter to obtain the attendance confidence score of the target student.

9. The method according to any one of claims 1 to 7, characterized in that, Determining the real-time attendance status of the target student based on the target student's attendance confidence score includes: Multiple attendance confidence scores of the target students are obtained at different time points within the preset time window; The real-time attendance status of the target students is obtained by weighted averaging of the attendance confidence scores of multiple target students.

10. The method according to any one of claims 1 to 7, characterized in that, Obtaining the face confidence score includes: Based on the valid check-in information, it is determined that the target student has made a valid check-in. At the same time, it is determined that the visual trajectory of the target student is missing more than the missing threshold. Multiple frames of facial images are acquired within a preset observation time window to determine the facial confidence based on the multiple frames of facial images.

Images