A method for attendance tracking based on video context analysis

Abstract

This invention relates to the technical field of attendance tracking and discloses an attendance tracking method based on video context analysis. The method includes: adaptive sampling and video frame preprocessing of the surveillance video; performing individual face recognition on the preprocessed surveillance video frames using a face recognition model; matching the recognized individual faces with the face information of absent students in a student attendance database to construct behavioral trajectory data of the absent students; measuring the similarity between the behavioral trajectory data and an attendance tracking template; and automatically tracking attendance if the similarity measurement result exceeds a preset trajectory similarity threshold. This invention uses color similarity between color feature vectors to set an adaptive sampling frequency for video segments, improving the sampling efficiency of the surveillance video. It constructs behavioral trajectory data of absent students based on face recognition, builds an attendance tracking template based on preset attendance tracking rules, and performs similarity measurement to achieve automated attendance tracking.

Description

Technical Field

[0001] This invention relates to the technical field of attendance verification, and more particularly to an attendance verification method based on video context analysis. Background Technology

[0002] With increasingly stringent requirements for student attendance management in schools, traditional manual attendance methods can no longer meet the demands for efficiency and accuracy. Especially in special circumstances, such as students being followed or having their faces obscured, traditional methods often rely on manual verification and make-up signatures. Manual sign-in requires teachers or administrators to record each student's attendance, which is time-consuming and inefficient. Students may also sign on behalf of others or impersonate others, leading to inaccurate attendance data. Traditional methods are also inconvenient for handling temporary make-up signatures or special situations (such as lateness or forgetting to clock in), usually requiring manual intervention and increasing management costs. Meanwhile, with the development of information technology, intelligent attendance systems are gradually being introduced into school settings, using technologies such as facial recognition and fingerprint recognition for identity verification to ensure the authenticity and accuracy of attendance data. Summary of the Invention

[0003] In view of this, the present invention provides an attendance correction method based on video scenario analysis. By combining a video surveillance system with artificial intelligence technology, the method dynamically analyzes the student activities in the video to verify the accuracy of attendance records and achieve automated attendance correction.

[0004] To achieve the above objectives, this invention provides a method for attendance tracking based on video context analysis, comprising the following steps:

[0005] S1: Collect surveillance videos in different scenarios, perform adaptive sampling and video frame preprocessing on the surveillance videos to obtain preprocessed surveillance video frames.

[0006] S2: Use a face recognition model to perform individual face recognition on the preprocessed monitoring video frames, and match the recognized individual faces with the face information of students who did not attend the class in the student attendance database.

[0007] S3: Record the timestamp information and monitoring scene information of the preprocessed monitoring video frames associated with the successfully matched individual faces to form the behavioral trajectory data of the absent students. The monitoring scene information is the location of the monitoring camera associated with the preprocessed monitoring video frames.

[0008] S4: Measure the similarity between the behavioral trajectory data and the attendance make-up trajectory template. If the similarity measurement result exceeds the preset trajectory similarity threshold, the student who did not attend the class meets the attendance make-up rule and will be automatically made up in the student attendance database.

[0009] As a further improvement of the present invention:

[0010] Optionally, step S1 involves acquiring surveillance videos under different scenarios and performing adaptive sampling processing on the surveillance videos, including:

[0011] Surveillance cameras are deployed in different scenarios to collect surveillance videos. These scenarios include corridors outside classrooms, canteens, playgrounds, outside dormitories, school gates, and internal traffic roads.

[0012] The surveillance video is divided into multiple video segments of equal duration, and continuous video frames are extracted from the video segments. Based on the frequency distribution of pixels in the video frames, the color feature vector of the video frames is calculated.

[0013] Calculate the color similarity of color feature vectors of consecutive video frames, and convert the color similarity into the sampling frequency of video segments;

[0014] The video segments are sampled based on their sampling frequency, and the sampling results are used as monitoring video frames in the video segments; the monitoring video frames in all video segments are used to form a monitoring video frame sequence of the monitoring video associated with the video segment.

[0015] Optionally, the monitoring video frames in the monitoring video frame sequence are preprocessed to obtain preprocessed monitoring video frames, including:

[0016] The preprocessing method includes grayscale processing and gradient feature extraction processing. The grayscale processing method calculates the grayscale value of the pixels in the monitoring video frame and constructs a grayscale pixel matrix of the monitoring video frame using the grayscale values ​​of the pixels. The gradient feature extraction processing method calculates the gradient value of the pixels based on the grayscale values ​​of the pixels, uses non-maximum suppression and dual threshold detection algorithms in the Canny edge detection algorithm to detect the edge pixels of the monitoring video frame, and marks the edge pixels in the grayscale pixel matrix of the monitoring video frame. The grayscale pixel matrix with marked edge pixels is used as the preprocessed monitoring video frame.

[0017] Optionally, step S2 involves using a face recognition model to perform individual face recognition on the preprocessed monitoring video frames, including:

[0018] The face recognition model includes an input layer, three consecutive spatial channel cross-attention units, and a head layer;

[0019] The input layer is used to receive preprocessed monitoring video frames and perform convolution processing on the preprocessed monitoring video frames to obtain the convolutional feature maps of the preprocessed monitoring video frames.

[0020] The spatial channel cross attention unit is used to perform attention processing on the feature map output by the previous layer and output the attention-processed feature map. The feature map output by the third spatial channel cross attention unit is the attention feature map of the preprocessed monitoring video frame. The attention processing includes channel attention processing and spatial attention processing.

[0021] The Head layer includes a classification branch and a bounding box regression branch. It is used to divide the attention feature map into multiple target region feature maps and generate bounding box parameters for each target region feature map. The bounding box parameters include the center coordinates, width offset, and length offset of the bounding box. It also calculates the face region confidence score of the target region feature map. Combined with the edge pixels of the preprocessed monitoring video frame corresponding to the bounding box parameters, the face region confidence score is corrected. Bounding box parameters with corrected face region confidence scores higher than a preset threshold are retained. The retained bounding box parameters are used to generate bounding boxes in the preprocessed monitoring video frame. The image region enclosed by the bounding box is regarded as the individual face in the preprocessed monitoring video frame.

[0022] Optionally, matching the individual's face with the facial information of students who did not attend classes in the student attendance database includes:

[0023] The system acquires the face images of students who did not attend class from the student attendance database. It then uses three consecutive spatial channels to cross attention units to generate attention feature maps of the individual face and the student face images. The similarity of the attention feature maps is calculated. If the similarity calculation result is higher than a preset similarity threshold, it means that the face information of the student who did not attend class and the identified individual face are successfully matched.

[0024] Optionally, the behavioral trajectory data constituting the absent student in step S3 includes:

[0025] The timestamp information of the preprocessed surveillance video frames associated with the successfully matched individual faces, along with the surveillance scene information, are used as a set of behavioral information for the absent students. This behavioral information is then sorted according to the timestamp information to obtain the behavioral trajectory data x of the absent students.

[0026] x=(x1,x2,...,x m ,...,x M );

[0027] ;

[0028] in:

[0029] x1,x2,...,x m ,...,x M This represents the M groups of behavioral information in the behavioral trajectory data x, where x is...m This represents the m-th group of behavioral information in the behavioral trajectory data x. The behavioral information x are listed in order. m The timestamp information and monitoring scene information in the data, m∈[1,M], where M represents the number of behavioral information groups in the behavioral trajectory data x.

[0030] Optionally, step S4 involves measuring the similarity between the behavioral trajectory data and the attendance check-in template, including:

[0031] Based on preset attendance check-in rules, two attendance check-in trajectory templates are constructed, each corresponding to trajectory data from the school's teaching building to the dormitory. L 1. Attendance tracking data from the school playground to the dormitory L 2. Each attendance trajectory data contains G trajectory points, and each trajectory point contains attendance timestamp information and trajectory location information;

[0032] The similarity measurement process between the behavioral trajectory data x and the j-th type of attendance make-up trajectory template is as follows:

[0033] Obtain the trajectory data corresponding to the attendance check-in template. L j :

[0034] ;

[0035] in:

[0036] j ∈{1,2};

[0037] Representing trajectory data L j G trajectory points in the middle, , Representing trajectory data L j The Middle g Attendance timestamp information and trajectory location information for each trajectory point. g ∈[1, G ];

[0038] Constructing trajectory data L j The cumulative distance matrix Q between the trajectory and the behavior trajectory data x j :

[0039] ;

[0040] ;

[0041] in:

[0042] Represents behavioral information x m With trajectory points The distance between the trajectories, Q j [ g ][ m ] represents behavioral information x m With trajectory points Cumulative distance between trajectories;

[0043] Extracting the trajectory distance cumulative matrix Q j Q in j [ G ][ M The similarity measure Sim(x) between the behavioral trajectory data x and the attendance check-in trajectory template is calculated:

[0044] ;

[0045] in:

[0046] σ represents the trajectory distance control parameter;

[0047] Indicates selection The maximum value in.

[0048] Optionally, the automatic attendance record-keeping in the student attendance database includes:

[0049] The student attendance database judges the similarity measurement results. If the similarity measurement results exceed the preset trajectory similarity threshold, the absent student meets the attendance make-up rules. The absent student is automatically made up in the student attendance database, and the behavioral trajectory data of the absent student and the preprocessed monitoring video frames associated with the behavioral trajectory data are recorded.

[0050] To address the above problems, the present invention provides an electronic device, the electronic device comprising:

[0051] Memory, storing at least one instruction;

[0052] Communication interfaces enable communication between electronic devices; and

[0053] The processor executes the instructions stored in the memory to implement the attendance tracking method based on video scenario analysis described above.

[0054] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned video-based scene analysis-based attendance tracking method.

[0055] Compared with existing technologies, this invention proposes a video-based scenario analysis-based attendance tracking method, which has the following advantages:

[0056] First, this scheme proposes an adaptive video sampling processing method and a face recognition method. The surveillance video is divided into multiple video segments, and color feature vectors of the video frames are extracted. Based on the color similarity between the color feature vectors, an adaptive sampling frequency is set for each video segment. The higher the color similarity of the color feature vectors of consecutive video frames, the lower the sampling frequency of the video segment, avoiding sampling too many duplicate video frames and improving the sampling efficiency of the surveillance video. Based on the pixel grayscale information and gradient edge information of the surveillance video frames, a face recognition model is constructed for individual face recognition. During the face recognition process, channel attention and spatial attention are cross-fused to focus on the spatial location, color features, and edge features of the face target in the surveillance video frame, thereby enhancing these features and improving the accuracy of face target recognition. The learning rate is adaptively updated during training. In the first stage, the learning rate gradually increases from a minimum to a maximum value. In the early stages of training, the learning rate is gradually increased to avoid gradient explosion, training instability, and premature entry into local optima. In the later stages of training, the learning rate is gradually decreased to improve the model's convergence accuracy and stability.

[0057] Meanwhile, this solution proposes an automated attendance make-up method. It matches the identified individual faces with the facial information of absent students in the student attendance database to form the behavioral trajectory data of the absent students. Based on the preset attendance make-up rules, an attendance make-up trajectory template is constructed. Combining location distance and timestamp distance, the similarity between the behavioral trajectory data and the attendance make-up trajectory template is measured. If the similarity measurement result exceeds the preset trajectory similarity threshold, the absent student meets the attendance make-up rules and is automatically made up in the student attendance database, thus realizing the automated make-up of attendance for absent students. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating an attendance tracking method based on video context analysis, as provided in an embodiment of the present invention.

[0059] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] This application provides a method for attendance verification based on video context analysis. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0062] Reference Figure 1 Embodiment 1 of the present invention is as follows:

[0063] A method for attendance tracking based on video context analysis includes the following steps:

[0064] S1: Collect surveillance videos from different scenarios, perform adaptive sampling and video frame preprocessing on the surveillance videos to obtain preprocessed surveillance video frames.

[0065] Step S1 involves acquiring surveillance videos from different scenarios and performing adaptive sampling processing on the surveillance videos, including:

[0066] Surveillance cameras are deployed in different scenarios to collect surveillance videos. These scenarios include corridors outside classrooms, canteens, playgrounds, outside dormitories, school gates, and internal traffic roads.

[0067] The surveillance video is divided into multiple video segments of equal duration, and continuous video frames are extracted from the video segments. Based on the frequency distribution of pixels in the video frames, the color feature vector of the video frames is calculated.

[0068] Calculate the color similarity of color feature vectors of consecutive video frames, and convert the color similarity into the sampling frequency of the video segment; as a preferred embodiment of the present invention, the formula for calculating the sampling frequency of the video segment is:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] .

[0074] in:

[0075] f represents the sampling frequency of the video segment, f0 represents the base sampling frequency, β represents the sampling adjustment coefficient, μ represents the color similarity conversion coefficient of consecutive video frames in the video segment, Sim(n,n+1) represents the color similarity between the color feature vectors of the nth video frame and the (n+1)th video frame in the video segment, and V n V n+1 Let represent the color feature vectors of the nth and (n+1)th video frames in the video segment, respectively. Represents the L2 norm; Sim max Sim min These are the preset maximum and minimum color similarity;

[0076] This represents the color vector of the nth video frame in the RGB color channels, where T represents the transpose.

[0077] This represents the color value distribution of the nth video frame in the i-th color channel. This represents the percentage of pixels in the nth video frame where the color value of the i-th color channel is 255, where i includes the R, G, and B color channels.

[0078] The sampling frequency of the video segment is the number of video frames sampled per second in the video segment. The higher the color similarity of the color feature vectors of consecutive video frames, the lower the sampling frequency of the video segment.

[0079] The video segments are sampled based on their sampling frequency, and the sampling results are used as monitoring video frames in the video segments; the monitoring video frames in all video segments are used to form a monitoring video frame sequence of the monitoring video associated with the video segment.

[0080] Preprocessing the monitoring video frames in the monitoring video frame sequence to obtain preprocessed monitoring video frames includes:

[0081] The preprocessing method includes grayscale processing and gradient feature extraction processing. The grayscale processing involves calculating the grayscale values ​​of pixels in the surveillance video frame and constructing a grayscale pixel matrix for the surveillance video frame. The gradient feature extraction processing involves calculating the gradient values ​​of pixels based on their grayscale values, and using non-maximum suppression and dual-threshold detection algorithms from the Canny edge detection algorithm to detect edge pixels in the surveillance video frame. These edge pixels are then marked in the grayscale pixel matrix of the surveillance video frame, and the grayscale pixel matrix with marked edge pixels is used as the preprocessed surveillance video frame. Specifically, the dual-threshold detection algorithm identifies strong and weak edge pixels. If a weak edge pixel is connected to at least one strong edge pixel by an 8-neighborhood, it is retained as an edge pixel; otherwise, the weak edge pixel is discarded.

[0082] S2: Use a face recognition model to perform individual face recognition on the preprocessed monitoring video frames, and match the recognized individual faces with the face information of students who did not attend the class in the student attendance database.

[0083] Step S2 involves using a face recognition model to perform individual face recognition on the preprocessed surveillance video frames, including:

[0084] The face recognition model includes an input layer, three consecutive spatial channel cross-attention units, and a head layer;

[0085] The input layer is used to receive preprocessed monitoring video frames and perform convolution processing on the preprocessed monitoring video frames to obtain the convolutional feature maps of the preprocessed monitoring video frames.

[0086] The spatial channel cross attention unit is used to perform attention processing on the feature map output from the previous layer and output the attention-processed feature map. The feature map output by the third spatial channel cross attention unit is the attention feature map of the preprocessed monitoring video frame. The attention processing includes channel attention processing and spatial attention processing.

[0087] As an embodiment of the present invention, by cross-fusion of channel attention and spatial attention, the spatial location, color features, and edge features of facial targets in the surveillance video frame are considered, and then these features are enhanced to improve the accuracy of facial target recognition. The channel attention processing flow is as follows:

[0088] The feature map is nonlinearly processed using an activation function, and the result is batch normalized. Average pooling and max pooling operations are used to generate two spatial attention vectors from the batch normalized result. A multilayer perceptron is then used to perform multi-channel mapping on these two spatial attention vectors to obtain the channel attention of the feature map. The formula for calculating the channel attention is as follows:

[0089] ;

[0090] .

[0091] in:

[0092] F This represents the feature map received by the spatial channel cross-attention unit. BN (·) indicates batch normalization. ReLU (·) Sigmoid (·) is the activation function. MLP (·) represents a multilayer perceptron. AvgPool (·) indicates the average pooling operation. MaxPool (·) indicates a max pooling operation;

[0093] This represents the batch normalization result of feature map F;

[0094] S 1( F ) represents the channel attention of feature map F.

[0095] The spatial attention processing flow is as follows:

[0096] Two spatial attention vectors are extracted from the channel attention and merged. The merged result is then subjected to convolution to obtain the spatial attention of the feature map. The formula for calculating the spatial attention is as follows:

[0097] .

[0098] in:

[0099] S 2( F ) represents spatial attention to feature map F;

[0100] Conv 5×5 (·) indicates a convolution operation using a 5x5 convolution kernel;

[0101] The feature map after attention processing is:

[0102] .

[0103] in:

[0104] F * indicates the attention processing result of feature map F;

[0105] This indicates element-wise multiplication;

[0106] The Head layer includes a classification branch and a bounding box regression branch, used to divide the attention feature map into multiple target region feature maps and generate bounding box parameters for each target region feature map. These bounding box parameters include the center coordinates, width offset, and length offset of the bounding box. The face region confidence score of the target region feature map is calculated. Combined with the edge pixels of the preprocessed monitoring video frame corresponding to the bounding box parameters, the face region confidence score is corrected. Bounding box parameters with corrected face region confidence scores higher than a preset threshold are retained. These retained bounding box parameters are used to generate bounding boxes in the preprocessed monitoring video frame, and the image region enclosed by the bounding box is considered the individual face in the preprocessed monitoring video frame. In a preferred embodiment of the invention, the face region confidence score is corrected based on the curvature of the edge pixels of the image region enclosed by the bounding box. A lower curvature indicates that the region enclosed by the edge pixels in the image region is circular or elliptical, and this image region is more likely to be a face region. The formula for correcting the face region confidence score is:

[0107] .

[0108] in:

[0109] Smooth This represents the confidence score for the face region. Smooth It represents the average curvature of the edge pixels in the closed edge pixel contour with the largest area in the preprocessed video frame image region enclosed by the bounding box corresponding to the face region confidence score, where the closed edge pixel contour is the curve obtained by connecting consecutive edge pixels;

[0110] Indicates selection The maximum value in;

[0111] This indicates the preset curvature threshold.

[0112] As an embodiment of the present invention, a face recognition model can be trained using a loss function, the expression of which is Loss:

[0113] ;

[0114] .

[0115] in:

[0116] b h,c This represents the c-th true bounding box parameter of the h-th training sample. Let H represent the c-th predicted bounding box parameter of the h-th training sample, where H represents the number of training samples; the first to fourth bounding box parameters are the center x-coordinate, center y-coordinate, width offset, and length offset, respectively.

[0117] U(·) represents the smoothing function.

[0118] The true bounding box parameters are the bounding box parameters of the manually labeled face regions in the training samples, and the predicted bounding box parameters are the bounding box parameters of individual faces in the training samples predicted by the face recognition model.

[0119] Specifically, the training algorithm can employ one or more combinations of ADAM optimization algorithm, Newton's iteration method, and gradient descent method. As a preferred embodiment of this invention, the learning rate is adaptively updated during training. In the first stage, the learning rate gradually increases from its minimum to its maximum value. In the early stages of training, the learning rate is gradually increased to avoid gradient explosion, training instability, and premature entry into local optima. In the later stages of training, the learning rate is gradually decreased to improve the model's convergence accuracy and stability. The formula for updating the learning rate is:

[0120] .

[0121] in:

[0122] α t This represents the learning rate of the model parameters in the t-th training iteration, and Max represents the number of training iterations for the model parameters. α max The preset maximum learning rate, α min This represents the preset minimum learning rate. This indicates the iteration step size in the first stage.

[0123] Matching the individual's face with the facial information of students who did not attend class in the student attendance database includes:

[0124] The system acquires facial images of absent students from the student attendance database. It then generates attention feature maps of the individual student's face and the student's face image using three consecutive spatial channel cross-attention units. The similarity of these attention feature maps is calculated. If the similarity score is higher than a preset similarity threshold, it indicates a successful match between the absent student's face and the identified individual student's face. The similarity of the attention feature maps is calculated using cosine similarity.

[0125] S3: Record the timestamp information of the preprocessed monitoring video frames associated with the successfully matched individual faces, as well as the monitoring scene information, to form the behavioral trajectory data of students who did not attend class.

[0126] The behavioral trajectory data of the absent students in step S3 includes:

[0127] The timestamp information of the preprocessed surveillance video frames associated with the successfully matched individual faces, along with the surveillance scene information, are used as a set of behavioral information for the absent students. This behavioral information is then sorted according to the timestamp information to obtain the behavioral trajectory data x of the absent students.

[0128] x=(x1,x2,...,x m ,...,x M );

[0129] ;

[0130] in:

[0131] x1,x2,...,x m ,...,x M This represents the M groups of behavioral information in the behavioral trajectory data x, where x is... m This represents the m-th group of behavioral information in the behavioral trajectory data x. The behavioral information x are listed in order. m The timestamp information and monitoring scene information in the data, m∈[1,M], where M represents the number of behavioral information groups in the behavioral trajectory data x.

[0132] S4: Measure the similarity between the behavioral trajectory data and the attendance make-up trajectory template. If the similarity measurement result exceeds the preset trajectory similarity threshold, the student who did not attend the class meets the attendance make-up rule and will be automatically made up in the student attendance database.

[0133] Step S4 involves measuring the similarity between the behavioral trajectory data and the attendance check-in template, including:

[0134] Based on preset attendance check-in rules, two attendance check-in trajectory templates are constructed, each corresponding to trajectory data from the school's teaching building to the dormitory. L 1. Attendance tracking data from the school playground to the dormitory L 2. Each attendance trajectory data contains G trajectory points, and each trajectory point contains attendance timestamp information and trajectory location information;

[0135] The similarity measurement process between the behavioral trajectory data x and the j-th type of attendance make-up trajectory template is as follows:

[0136] Obtain the trajectory data corresponding to the attendance check-in template. L j :

[0137] .

[0138] in:

[0139] j ∈{1,2};

[0140] Representing trajectory data L j G trajectory points in the middle, , Representing trajectory data L j The Middle g Attendance timestamp information and trajectory location information for each trajectory point. g ∈[1, G ];

[0141] Constructing trajectory data L j The cumulative distance matrix Q between the trajectory and the behavior trajectory data x j :

[0142] ;

[0143] ;

[0144] .

[0145] in:

[0146] Indicates monitoring scene information With trajectory location information The Euclidean distance between them time Indicates the timestamp adjustment value;

[0147] Represents behavioral information x m With trajectory points The distance between the trajectories, Q j [ g ][ m ] represents behavioral information x m With trajectory points The cumulative value of the trajectory distance between them.

[0148] Extracting the trajectory distance cumulative matrix Q j Q in j [ G ][ M The similarity measure Sim(x) between the behavioral trajectory data x and the attendance check-in trajectory template is calculated:

[0149] ;

[0150] in:

[0151] σ represents the trajectory distance control parameter;

[0152] Indicates selection The maximum value in.

[0153] The automatic attendance record-keeping process in the student attendance database includes:

[0154] The student attendance database assesses the similarity measurement results. If the similarity measurement result exceeds a preset trajectory similarity threshold, the absent student meets the attendance make-up rules. The absent student is automatically made up in the student attendance database, and the student's behavioral trajectory data, along with the associated pre-processed surveillance video frames, are recorded. Specifically, administrators can verify the authenticity of the make-up attendance by reviewing the behavioral trajectory data and the associated pre-processed surveillance video frames.

[0155] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0156] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0158] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for attendance tracking and make-up appointments based on video context analysis, characterized in that, The method includes: S1: Collect surveillance videos in different scenarios, perform adaptive sampling and video frame preprocessing on the surveillance videos to obtain preprocessed surveillance video frames. S2: Use a face recognition model to perform individual face recognition on the preprocessed monitoring video frames, and match the recognized individual faces with the face information of students who did not attend the class in the student attendance database. S3: Record the timestamp information and monitoring scene information of the preprocessed monitoring video frames associated with the successfully matched individual faces to form the behavioral trajectory data of the absent students. The monitoring scene information is the location of the monitoring camera associated with the preprocessed monitoring video frames. S4: Measure the similarity between the behavioral trajectory data and the attendance make-up trajectory template. If the similarity measurement result exceeds the preset trajectory similarity threshold, the student who did not attend the class meets the attendance make-up rule and will be automatically made up in the student attendance database. Step S4 involves measuring the similarity between the behavioral trajectory data and the attendance check-in template, including: Based on the preset attendance check-in rules, two attendance check-in trajectory templates are constructed. The two attendance check-in trajectory templates correspond to the trajectory data L1 from the school teaching building to the dormitory and the attendance trajectory data L2 from the school playground to the dormitory, respectively. Each attendance trajectory data has G trajectory points, and each trajectory point contains attendance timestamp information and trajectory location information. The similarity measurement process between the behavioral trajectory data x and the j-th type of attendance make-up trajectory template is as follows: Obtain the trajectory data L corresponding to the attendance check-in template. j : in: j∈{1,2}； Represents trajectory data L j G trajectory points in the middle, Representing trajectory data L respectively j The attendance timestamp information and trajectory location information of the g-th trajectory point in the middle, where g∈[1,G]; Constructing trajectory data L j The cumulative distance matrix Q between the trajectory and the behavior trajectory data x j : Q j ={Q j [g][m]|m∈[1,M],g∈[1,G]}; in: Represents behavioral information x m With trajectory points The distance between the trajectories, Q j [g][m] represents behavioral information x m With trajectory points Cumulative distance between trajectories; Extracting the trajectory distance cumulative matrix Q j Q in j [G][M], calculate the similarity measurement result Sim(x) between the behavioral trajectory data x and the attendance check-in trajectory template: in: σ represents the trajectory distance control parameter; Indicates selection The maximum value in.

2. The attendance tracking method based on video context analysis as described in claim 1, characterized in that, Step S1 involves acquiring surveillance videos from different scenarios and performing adaptive sampling processing on the surveillance videos, including: Surveillance cameras are deployed in different scenarios to collect surveillance videos. These scenarios include corridors outside classrooms, canteens, playgrounds, outside dormitories, school gates, and internal traffic roads. The surveillance video is divided into multiple video segments of equal duration, and continuous video frames are extracted from the video segments. Based on the frequency distribution of pixels in the video frames, the color feature vector of the video frames is calculated. Calculate the color similarity of color feature vectors of consecutive video frames, and convert the color similarity into the sampling frequency of video segments; The video segments are sampled based on their sampling frequency, and the sampling results are used as monitoring video frames in the video segments; the monitoring video frames in all video segments are used to form a monitoring video frame sequence of the monitoring video associated with the video segment.

3. The attendance tracking method based on video context analysis as described in claim 2, characterized in that, Preprocessing the monitoring video frames in the monitoring video frame sequence to obtain preprocessed monitoring video frames includes: The preprocessing method includes grayscale processing and gradient feature extraction processing. The grayscale processing method calculates the grayscale value of the pixels in the monitoring video frame and constructs a grayscale pixel matrix of the monitoring video frame using the grayscale values ​​of the pixels. The gradient feature extraction processing method calculates the gradient value of the pixels based on the grayscale values ​​of the pixels, uses non-maximum suppression and dual threshold detection algorithms in the Canny edge detection algorithm to detect the edge pixels of the monitoring video frame, and marks the edge pixels in the grayscale pixel matrix of the monitoring video frame. The grayscale pixel matrix with marked edge pixels is used as the preprocessed monitoring video frame.

4. The attendance tracking method based on video context analysis as described in claim 1, characterized in that, Step S2 involves using a face recognition model to perform individual face recognition on the preprocessed surveillance video frames, including: The face recognition model includes an input layer, three consecutive spatial channel cross-attention units, and a head layer; The input layer is used to receive preprocessed monitoring video frames and perform convolution processing on the preprocessed monitoring video frames to obtain the convolutional feature maps of the preprocessed monitoring video frames. The spatial channel cross attention unit is used to perform attention processing on the feature map output by the previous layer and output the attention-processed feature map. The feature map output by the third spatial channel cross attention unit is the attention feature map of the preprocessed monitoring video frame. The attention processing includes channel attention processing and spatial attention processing. The Head layer includes a classification branch and a bounding box regression branch. It is used to divide the attention feature map into multiple target region feature maps and generate bounding box parameters for each target region feature map. The bounding box parameters include the center coordinates, width offset, and length offset of the bounding box. It also calculates the face region confidence score of the target region feature map. Combined with the edge pixels of the preprocessed monitoring video frame corresponding to the bounding box parameters, the face region confidence score is corrected. Bounding box parameters with corrected face region confidence scores higher than a preset threshold are retained. The retained bounding box parameters are used to generate bounding boxes in the preprocessed monitoring video frame. The image region enclosed by the bounding box is regarded as the individual face in the preprocessed monitoring video frame.

5. The attendance tracking method based on video context analysis as described in claim 4, characterized in that, Matching the individual's face with the facial information of students who did not attend class in the student attendance database includes: The system acquires the face images of students who did not attend class from the student attendance database. It then uses three consecutive spatial channels to cross attention units to generate attention feature maps of the individual face and the student face images. The similarity of the attention feature maps is calculated. If the similarity calculation result is higher than a preset similarity threshold, it means that the face information of the student who did not attend class and the identified individual face are successfully matched.

6. The attendance tracking method based on video context analysis as described in claim 5, characterized in that, The behavioral trajectory data of the absent students in step S3 includes: The timestamp information of the preprocessed surveillance video frames associated with the successfully matched individual faces, along with the surveillance scene information, are used as a set of behavioral information for the absent students. This behavioral information is then sorted according to the timestamp information to obtain the behavioral trajectory data x of the absent students. x=(x1,x2,…,x m ,…,x M ): in: x1,x2,...,x m ,...,x M This represents the M groups of behavioral information in the behavioral trajectory data x, where x is... m This represents the m-th group of behavioral information in the behavioral trajectory data x. The behavioral information x are listed in order. m The timestamp information and monitoring scene information in the data, m∈[1,M], where M represents the number of behavioral information groups in the behavioral trajectory data x.

7. The attendance tracking method based on video context analysis as described in claim 1, characterized in that, The student attendance database determines the similarity measurement results. If the similarity measurement results exceed the preset trajectory similarity threshold, then the absent student meets the attendance make-up rules. The absent student is automatically made up in the student attendance database, and the behavioral trajectory data of the absent student and the preprocessed monitoring video frames associated with the behavioral trajectory data are recorded.

Images