Teacher identification and tracking methods, devices, classroom monitoring equipment and media

Abstract

The application discloses a teacher identity authentication and tracking method and device, a classroom monitoring device and a medium. The method comprises the following steps: acquiring a video image stream of a classroom monitoring device; performing human body and key point detection on each frame of image based on a human body posture estimation network, and performing blackboard, head and face detection on each frame of image by using a three-class target detection network; tracking each detected target human body based on an improved multi-target tracking algorithm, and attributing the detected head and face to each target human body; determining position information and time sequence information of each target human body relative to a blackboard region, and performing preliminary screening on the teacher identity to obtain a teacher target; and performing identity re-authentication based on a pre-trained cascade model of posture classification and pedestrian ReID feature extraction, so as to prevent the identity exchange between a teacher and a student. The application realizes the continuous authentication of the teacher identity with the characteristics of non-sensing, low consumption, simple algorithm form, high robustness, high credibility and scalability.

Description

Technical Field

[0001] This application relates to the interdisciplinary fields of computer vision, biometric recognition and educational informatization, specifically to a teacher identity authentication and tracking method, device, classroom monitoring equipment and medium. Background Technology

[0002] In scenarios such as teaching quality monitoring, teacher attendance, teaching supervision, and education big data governance, common problems encountered by existing technologies during implementation are:

[0003] (1) Traditional methods such as manual roll call, card swiping, or mobile check-in with facial or body features require schools to formulate relevant systems within the legal framework. Their legality and effective binding force are controversial, and there are human or system risks such as proxy signing, forgetting, and resistance. Moreover, they violate the principle of "seamless" and have a poor user experience, which is not conducive to implementation.

[0004] (2) The reliability of general static face recognition algorithms drops significantly in classroom environments. Teachers frequently turn to the side of the blackboard, lower their heads to operate experimental equipment, or bend over to tutor students, resulting in a low proportion of video frames with visible faces, which leads to a significant increase in the false negative rate.

[0005] (3) Actions such as students standing up to ask questions, students on duty wiping the blackboard, and students patrolling the back row are easily confused with the actions of teachers. Summary of the Invention

[0006] To address the aforementioned issues, embodiments of this application provide a teacher identity authentication and tracking method, device, classroom monitoring equipment, and medium. The aim is to provide a non-intrusive, low-consumption, simple algorithm, highly robust, highly reliable, and scalable method for continuous teacher identity authentication, thereby overcoming or at least partially overcoming the shortcomings of the prior art.

[0007] The embodiments of this application adopt the following technical solutions:

[0008] Firstly, this application provides a method for teacher identity authentication and tracking, including:

[0009] Acquire video image streams from classroom monitoring equipment;

[0010] Human body and key point detection are performed on each frame of the image based on a human pose estimation network, and blackboard, head and face detection are performed on each frame of the image based on a three-class object detection network.

[0011] Based on the improved multi-target tracking algorithm, each detected target human body is tracked, and the detected heads and faces are attributed to each of the target human bodies;

[0012] The positional and temporal information of each target human body relative to the blackboard area are determined, and based on the positional and temporal information, the teacher identity of each target human body is initially screened to obtain teacher targets;

[0013] A cascaded model based on pre-trained pose classification and pedestrian ReID feature extraction is used for identity re-authentication to prevent teachers and students from exchanging identities.

[0014] Secondly, this application also provides a teacher identity authentication and tracking device, the device comprising:

[0015] The acquisition unit is used to acquire video image streams from classroom monitoring equipment.

[0016] The target detection unit is used to detect human bodies and key points in each frame of images based on a human pose estimation network, and to detect blackboards, heads and faces in each frame of images using a three-category target detection network.

[0017] The tracking unit is used to track each detected target human body based on an improved multi-target tracking algorithm, and to assign the detected heads and faces to each of the target human bodies.

[0018] The initial screening unit is used to determine the position information and timing information of each target human body relative to the blackboard area, and to perform preliminary screening of the teacher identity of each target human body based on the position information and the timing information to obtain teacher targets;

[0019] The re-authentication unit is used to perform identity re-authentication based on a cascaded model of pre-trained pose classification and pedestrian ReID feature extraction to prevent teachers and students from exchanging identities.

[0020] Thirdly, this application also provides a classroom monitoring device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned teacher authentication and tracking method.

[0021] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when instructed by a processor, implements the steps of the aforementioned teacher authentication and tracking method.

[0022] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0023] This application first acquires video image streams from classroom monitoring equipment; it then performs human body and key point detection on each frame of the image based on a human pose estimation network, and performs blackboard, head, and face detection on each frame of the image using a three-class object detection network; next, it tracks each detected human body based on an improved multi-object tracking algorithm, further classifying the detected heads and faces to their respective target bodies; then, it performs initial screening of teacher targets based on the detection and tracking results, specifically determining the position and temporal information of each target body relative to the blackboard area, and performs preliminary screening of the teacher's identity for each target body, obtaining the teacher target; finally, it performs identity re-authentication based on a cascaded model of pre-trained pose classification and pedestrian ReID feature extraction, effectively preventing identity exchange between teachers and students. This application achieves seamless, low-cost, simple algorithmic form, high robustness, high reliability, and scalability for continuous teacher identity authentication.

[0024] First, this application automatically locates the teacher's position and identifies the teacher's identity throughout the entire process without the need for manual input of teacher characteristics. Compared with traditional methods, it is more intelligent and avoids the legal risks and user experience obstacles of traditional manual or biometric input methods. Through a purely visual passive perception mechanism, it achieves de-identified feature extraction based on public monitoring equipment without the need for active cooperation. This not only meets the requirements of privacy protection regulations, but also eliminates management loopholes such as proxy signing and forgetting.

[0025] Second, this application innovatively integrates semantic understanding of teaching scenarios with multimodal biometrics. It initially locates the teacher target through spatiotemporal correlation analysis of human posture and target, and then uses a cascaded model of pre-trained posture classification and pedestrian ReID feature extraction for identity re-authentication, effectively overcoming the recognition failure problem caused by teachers frequently turning their backs, looking down, and interference from student activities.

[0026] Third, this application achieves embedded real-time computing through lightweight design, introducing only a small amount of tolerable latency while ensuring no loss of matching accuracy; at the same time, it has highly flexible algorithm configuration capabilities, and for high-performance computing platforms or video stream processing scenarios, it can seamlessly switch to the traditional full-function ReID mode with only minimal parameter adjustments, adapting to multi-level hardware ecosystems. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 A flowchart illustrating a machine vision-based teacher authentication and tracking method in classroom observation according to an embodiment of this application is shown.

[0029] Figure 2A schematic diagram of 17 key points according to an embodiment of this application is shown;

[0030] Figure 3 A schematic diagram showing the result of human pose estimation of a frame of image according to an embodiment of this application is illustrated.

[0031] Figure 4 A schematic diagram showing the result of blackboard region detection on a frame of image according to an embodiment of this application is illustrated.

[0032] Figure 5 This illustration shows a schematic diagram of the result of head detection on a frame of image according to an embodiment of this application;

[0033] Figure 6 This illustration shows a schematic diagram of the result of face detection on a frame of an image according to an embodiment of the present application;

[0034] Figure 7 A schematic diagram showing the merging of multiple blackboard areas to obtain a unified blackboard area according to an embodiment of this application is shown;

[0035] Figure 8 A schematic diagram of a process for tracking detected human targets based on an improved multi-target tracking algorithm according to an embodiment of this application is shown.

[0036] Figure 9-a This represents the intersection of two rectangular detection boxes in a set of detection boxes. Figure 9-b This represents the union of two rectangular detection boxes in a set of detection boxes;

[0037] Figure 10 A schematic diagram of four standing postures of a pedestrian according to an embodiment of this application is shown;

[0038] Figure 11 A schematic diagram of the comparison objects stored in a candidate pedestrian database according to an embodiment of this application is shown;

[0039] Figure 12 A schematic diagram illustrating identity re-authentication using a cascaded model according to an embodiment of this application is shown;

[0040] Figure 13 A schematic diagram illustrating identity re-authentication using a cascaded model according to another embodiment of this application is shown;

[0041] Figure 14 This diagram illustrates the structure of a teacher identity authentication and tracking method apparatus according to an embodiment of the present application.

[0042] Figure 15 A schematic diagram of the structure of a classroom monitoring device according to an embodiment of this application is shown. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] To enable those skilled in the art to more clearly understand the technical solutions provided in the various embodiments of this application, the technical concept of this application will first be described.

[0045] This application aims to provide a non-intrusive, low-consumption algorithm that can be deployed on low-power embedded platforms, especially with recording and broadcasting equipment equipped with cameras or other forms of AIoT edge devices, to achieve highly robust, highly reliable, and scalable continuous authentication of teacher identity in classroom observation scenarios, providing a reliable foundation for the correlation analysis of teaching behaviors.

[0046] Figure 1 This application illustrates a method for teacher authentication and tracking according to an embodiment of the present application, from... Figure 1 As can be seen, this embodiment includes steps S110 to S150:

[0047] Step S110: Obtain the video image stream from the classroom monitoring equipment.

[0048] This application is primarily used in classroom observation scenarios and is based on machine vision technology. Classroom monitoring equipment can be, but is not limited to, cameras or cameras with video recording capabilities, installed in appropriate locations within the classroom.

[0049] This application can be used for teacher authentication and tracking in video streams over historical periods, as well as for real-time monitoring of church conditions, in which case the real-time video stream can be read from classroom monitoring equipment. Recommended, but not limited to, video stream resolution is 1280×720.

[0050] Step S120: Human body and key point detection are performed on each frame of the image based on the human pose estimation network, and blackboard, head and face detection are performed on each frame of the image using a three-class target detection network.

[0051] A human pose estimation network or model can be pre-built and trained. This network can be based on any network structure and can simultaneously detect human bounding boxes and multiple key points of each human body in a single image. In some embodiments of this application, the recommended multiple key points include, but are not limited to: both eyes, both ears, nose, both hands, both elbows, both shoulders, both hips, both knees, and both feet, totaling 17 key points. For details, please refer to [reference needed]. Figure 2 . Figure 2 A schematic diagram of 17 key points according to an embodiment of this application is shown.

[0052] Specifically, in some embodiments of this application, for each frame of image, it is recommended to use a human pose estimation model based on YOLOv5-Pose for detection, to obtain detection boxes of several target human bodies and multiple key points of each target human body. It should be noted that for occluded key points, the human pose estimation model can be estimated. The detection box and key points of each target human body can be represented by a 38-dimensional vector, as shown in equation (1):

[0053] Equation (1);

[0054] in, These are the coordinates of the top left corner of the human body rectangular detection box (or the center point of the human body target box). The length and width of the rectangular detection box; The coordinates are for 17 key points.

[0055] The bounding boxes and key point coordinates of each target human are normalized. These coordinate values ​​are normalized by dividing the horizontal value by the image width and the vertical value by the image height, so that the values ​​range from [0,1].

[0056] A three-class object detection network or model can be pre-built and trained, based on any network structure, which can simultaneously detect all blackboard areas and all heads and faces in a single image.

[0057] Specifically, in some embodiments of this application, for a single frame of image, it is recommended to use a three-category object detection model based on YOLOv5 to detect objects and obtain detection boxes for several blackboard regions, heads, and faces.

[0058] Each blackboard area is represented by the following vector, as shown in equation (2):

[0059] Equation (2);

[0060] Each head detection box is represented by the following vector, as shown in equation (3):

[0061] Equation (3);

[0062] Each face detection bounding box is represented by the following vector, as shown in equation (4):

[0063] Equation (4);

[0064] Similarly, x b , y b ; x h , y h ; x f , y f These are the coordinates of the top-left corner vertex of the rectangular detection box. w b , h b ; w h , h h ; w f , h f The length and width of the rectangular detection box.

[0065] Similarly, these coordinate values ​​are normalized so that they range from [0,1].

[0066] Upon receiving each frame of the image, the aforementioned human pose estimation and three-class object detection processes are performed to obtain the set of human poses of the target human and the set of target results.

[0067] The set of human poses of multiple target human bodies in a single frame of video can be represented by equation (5). Please refer to [the relevant equation]. Figure 3 , Figure 3 This diagram illustrates the result of human pose estimation on a frame of image according to an embodiment of this application. The detected target human body includes X0-X1. k :

[0068] Equation (5);

[0069] The set of blackboard regions can be represented by equation (6), please refer to it. Figure 4 , Figure 4 This diagram illustrates the result of blackboard region detection on a frame of image according to an embodiment of this application. Figure 4 The document includes a blackboard area detection frame. The blackboard area includes B0-B1. k :

[0070] Equation (6);

[0071] The set of heads can be represented by equation (7), please refer to it. Figure 5 , Figure 5 This diagram illustrates the result of head detection on a frame of image according to an embodiment of this application, where the heads include H0-H0. k :

[0072] Equation (7);

[0073] The set of faces can be represented by equation (8), please refer to it. Figure 6 , Figure 6 This diagram illustrates the result of face detection on a frame of image according to an embodiment of this application, where the face includes F0-F1. k :

[0074] Equation (8);

[0075] Similarly, the coordinate values ​​of the detection boxes for the blackboard area, the head, and the face are normalized to a range of [0,1].

[0076] It should be noted that k in equations (5) to (8) represents the quantity. k is an independent parameter in each formula, and its value is determined based on the actual detection situation of each frame.

[0077] exist Figure 4 In the scenario shown, there are multiple blackboard areas. If there are multiple blackboard areas, then these multiple blackboard areas are merged to obtain a single, unified blackboard area.

[0078] Specifically, in some embodiments of this application, merging multiple blackboard regions includes: calculating the pixel area of ​​each blackboard region and sorting them in descending order of pixel area; setting an area ratio threshold and removing detection boxes whose pixel area is less than the area ratio threshold of the largest blackboard region; and obtaining bounding boxes for the remaining blackboard regions to form an overall blackboard region.

[0079] The method of removing detection boxes whose pixel area is less than the area ratio threshold of the largest blackboard area includes: determining the area ratio of each remaining blackboard area to the largest blackboard area based on the largest blackboard area; if the area ratio of a remaining blackboard area is less than the area ratio threshold, then the remaining blackboard area is removed; otherwise, the remaining blackboard area is retained.

[0080] Specifically, the process of obtaining bounding boxes for the retained blackboard areas to form the overall blackboard area includes: for each retained blackboard area, calculating the coordinate values ​​of the top-left and bottom-right vertices; calculating the maximum and minimum values ​​of the top-left and bottom-right vertex coordinate values ​​for multiple blackboard areas; and constructing bounding boxes based on the minimum value of the top-left vertex coordinates and the maximum value of the bottom-right vertex coordinates to form the overall blackboard area.

[0081] In real-world scenarios, a blackboard may be divided into multiple areas, such as... Figure 4 Therefore, it is necessary to merge the blackboard area. The specific steps for merging the blackboard area are as follows:

[0082] First, the pixel area of ​​each blackboard region is calculated using equation (9):

[0083] Equation (9);

[0084] S i This represents the pixel area of ​​the i-th blackboard region.

[0085] Second, sort the pixels from largest to smallest, as shown in equation (10):

[0086] Equation (10).

[0087] Third, to For each element, calculate its pixel area ratio with the largest blackboard region, as shown in Equation (11):

[0088] Equation (11).

[0089] Fourth, set an area ratio threshold. T B ,like If the conditions are met, the corresponding blackboard areas will be removed from the detection results, and the area ratio threshold will be used. T B As an empirical value, this application recommends a value of 0.20, which means that if the area of ​​the currently detected blackboard region is less than one-fifth of the largest blackboard region, then it is very likely to be a false detection and needs to be excluded. Thus, the set of regions to be excluded is obtained, as shown in Equation (12):

[0090] Equation (12);

[0091] Fifth, regarding B n For each element in the rectangle, calculate the coordinates of its top-left and bottom-right vertices, represented as a vector according to equation (13):

[0092] Equation (13).

[0093] Sixth, find The maximum and minimum values ​​are listed below:

[0094] .

[0095] Seventh, use the maximum and minimum values ​​to synthesize a new blackboard region and construct a bounding box to form the overall blackboard region. B minmax For reference Figure 7 , Figure 7 A schematic diagram of merging multiple blackboard areas to obtain a whole blackboard area according to an embodiment of this application is shown, as shown in the red rectangle in the figure, and the formula is as shown in equation (14):

[0096] Equation (14).

[0097] Step S130: Track each detected target human body based on the improved multi-target tracking algorithm, and assign the detected heads and faces to each of the target human bodies.

[0098] Multi-target tracking is a key component of video analytics, which maintains a unique trackID for each detected target in a video frame sequence.

[0099] The currently popular multi-object tracking framework is the SORT model. SORT (Simple Online and Realtime Tracking, ICIP16, A. Bewley et al.) is the mainstream Tracking-by-Detection framework. Its basic idea is to detect first and then track. It argues that if the frame rate is high enough and the detection effect is good enough, the Intersection over Union (IoU) between the detection boxes of each object in two consecutive frames can serve as a strong basis for association.

[0100] In some embodiments of this application, the SORT model has been improved. The main improvements are: first, an AlphaBeta filter is used instead of the existing Kalman filter; second, the specific algorithm for the cross-union ratio (CUI) has been improved; and third, during tracking, this application tracks not only the target's bounding box but also 17 key points of the target.

[0101] Specifically, in some embodiments, the improved multi-target tracking algorithm is implemented using a SORT model improved with an AlphaBeta filter. Tracking each detected human target based on the improved multi-target tracking algorithm includes: assigning a unique trackID to each detected human target and recording the tracking time, designating it as a tracking object; using an AlphaBeta filter to associate each detected human target in the current frame with several existing tracking objects; if a human target is associated with a tracking object, assigning the trackID of the tracking object to the human target; if a human target is not associated with a tracking object, assigning it a new unique trackID and recording the tracking time, designating it as a tracking object; if a tracking object is not associated with a human target within a preset time period, discarding the trackID of the tracking object; and updating the key points of each human target in the current frame.

[0102] The tracking approach is: first detect, then associate. As mentioned earlier, if the current frame is the initial frame image, then for the set of detected target human bodies in that frame... For each detected human body (denoted as the target human body) and its key points, a unique trackID is assigned to each target human body. This trackID does not change over time. Therefore, the vector representing the target human body is... By expanding, we obtain the vector shown in equation (15). :

[0103] Equation (15);

[0104] Here, z represents the trackID of the target human body, and t represents the tracking time.

[0105] Each target human body assigned a unique trackID is treated as a tracking object.

[0106] As mentioned above, the current frame is the first frame, and five target human bodies are detected. Each target human body is assigned a unique trackID, which is denoted as A1, A2, A3, A4, and A5. That is, the existing tracked objects are A1, A2, A3, A4, and A5.

[0107] Then, the AlphaBeta filter is used to associate each detected human target in the current frame image with several existing tracking objects.

[0108] Then, in the second frame, the detection in step S120 indicates that there are several target human bodies in the second frame image. The AlphaBeta filter is then used to associate these target human bodies with several existing tracking objects.

[0109] Specifically, in some embodiments of this application, an AlphaBeta filter is used to associate each detected target human body in the current frame image with several existing tracking objects. This includes: using the AlphaBeta filter to calculate the correlation degree between each tracking object and each target human body to form a correlation degree matrix; using the Hungarian algorithm to find the optimal match based on the correlation degree matrix, and setting a minimum correlation degree threshold during the matching process; associating each target human body with several existing tracking objects according to the optimal matching result; if there is no tracking object at present, assigning a unique trackID and recording the tracking time to the newly detected target human body as a tracking object; if a tracking object is not associated with a target human body, using the AlphaBeta filter to perform prediction and modeling, associating the modeled target human body with the tracking object, and updating the modeled target human body to the set of detected target human bodies.

[0110] Then, in the second frame, based on the detection in step S120, the correlation between each existing tracked object and the target human body detected in the frame is calculated to form a correlation matrix. Assuming that four target human bodies are detected in the second frame image, denoted as B1, B2, B3, and B4 respectively, the correlation matrix formed by the data elements shown in Table 1 is formed.

[0111] Table 1

[0112]

[0113] The Hungarian algorithm is used to find the optimal match based on the correlation matrix, and a minimum correlation threshold is set during the matching process.

[0114] A minimum correlation threshold is set. If the correlation between a tracked object and a target human body is less than the minimum correlation threshold, the two are marked as an unacceptable match, thus performing the initial screening for matching.

[0115] Then, the Hungarian algorithm is used to find the optimal match between the tracked object and the target human body, matching a tracked object for each target human body. After matching, the matching results are obtained, which typically include the following situations.

[0116] If a target human body is successfully matched with a tracked object, the two are associated, and the trackID of the tracked object is assigned to the target human body. As shown in Table 1, if the tracked object A1 and the target human body B1 are successfully matched, and the trackID of the tracked object A1 is recorded as ID0, then ID0 is assigned to the target human body B1.

[0117] If a target human body is not associated with a tracked object, a new unique trackID is assigned to it, its tracking time is recorded, and it is also treated as a tracked object. For example, if there are 5 tracked objects in the first frame and 6 target human bodies in the second frame, and each tracked object in the first frame has been matched with one of the 5 target human bodies in the second frame, then a new target human body has appeared in the second frame. In this case, a new unique trackID is assigned to this unassociated target human body, its tracking time is recorded, and it is also treated as a tracked object.

[0118] If a tracked object is not associated with a target human body, an AlphaBeta filter is used for prediction and modeling. The modeled target human body is then associated with the tracked object, and the modeled target human body is updated in the set of detected target human bodies. As shown in Table 1, assuming tracked object A1 matches target human body B1 successfully, tracked object A2 matches target human body B2 successfully, tracked object A3 matches target human body B3 successfully, tracked object A4 matches target human body B4 successfully, but tracked object A5 does not match a target human body. This could be due to various reasons, such as tracked object A5 moving out of the current frame, or tracked object A5 being occluded by other people or objects. However, it may return to the current frame in the next frame, or the occlusion may be removed. In this case, an AlphaBeta filter is used for prediction and modeling, and the modeled target human body is associated with tracked object A5, and the modeled target human body is updated in the set of detected target human bodies.

[0119] Specifically, the SORT model can model the target human body in this frame using a Kalman filter based on measurements such as center coordinates, area, and aspect ratio. The target human body's position information is used to continuously update its state, and the modeled target human body is associated with the tracked object A5. It is recommended to use interpolation during modeling, that is, to model based on the target's center coordinates, area, aspect ratio, and other measurements in the first and third frames.

[0120] If a tracked object is not associated with a target human, it will not be processed through modeling indefinitely. Modeling is only for situations where the target human is not present for a short period of time. If a target human is not present for a longer period of time, it means that it has left the screen. Therefore, if a tracked object is not associated with a target human within a preset time, the trackID of that tracked object will be discarded.

[0121] After each association, the key points of each target human in the current frame image are updated. One aspect that distinguishes this application from existing multi-target tracking technologies is that it not only tracks the detection bounding box but also tracks and updates the key points. For the aforementioned human, in addition to the detection bounding box (4 parameters), there are 17 key points (34 parameters), totaling 38 parameters, that need to be tracked. This has significant advantages over traditional techniques that only track the detection bounding box. This mechanism, by fusing key point features, can enhance the representation of the target's appearance, improve the ability to distinguish between occluded and similar targets, optimize the accuracy of data association, and reduce ID switching and trajectory fragmentation. Especially in crowded or dynamic scenarios such as classrooms, it can improve tracking robustness and positioning accuracy, providing more reliable target motion and posture information for downstream tasks such as behavior analysis.

[0122] The degree of association can be represented by the intersection-union ratio (IoU), or any of its variants DIoU, GIoU, and CIoU.

[0123] Figure 8 This diagram illustrates a process for tracking detected human targets based on an improved multi-target tracking algorithm according to an embodiment of this application. Figure 8 As can be seen, this embodiment includes:

[0124] First, acquire several target human bodies and existing tracking targets in the current frame image.

[0125] Then it is determined whether the number of current tracking targets is zero. If the number is zero, a unique trackID is assigned to each target human body and the tracking time is recorded as the tracking object, and then the process proceeds to the next frame.

[0126] If the number is not zero, first calculate the correlation degree between each tracked object and each target human body to form a correlation degree matrix. Then, use the Hungarian algorithm to find the optimal match based on the correlation degree matrix. Based on the optimal match result, associate each target human body with several existing tracked objects.

[0127] Then, for each target human body, it is determined whether it is associated with a tracking object. If it is associated, the trackID of the associated tracking object is assigned to the target human body; if it is not associated, a new unique trackID is assigned to the target human body and it is used as a new tracking object.

[0128] After completing the above steps, determine whether there are any tracking objects that are not associated with the target human body within the preset time period. If so, use the AlphaBeta filter to predict and model the tracking objects that are not associated with the target human body, associate the modeled target human body with the tracking object, and update the detected target human body set.

[0129] If it does not exist, update the key points of each target human body in the current frame image; finally, proceed to the next frame and repeat the above process.

[0130] In the above embodiments, the AlphaBeta filter is used to replace the Kalman filter in the original SORT tracker, thereby reducing computational complexity. This is the preferred recommended technique, but it can also be replaced with other lightweight filters.

[0131] To further improve the accuracy and robustness of detection, detected heads and faces are assigned to various target human bodies. Specifically, this includes: calculating the improved intersection-union ratio (IUU) of the detection bounding boxes of each detected head or face with each target human body; if the IUU of a head or face with a target human body is greater than a first empirical threshold, then the head or face is assigned to that target human body.

[0132] The improved crossover-union ratio can be expressed as equation (16):

[0133] Equation (16);

[0134] In the formula, A and B represent different detection box pairs; This represents the minimum value of the union of multiple detection box pairs.

[0135] In some embodiments of this application, the existing cross-union ratio (CUNR) definition is improved, and based on the improved CUNR, human heads and faces in the detected raw results are associated with each target human body.

[0136] Please refer to Figure 9-a and Figure 9-b Two rectangular detection boxes form a detection box pair. Figure 9-a This represents the intersection of two rectangular detection boxes in a set of detection boxes. Figure 9-b This represents the union of two rectangular detection boxes in a set of detection boxes. In the prior art, the intersection-union ratio can be expressed as equation (17):

[0137] Equation (17).

[0138] In this embodiment of the application, the existing crossover-union ratio has been improved, and the specific details can be found in the aforementioned formula (16). This indicates finding the minimum value in the union of multiple bounding box pairs.

[0139] For the head H i For each H i Calculate its relationship with X i Improved crossover ratio ,when At that time, associate it with thatX i Expand it to become the vector shown in equation (18):

[0140] Equation (18);

[0141] in, t h This is an empirical value, denoted as the first empirical threshold, with a recommended value of 0.90~0.95.

[0142] For faces, the embodiments of this application can be associated using either of the following two methods.

[0143] The first method can refer to the aforementioned method of associating heads to calculate... ,when Associate it with X i .

[0144] The second method involves classifying a face as belonging to the target human body if three designated key points of a target human body are covered by a detection bounding box for a face. This is achieved by using the pedestrian's 0th, 1st, and 2nd key points (e.g., ...). Figure 2 If these three key points are completely contained within the face detection bounding box of a target human body, then the face can be considered to belong to that target human body.

[0145] Step S140: Determine the position information and timing information of each target human body relative to the blackboard area, and perform preliminary screening of the teacher identity of each target human body based on the position information and timing information to obtain teacher targets.

[0146] The teacher's target is initially identified based on the positional and temporal information of the target human body relative to the blackboard area.

[0147] The position information of the target human body relative to the blackboard area can be calculated according to the following method: determine the overlap degree between each target human body and the blackboard area, and the overlap degree can be characterized by an improved intersection-union ratio; if the overlap degree is greater than a preset second empirical threshold, then mark the target human body as being on the podium; otherwise, mark the target human body as not being on the podium.

[0148] For each X i Calculate its relationship with the blackboard area The degree of overlap, such as the improved crossover ratio between the two. As a metric, when When the target human body is not on the podium, it can be considered that it is not on the podium and marked as not on the podium. The vector representing the target human body is expanded into the vector shown in equation (19):

[0149] Equation (19);

[0150] Here, b is a marker indicating whether the pedestrian is on the podium; a value of 0 indicates that the pedestrian is not on the podium, and a value of 1 indicates that the pedestrian is on the podium.

[0151] Temporal information from vectors representing the target human body X i The parameter t can be obtained from this.

[0152] Based on the location information and the time sequence information, a preliminary screening of the teacher identity is performed on each of the target human bodies to obtain teacher targets. Specifically, in some embodiments of this application, the process includes: traversing each target human body; for a target human body, determining whether the target human body is located on the podium, or determining whether the duration of the target human body on the podium is greater than a preset duration threshold based on the recorded time sequence information; if so, the target human body is selected as a candidate object; if the number of candidate objects is greater than zero, the duration of the multiple candidate objects on the podium is sorted in descending order, and the one with the highest ranking is selected as the teacher; if multiple candidate objects are tied for the highest ranking, the one with the minimum Y-axis value in the pedestrian detection box is selected as the teacher target.

[0153] Traversal Each X i , X i The vector is shown in equation (19), and the vectors with b=1 and T are selected. s Target human bodies >60 (60 is a preset threshold that can be modified), where T s The definition can be understood as the duration of a target human body in the podium area. If a target human body with the same trackID is continuously in the podium area and the duration is greater than the preset 60, then it is considered as a candidate object.

[0154] If there are 0 candidate objects, then return to the previous steps S110 to S130 for continued detection and observation.

[0155] If there is only one candidate, it will be directly used as the teacher target and the criteria will be applied.

[0156] If there are more than one candidate, the duration of time the candidates spend on the podium is sorted in descending order, and the candidate with the highest ranking is selected as the teacher.

[0157] If multiple candidate objects are tied for first place in the ranking results, meaning that the duration of each candidate object at the podium is equal and all are the longest, then the one with the highest detection box is selected as the teacher. In this embodiment, the pedestrian detection box has a Y-axis (y... pThe minimum value is used as the teacher. The specific selection method is related to the setting of coordinates. The core of this step is to select the highest value in the detection box, that is, the tallest value, as the teacher target.

[0158] Mark the teacher's goals.

[0159] Step S150: Based on the cascaded model of pre-trained pose classification and pedestrian ReID feature extraction, identity re-authentication is performed to prevent the teacher and student from exchanging identities.

[0160] In actual classroom observations, teachers don't always remain at the podium; it's also common for students to go up to the podium to write on the blackboard or solve problems. In these situations, an ID switch can easily occur between teachers and students. That is, when the teacher's and student's bounding boxes overlap, their trackIDs are swapped with a certain probability. T s ID-switching can also lead to misidentification of teachers. Therefore, this application provides an identity re-identification strategy to correct ID-switching.

[0161] Specifically, in some embodiments of this application, the teacher target is re-authenticated based on a cascaded model of pre-trained posture classification and pedestrian ReID feature extraction, including: constructing and training a posture classifier and a pedestrian ReID feature extraction model to form the cascaded model; constructing a candidate pedestrian database based on the cascaded model of the posture classifier and the pedestrian ReID feature extraction model, wherein the candidate pedestrian database includes pedestrian ReID features of the comparison object under different posture dimensions, wherein the posture dimensions include: facing forward, facing away, left-side turn, and right-side turn; determining at least one sampling target in the current frame image according to the position information and the temporal information to form a query map; and using the cascaded model to match the sampling target with each comparison object in the candidate pedestrian database to perform teacher identity re-authentication.

[0162] In the prior art, pedestrian RelD features do not distinguish between the four standing postures of pedestrians. In fact, the features of pedestrians will differ in the four standing postures. This application improves the robustness of re-identification by classifying the RelD features of the comparison objects in the pedestrian candidate library according to their standing posture.

[0163] I. Construction of the Cascade Model

[0164] In this embodiment, a posture classifier and a pedestrian ReID feature extraction model are pre-trained and cascaded together to form a cascaded network of posture classifier and pedestrian ReID feature extraction model, which continuously and re-authenticates the teacher target to prevent the teacher and student from exchanging identities.

[0165] In some embodiments of this application, the standing posture classifier is pre-trained according to the following method: constructing an initial model of the standing posture classifier, which can be built according to any one of the SVM mechanism, decision tree mechanism, and shallow neural network; acquiring sample data from several classroom scenarios and performing sample processing to construct a standing posture dataset, wherein the sample processing includes: using a human pose estimation network to perform key point detection on the sample data, and labeling the sample data with posture categories, wherein the key points include: both eyes, both ears, nose, both hands, both elbows, both shoulders, and both hips, and the posture categories include: facing forward, facing away, left side, and right side; training the initial model of the standing posture classifier based on the standing posture dataset to obtain the standing posture classifier.

[0166] First, an initial model for the standing posture classifier is established based on any one of the following mechanisms: SVM, decision tree, and shallow neural network. In some embodiments of this application, the SVM mechanism is recommended. The SVM mechanism has excellent ability to process high-dimensional data (such as human key point coordinates) and can effectively capture complex posture patterns. It can still ensure good generalization in the case of small samples and avoid overfitting. At the same time, it has strong noise robustness and strong classification stability.

[0167] The posture classifier is a four-class classifier, with categories including: facing forward, facing away, left-side posture, and right-side posture, as illustrated in the diagram below. Figure 10 , Figure 10 A schematic diagram of four standing postures of a pedestrian according to an embodiment of this application is shown.

[0168] When constructing the standing posture dataset for training, classroom videos were collected, and the aforementioned human pose estimation network, such as YOLOv5-Pose, was used to extract 17 key points of the teacher's target in the classroom, while removing key points of the legs and feet (e.g., ...). Figure 2 (Key points 13, 14, 15, and 16) were selected, and 13 key points were retained. The remaining key points were then labeled, with each label corresponding to one of the four different standing postures of the teacher in the classroom: facing forward, facing away, standing to the left, and standing to the right, thus forming a set of standing posture datasets.

[0169] The initial model of the above-mentioned standing posture classifier is trained using this standing posture dataset to obtain the standing posture classifier.

[0170] When the above pre-trained posture classifier is used for... When performing classification, after obtaining the prediction results, Extended to the vector shown in equation (20):

[0171] Equation (20);

[0172] Here, 'c' represents the pedestrian's standing posture.

[0173] Beforehand, a pedestrian ReID feature extraction model needs to be trained. It is recommended that this model be built on MobileFaceNet, trained using real-world teacher scenario data. The input to the pedestrian ReID feature extraction model is a color image, such as a 256×256×3 image, and the output is a multi-dimensional feature vector, such as a 512-dimensional feature vector. Taking a target human body in an image as an example, the ReID feature extraction model can extract multiple sets of feature combinations for re-identification at different times in a very short time, denoted as ReID features. ReID features are a highly refined, robust, and highly discriminative vectorized representation. Their core lies in accurately capturing the uniqueness of an individual's identity across time, space, and perspective changes.

[0174] A cascaded model is formed by combining a trained posture classifier with a pedestrian ReID feature extraction model.

[0175] Second, the construction of the candidate pedestrian database (Gallery sets).

[0176] A candidate pedestrian gallery can be constructed using the cascaded model described above. For a single human figure in an image, a posture classifier is used to classify its posture. Under each posture, a pedestrian ReID feature extraction model is used to extract its pedestrian ReID features, resulting in the following: Figure 11 The candidate pedestrian database is shown. From Figure 11 As can be seen, for a target, such as pedestrian 0, the candidate pedestrian database contains multiple sets of pedestrian ReID features (feature vector 0, feature vector 1, etc.) at different times under four different standing posture dimensions. And so on.

[0177] It should be noted that the candidate pedestrian database can be continuously updated based on the detection results. When a target is detected but is not included in the candidate pedestrian database, it can be added to it at any time.

[0178] The candidate pedestrian database takes into account posture and has an update mechanism, which can improve the robustness of ReID features and the recall rate.

[0179] III. Selection of Sampling Targets

[0180] Unlike existing technologies, this application uses a sampling method when performing identity re-verification. Specifically, based on the location information and the timing information, at least one sampling target is determined in the current frame image to form a query set.

[0181] Specifically, in some embodiments of this application, the step of determining at least one inspection target in the current frame image based on the location information and the timing information to form a query map includes: traversing each target human body; if the target human body is located on the podium, or if the duration of the target human body on the podium is determined to be greater than a preset duration threshold based on the recorded timing information, then the target human body is taken as an inspection target; recording the number of inspections of the inspection target within a preset duration; if the number of inspections is greater than or equal to a preset number threshold, then the inspection target is abandoned.

[0182] Traversal Each X i , X i The vector is shown in equation (19), and the vectors with b=1 or T are selected. s Targets with a population >60 (60 is a preset threshold that can be modified) are selected for random sampling, where T s The definition is the same as before. That is, people who are currently in the podium area, or who are not currently in the podium area but have stayed on the podium for more than a preset time threshold (such as 60 seconds) are selected as candidate targets for random inspection, and are recorded as inspection targets.

[0183] For a given target, each inspection is recorded to determine the number of times the target is inspected within a preset time period (e.g., 5 seconds). If the number of inspections within the preset time period is greater than or equal to a preset threshold (e.g., 2 times), the target is discarded. This filters out targets that are inspected twice within 5 seconds, preventing the same person from being inspected multiple times in a short period.

[0184] If, after screening, there are still multiple targets for inspection, random sampling can be used to further screen them, and at least one target should be retained in the end.

[0185] This application solves the problems of high re-authentication time and overall computational cost in classroom scenarios with a large number of students by setting sampling conditions to select only a small number of targets for feature comparison. Compared with the existing technology that compares the features of all targets one by one, this is particularly suitable for embedded platforms with low computational power, such as classroom supervision, and is more practical. Furthermore, the mechanism of further filtering the selected targets from the candidate targets reduces the number of feature comparisons for a single identity re-authentication, controls the time consumption of identity re-authentication and saves computational power, thereby avoiding blocking or affecting other algorithm modules.

[0186] IV. Identity Re-authentication

[0187] Based on the query graph, the cascaded model is used to perform feature and identity matching between the sampled target and each comparison object in the candidate pedestrian database for teacher identity re-authentication. Using the trained cascaded model, with the query graph as the verification object and each comparison object in the candidate pedestrian database as the benchmark, identity re-authentication can be performed.

[0188] Specifically, in some embodiments of this application, based on the query graph, the cascaded model is used to perform identity matching between the sampled target and each comparison object in the candidate pedestrian database for teacher identity re-authentication, including: classifying the sampled target in the query graph by standing posture based on the standing posture classifier of the cascaded model; calculating the matching degree between the sampled target and each comparison object in the candidate pedestrian database according to the standing posture classification result; and performing teacher identity re-authentication on the sampled target according to the matching degree.

[0189] First, for a sample target k, a standing posture classifier is used to classify the standing posture of the sample target in the query graph. The resulting standing posture classification result is one of facing forward, facing away, left side, and right side.

[0190] Then, based on the posture classification results, the matching degree between the sampled target and each of the comparison objects in the candidate pedestrian database is calculated. Specifically, for a comparison object, the similarity between the sampled target and the comparison object under different posture dimensions is calculated based on Euclidean distance or cosine distance; if the similarity under the posture dimension of the posture classification results is greater than a preset similarity threshold, then the similarity is used as the matching degree between the sampled target and the comparison object; if the similarity under the posture dimension of the posture classification results is not greater than the preset similarity threshold, then the similarities under each posture dimension are weighted and fused to obtain the matching degree between the sampled target and the comparison object.

[0191] The step of calculating the similarity between the sampled target and the comparison object under different standing posture dimensions based on Euclidean distance or cosine distance includes: extracting the sampled ReID features of the sampled target using the pedestrian ReID feature extraction model of the cascaded model; for each standing posture dimension, calculating the Euclidean similarity or cosine similarity between the sampled ReID features and the comparison object's ReID features at different times; and selecting the maximum value of the Euclidean similarity or cosine similarity under each standing posture dimension as the similarity under that standing posture dimension.

[0192] First, the pedestrian ReID feature extraction model of the cascaded model is used to extract the ReID features of the sampled target, denoted as the sampled ReID features. The sampled ReID features of the sampled target can have the same dimension as or different from the ReID features of the comparison object stored in the candidate pedestrian database. In some embodiments of this application, a 128-dimensional feature vector is recommended, such as the sampled ReID feature v of the sampled target, which represents the dimension v = [v0, v1, v2, ... v]. 127 Taking cosine distance as an example, for a comparison object, the cosine similarity between the sampled target and the comparison object is calculated in four posture dimensions. Multiple values ​​are obtained for each posture dimension. Specifically, the number of cosine similarities corresponds to the number of ReID features. Assuming that 5 ReID features are extracted from the comparison object in one posture dimension (in reality, there are far more than 5; this is just an example), namely feature vector 0, feature vector 1, feature vector 2, feature vector 3, and feature vector 4, the cosine similarity matrix between a sampled target and a comparison object is shown in Table 2.

[0193] Table 2

[0194]

[0195] For each standing posture dimension, the maximum value (Top 1) of the Euclidean similarity or cosine similarity under that dimension is selected as the similarity for that posture dimension. For example, in the frontal standing posture dimension, the cosine similarity v1-0 (frontal) value is the largest; therefore, the cosine similarity v1-0 (frontal) is selected as the similarity between the sampled target and the comparison object under the frontal standing posture dimension. For a set of sampled targets and comparison objects, the similarity under each of the four standing posture dimensions is obtained, forming a matching feature vector D. k , which can be expressed as equation (21):

[0196] Equation (21);

[0197] in, d 0~ d 3 indicates the similarity between the sampled target and the comparison object in four standing posture dimensions, and the subscripts 0 to 3 represent different standing posture dimensions.

[0198] Then, the final matching degree is calculated. Specifically, if the similarity is greater than a preset similarity threshold in the standing posture dimension of the standing posture classification result, the similarity is used as the matching degree between the sampled target and the comparison object; if the similarity is not greater than the preset similarity threshold in the standing posture dimension of the standing posture classification result, the similarities in each standing posture dimension are weighted and fused to obtain the matching degree between the sampled target and the comparison object. Please refer to formula (22):

[0199] Equation (22);

[0200] d i 'c' represents different standing posture dimensions, and 'c' represents the standing posture classification result.

[0201] The physical meaning of Equation (22) is that if the matching degree of the current standing posture exceeds the preset similarity threshold of 0.5 (0.5 is an empirical threshold that can be set according to actual needs), then the similarity of the current standing posture is directly used as the final matching degree; otherwise, the average of the similarities under the four standing postures is used as the final matching degree.

[0202] Finally, based on the matching degree, the teacher identity of the sampled target is re-authenticated.

[0203] Specifically, this includes: sorting the matching degree of the sampled target and each comparison object in descending order, and selecting the sampled target and the comparison object with the highest ranking as a comparison group; if the matching degree of the sampled target and the comparison object in the comparison group is greater than or equal to a preset matching degree threshold, and their trackIDs are the same, then the identity re-authentication is confirmed to be successful, and the current trackID of the sampled target is maintained; if the matching degree of the sampled target and the comparison object in the comparison group is greater than or equal to the preset matching degree threshold, and their trackIDs are different, then an identity exchange is confirmed, and the trackIDs of the target bodies corresponding to the sampled target and the comparison object are exchanged; if the matching degree of the sampled target and the comparison object in the comparison group is less than the preset matching degree threshold, and their trackIDs are the same, then the sampled target is marked as not being a teacher target.

[0204] The matching degree between the sampled targets and each comparison object is sorted in descending order, and the pair of sampled targets and comparison objects with the highest matching degree is selected as the comparison group. The characteristics and identities of the comparison group are then matched, mainly including the following three cases:

[0205] In the first case, if the matching degree between the two is greater than or equal to the preset matching degree threshold (e.g., 0.5, which is an empirical threshold and can be set according to actual needs), and the two trackIDs are the same, then the identity re-authentication is confirmed to be successful, and the current trackID of the sampled target is maintained.

[0206] Since the targets identified through the aforementioned screening process are highly likely to be teachers—whether they are the teachers themselves or students who briefly pause at the podium—determining whether the targets have switched roles can largely prevent such switching between teachers and students.

[0207] For the comparison group with the highest matching degree, firstly, it is determined that the matching degree of the two is greater than or equal to a preset matching degree threshold. This preset matching degree threshold is an empirical value, and a value of 0.5 is recommended. If the matching degree of a sampled target and a comparison object is greater than this matching degree threshold, it means that the two have very similar physiological characteristics and should be the same person. Then, it is determined whether the two trackIDs are the same. Assuming that the trackID of the sampled target is "pedestrian 0" and the trackID of the comparison object is also "pedestrian 0", if the two trackIDs are the same, it means that the sampled target with trackID "pedestrian 0" and the comparison object with trackID "pedestrian 0" in the candidate pedestrian database are the same person. There is no trackID exchange between the sampled targets. At this time, the identity re-authentication is confirmed to be successful, and the current trackID of the sampled target is maintained.

[0208] In the second scenario, if the matching degree between the sampled target and the comparison object in the comparison group is greater than or equal to the preset matching degree threshold, but their trackIDs are different, then it is confirmed that the sampled target has exchanged identities, and the trackIDs of the sampled target and the comparison object corresponding to the target human body are exchanged.

[0209] Similarly, if the matching degree between a sampled target and a comparison object is greater than the matching degree threshold, it indicates that their physiological characteristics are very similar and they should be the same person. However, it is determined that their trackIDs are different. Suppose the trackID of the sampled target is "pedestrian 0" and the trackID of the comparison object is "pedestrian 1". This means that the sampled target with trackID "pedestrian 0" is not the same person as the comparison object with trackID "pedestrian 0" in the candidate pedestrian database, but is the same person as the comparison object with trackID "pedestrian 1" in the candidate pedestrian database. That is, the trackIDs of the target and the comparison object have been exchanged in the current frame. At this time, the trackIDs of the two are exchanged. That is, the trackID of the target corresponding to the comparison object in the current frame is assigned to the sampled target, and the trackID of the sampled target is assigned to the target corresponding to the comparison object. That is, the trackID of the sampled target is changed to "pedestrian 1", and the trackID of the target corresponding to the comparison object is changed to "pedestrian 0".

[0210] In the third scenario, if the matching degree between the sampled target and the comparison object in the comparison group is less than a preset matching degree threshold, then the sampled target is marked as not being a teacher target.

[0211] Since the comparison group selects the combination of the sampling target and the comparison object with the highest matching degree, if the matching degree between the sampling target and the comparison object in the comparison group is still less than the preset matching degree threshold, it means that the sampling target is not similar to the physiological characteristics of any of the comparison objects in the current candidate pedestrian database, indicating that it is not stored in the current candidate pedestrian database. In this case, identity re-authentication fails, and the sampling target is temporarily marked as not being a teacher target.

[0212] Furthermore, a cascaded model is used to classify the standing posture of the sampled target and extract ReID features. The extracted results are then stored in the candidate pedestrian database as features of the comparison object.

[0213] In some embodiments of this application, regardless of whether the sampled target successfully matches the comparison object, the matching degree between the sampled target and the comparison object with the same trackID in the candidate pedestrian database is calculated (if it has already been calculated, the result can be directly used here). If the matching degree is within a preset high similarity threshold range, such as (0.70, 0.90), the extracted ReID features are saved in the candidate pedestrian database, that is, the extracted ReID features of the sampled target are saved in the ReID features of the comparison object under the corresponding standing posture in the candidate pedestrian database. This can improve the accuracy of the ReID features of the comparison object, provide a guarantee for subsequent re-authentication, and improve the accuracy of re-authentication.

[0214] Figure 12 This diagram illustrates an embodiment of identity re-authentication using a cascading model according to this application. Figure 12 As can be seen, this embodiment includes:

[0215] First, a target for inspection is determined from the current frame. Then, it is determined whether the number of times the target is inspected within a preset time period is greater than a preset threshold. If it is greater, the target is abandoned.

[0216] If the value is not greater than the target value, a standing posture classifier is used to classify the standing posture of the sampled target. Then, the matching degree between the sampled target and each comparison object in the candidate pedestrian database is calculated, and the group with the highest matching degree is selected as the comparison group.

[0217] Then, it is determined whether the matching degree of the comparison group is greater than the preset experience threshold. If it is not greater, the re-authentication is confirmed to have failed and the target of the sampling is marked as not being a teacher. If it is greater, it is determined whether the trackID of the target of the comparison group and the comparison object are consistent. If they are consistent, the re-authentication is confirmed to have succeeded and the current track ID of the target of the sampling is maintained. If they are inconsistent, it is confirmed that an identity exchange has occurred and the trackID of the target body corresponding to the selected target of the sampling and the selected comparison object is exchanged.

[0218] After completing the above operations, record the sampling time of the target and update the cumulative number of samplings.

[0219] Calculate the matching degree between the sampled target and the comparison object with the same trackID in the candidate pedestrian database. If the matching degree is within the high similarity threshold range, save the extracted ReID features in the candidate pedestrian database and finally proceed to the next frame.

[0220] Figure 13 This diagram illustrates identity re-authentication using a cascading model according to another embodiment of this application. Figure 13 As can be seen, this embodiment includes:

[0221] Two targets were randomly selected, pedestrian 0 and pedestrian 1, and each was matched against the candidates in the pedestrian database. d This indicates the degree of matching between the sampled target and the comparison object.

[0222] exist Figure 13 In the example of pedestrian 0, assume there are n edges connected to the query graph. In the equation, the edge with the highest similarity is... d 00 If the value is greater than 0.5, it means that among the n edges connected to the query graph, the pedestrian features are similar to those of the pedestrian with trackID 0 in the pedestrian feature database, and are considered to be the same person.

[0223] Suppose there are n edges connected to the query graph. In the middle, the edge with the maximum similarity is not... d 00 Assuming d 01 This indicates that the target of the sampling with trackID=0 is not the same person as the person with trackID=0 in the candidate pedestrian database. In this case, if d 01 A value greater than 0.5 indicates that an ID switch has occurred for the person with trackID=0, and their trackIDs need to be exchanged.

[0224] The identity re-authentication in this application takes into account the differences in different standing posture features and the update mechanism of the candidate pedestrian database, which not only improves the robustness of ReID features, but also improves the authentication recall rate.

[0225] The algorithm in this application is simple, suitable for embedded real-time computing, does not sacrifice matching accuracy, and only incurs a small amount of acceptable latency; moreover, the algorithm is flexible in configuration, and for platforms with high computing power or video processing, it can be restored to the traditional RelD method with only a few configuration modifications.

[0226] Figure 14 This diagram illustrates the structure of a teacher identity authentication and tracking device according to an embodiment of this application. Figure 14It can be seen that the teacher identity authentication and tracking device 200 includes:

[0227] Acquisition unit 210 is used to acquire video image streams from classroom monitoring equipment;

[0228] The target detection unit 220 is used to detect human bodies and key points in each frame of images based on a human pose estimation network, and to detect blackboards, heads and faces in each frame of images using a three-class target detection network.

[0229] The tracking unit 230 is used to track each detected target human body based on an improved multi-target tracking algorithm, and to classify the detected human heads and faces to each of the target human bodies;

[0230] The initial screening unit 240 is used to determine the position information and timing information of each target human body relative to the blackboard area, and to perform preliminary screening of the teacher identity of each target human body based on the position information and the timing information to obtain teacher targets;

[0231] The re-authentication unit 250 is used for identity re-authentication based on a cascaded model of pre-trained pose classification and pedestrian ReID feature extraction to prevent teachers and students from exchanging identities.

[0232] In some embodiments of this application, the key points in the above-described device include: both eyes, both ears, nose, both hands, both elbows, both shoulders, both hips, both knees, and both feet.

[0233] In some embodiments of this application, in the above-described apparatus, the target detection unit 220 is used to detect a human pose estimation model based on YOLOv5-Pose for a frame of image, and obtain a number of target human body detection boxes and multiple key points of each target human body; and normalize the coordinate values ​​of the detection boxes and key points of each target human body.

[0234] In some embodiments of this application, in the above-described apparatus, the target detection unit 220 is used to perform detection on a frame of image using a three-category target detection model built based on YOLOv5, to obtain several detection boxes for blackboard regions, heads, and faces; normalize the coordinate values ​​of the detection boxes for each of the blackboard regions, heads, and faces; and if there are multiple blackboard regions, merge the multiple blackboard regions.

[0235] In some embodiments of this application, in the above-described apparatus, the target detection unit 220 is used to calculate the pixel area of ​​each blackboard region and sort them in descending order of pixel area; set an area ratio threshold and remove blackboard regions whose pixel area is less than the area ratio threshold of the largest blackboard region; and obtain bounding boxes for the remaining blackboard regions to form an overall blackboard region.

[0236] In some embodiments of this application, in the above-described apparatus, the target detection unit 220 is used to determine the area ratio of each remaining blackboard area to the maximum blackboard area, based on the maximum blackboard area; if the area ratio of a remaining blackboard area to the maximum blackboard area is less than the area ratio threshold, then the remaining blackboard area is discarded; otherwise, the remaining blackboard area is retained.

[0237] In some embodiments of this application, in the above-described apparatus, the target detection unit 220 is used to: count the vertex coordinates of the upper left and lower right corners of the detection frame for each retained blackboard area; calculate the maximum and minimum values ​​of the upper left and lower right vertex coordinates of multiple blackboard areas respectively; and construct a bounding box based on the minimum value of the upper left vertex coordinates and the maximum value of the lower right vertex coordinates to form the overall blackboard area.

[0238] In some embodiments of this application, in the above-described apparatus, the improved multi-target tracking algorithm is implemented using a SORT model improved with an AlphaBeta filter; the tracking unit 230 is used to assign a unique trackID to each detected target human and record the tracking time as a tracking object; the AlphaBeta filter is used to associate each target human detected in the current frame image with several existing tracking objects; if a target human is associated with a tracking object, the trackID of the tracking object is assigned to the target human; if a target human is not associated with a tracking object, a new unique trackID is assigned to it; if a tracking object is not associated with a target human within a preset time period, the trackID of the tracking object is discarded; the key points of each target human in the current frame image are updated.

[0239] In some embodiments of this application, in the above-described apparatus, the tracking unit 230 is configured to use an AlphaBeta filter to calculate the correlation degree between each tracking object and each target human body, forming a correlation degree matrix; use a Hungarian algorithm to find the optimal match based on the correlation degree matrix, setting a minimum correlation degree threshold during the matching process; associate each target human body with several existing tracking objects according to the optimal matching result; if there is no tracking object currently, assign a unique trackID and record the tracking time for the newly detected target human body, and use it as a tracking object; if a tracking object is not associated with a target human body, use an AlphaBeta filter to predict and model it, associate the modeled target human body with the tracking object, and update the modeled target human body to the set of detected target human bodies.

[0240] In some embodiments of this application, in the above-described apparatus, the correlation degree is any one of IoU, DIoU, GIoU, and CIoU.

[0241] In some embodiments of this application, in the above-described apparatus, the tracking unit 230 is used to calculate the improved cross-union ratio (IUU) of the detection frames of each detected head or face with each target human body; if the IUU of a head or face with a target human body is greater than a preset first empirical threshold, then the head or face belongs to the target human body; wherein, the improved IUU can be expressed as equation (16):

[0242] Equation (16);

[0243] In the formula, A and B represent different detection box pairs; This represents the minimum value of the union of multiple detection box pairs.

[0244] In some embodiments of this application, in the above-described apparatus, the tracking unit 230 is used to classify the face as belonging to the target human body if three designated key points of a target human body are covered by a detection frame of a face. The designated key points include: eyes and nose.

[0245] In some embodiments of this application, in the above-described apparatus, the initial screening unit 240 is used to determine the degree of overlap between each of the target human bodies and the blackboard area, wherein the degree of overlap can be characterized by an improved cross-union ratio; if the degree of overlap is greater than a second empirical threshold, the target human body is marked as being located on the podium; otherwise, the target human body is marked as not being located on the podium.

[0246] In some embodiments of this application, in the above-described apparatus, the initial screening unit 240 is used to traverse each of the target human bodies. For a target human body, it determines whether the target human body is located on the podium, or determines whether the duration of the target human body on the podium is greater than a preset duration threshold based on the recorded time sequence information. If so, the target human body is selected as a candidate object. If the number of candidate objects is greater than zero, the duration of the multiple candidate objects on the podium is sorted in descending order, and the one ranked first is selected as the teacher. If there are multiple candidate objects tied for first place, the tallest one is selected as the teacher target.

[0247] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to construct and train a posture classifier and a pedestrian ReID feature extraction model to form the cascaded model; construct a candidate pedestrian database based on the cascaded model, the candidate pedestrian database including the ReID features of the comparison object under different posture dimensions, wherein the posture dimensions include: facing forward, facing away, left-side posture, and right-side posture; determine at least one sampling target in the current frame image according to the location information and the temporal information to form a query map; based on the query map, use the cascaded model to perform feature and identity matching between the sampling target and each comparison object in the candidate pedestrian database to perform identity re-authentication.

[0248] In some embodiments of this application, in the above-described apparatus, the standing posture classifier is pre-trained according to the following method: constructing an initial model for the standing posture classifier, which can be built based on any one of the SVM mechanism, decision tree mechanism, and shallow neural network; acquiring sample data from several classroom scenarios and performing sample processing to construct a standing posture dataset, wherein the sample processing includes: using a human pose estimation network to perform key point detection on the sample data, and labeling the sample data with posture categories, wherein the key points include: both eyes, both ears, nose, both hands, both elbows, both shoulders, and both hips, and the posture categories include: facing forward, facing away, left side, and right side; training the initial model of the standing posture classifier based on the standing posture dataset to obtain the standing posture classifier.

[0249] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to traverse each of the target human bodies. If the target human body is located on the podium, or if the duration of the target human body on the podium is determined to be greater than a preset duration threshold based on the recorded timing information, then the target human body is selected as a random inspection target. The number of random inspections of the random inspection target within the preset duration is recorded. If the number of random inspections is greater than or equal to a preset number threshold, then the random inspection target is abandoned.

[0250] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to classify the standing posture of the sampled target in the query graph based on the standing posture classifier of the cascaded model; calculate the matching degree between the sampled target and each of the comparison objects in the candidate pedestrian database according to the standing posture classification result; and perform identity re-authentication on the sampled target according to the matching degree.

[0251] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to calculate, for a comparison object, the similarity between the target being sampled and the comparison object under different posture dimensions based on Euclidean distance or cosine distance; if the similarity under the posture dimension of the posture classification result is greater than a preset similarity threshold, then the similarity is used as the matching degree between the target being sampled and the comparison object; if the similarity under the posture dimension of the posture classification result is not greater than the preset similarity threshold, then the similarities under each posture dimension are weighted and fused to obtain the matching degree between the target being sampled and the comparison object.

[0252] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to extract the sampled ReID features of the sampled target using the pedestrian ReID feature extraction model of the cascaded model; for each of the standing posture dimensions, the Euclidean similarity or cosine similarity between the sampled ReID features and the ReID features of the comparison object at different times is calculated; the maximum value of the Euclidean similarity or cosine similarity under each of the standing posture dimensions is selected as the similarity under that standing posture dimension.

[0253] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is used to sort the matching degree of the sampled target and each of the comparison objects in descending order, and select the sampled target and the comparison object with the highest ranking as a comparison group; if the matching degree of the sampled target and the comparison object in the comparison group is greater than or equal to a preset matching degree threshold, and their trackIDs are the same, then the identity re-authentication is confirmed to be successful, and the current trackID of the sampled target is maintained; if the matching degree of the sampled target and the comparison object in the comparison group is greater than or equal to the preset matching degree threshold, and their trackIDs are different, then an identity exchange is confirmed to have occurred, and the trackIDs of the target human bodies corresponding to the sampled target and the comparison object are exchanged; if the matching degree of the sampled target and the comparison object in the comparison group is less than the preset matching degree threshold, and their trackIDs are the same, then the sampled target is marked as not being a teacher target.

[0254] In some embodiments of this application, in the above-described apparatus, the re-authentication unit 250 is further configured to calculate or read the matching degree between the sampled target and the comparison object with the same trackID in the candidate pedestrian database. If the obtained matching degree is within a preset high similarity threshold range, the sampled ReID feature of the sampled target extracted this time is saved in the candidate pedestrian database.

[0255] It should be noted that the aforementioned teacher identity authentication and tracking device 200 can implement all of the aforementioned teacher identity authentication and tracking methods, which will not be elaborated further.

[0256] Figure 15 This application shows a schematic diagram of the structure of a classroom monitoring device according to an embodiment of the present application. Figure 15 As shown, the classroom monitoring equipment includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When executed by the processor, the computer program implements the functions or steps of the teacher authentication and tracking method.

[0257] In one embodiment, the classroom monitoring device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the aforementioned teacher identity authentication and tracking method.

[0258] The above is as stated in this application. Figure 14 The method for teacher authentication and tracking disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0259] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned machine vision-based teacher identification and tracking method in classroom observation.

[0260] It should be noted that the functions or steps that the classroom monitoring equipment or computer-readable storage medium can achieve are described in the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0261] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0262] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0263] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A teacher identity authentication and tracking method, characterized in that, The method comprises the following steps: acquiring a video image stream of a classroom monitoring device; detecting human bodies and key points in each frame image based on a human pose estimation network, and detecting blackboards, heads and faces in each frame image using a three-class target detection network; tracking each detected target human body based on an improved multi-target tracking algorithm, and attributing each detected head and face to a target human body; determining position information and timing information of each target human body relative to a blackboard region, and preliminarily screening teacher identities of each target human body based on the position information and the timing information to obtain a teacher target; performing identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction to prevent identity exchange between a teacher and a student; The method of performing identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction to prevent identity exchange between a teacher and a student comprises: constructing and training a pose classifier and a pedestrian ReID feature extraction model to form the cascade model; constructing a candidate pedestrian library based on the cascade model, wherein the candidate pedestrian library includes ReID features of comparison objects in different pose dimensions, and the pose dimensions include facing, back to, left side and right side; determining at least one spot-check target in a current frame image based on the position information and the timing information to form a query image; performing feature and identity matching between the spot-check target and each comparison object in the candidate pedestrian library based on the query image using the cascade model to perform identity re-authentication, comprising: arranging matching degrees of the spot-check target and each comparison object in descending order, and selecting the spot-check target and the comparison object with the first order as a comparison group; if the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the track IDs of the two are the same, it is confirmed that the identity re-authentication is successful, and the track ID of the spot-check target is maintained; if the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the track IDs of the two are different, it is confirmed that identity exchange occurs, and the track IDs of the target human bodies corresponding to the spot-check target and the comparison object are exchanged; if the matching degrees of the spot-check target and the comparison object in the comparison group are less than a preset matching degree threshold, and the track IDs of the two are the same, it is marked that the spot-check target is not a teacher target.

2. The method of claim 1, wherein, The key points include: eyes, ears, nose, hands, elbows, shoulders, hips, knees and feet.

3. The method of claim 1, wherein, The method of detecting human bodies and key points in each frame image based on a human pose estimation network comprises: for a frame image, a human pose estimation model based on YOLOv5-Pose is used for detection to obtain detection boxes of a plurality of target human bodies and a plurality of key points of each target human body; normalizing coordinate values of the detection boxes and the key points of each target human body.

4. The method of claim 1, wherein, The method of detecting blackboards, heads and faces in each frame image using a three-class target detection network comprises: For a frame of image, a three-class target detection model based on YOLOv5 is used for detection to obtain detection boxes of blackboard regions, human heads and human faces; Coordinate values of the detection boxes of the blackboard regions, the human heads and the human faces are normalized; If the blackboard regions are multiple, the multiple blackboard regions are merged.

5. The method of claim 4, wherein, The merging of the multiple blackboard regions comprises: Pixel areas of the blackboard regions are calculated, and the blackboard regions are sorted in descending order according to the pixel areas from large to small; An area proportion threshold is set, and a blackboard region with a pixel area less than the area proportion threshold of the maximum blackboard region area is removed; Bounding boxes of the remaining blackboard regions are calculated to form an overall blackboard region.

6. The method of claim 5, wherein, The removing of the blackboard region with the pixel area less than the area proportion threshold of the maximum blackboard region area comprises: The area ratio of each remaining blackboard region to the maximum blackboard region is determined; If the area ratio of a remaining blackboard region to the maximum blackboard region is less than the area proportion threshold, the remaining blackboard region is removed; otherwise, the remaining blackboard region is retained.

7. The method of claim 5, wherein, The calculation of the bounding boxes of the remaining blackboard regions to form the overall blackboard region comprises: For each remaining blackboard region, the coordinate values of the top-left corner and the bottom-right corner of the detection box are counted; The maximum and minimum values of the top-left corner coordinate values and the maximum and minimum values of the bottom-right corner coordinate values of the multiple remaining blackboard regions are calculated respectively; A bounding box is constructed based on the minimum value of the top-left corner coordinate values and the maximum value of the bottom-right corner coordinate values to form the overall blackboard region.

8. The method of claim 1, wherein, The improved multi-target tracking algorithm is realized by using an AlphaBeta filter to improve a SORT model; The tracking of the detected target human bodies based on the improved multi-target tracking algorithm comprises: Each target human body is assigned a unique trackID and a tracking time, serving as a tracking object; The AlphaBeta filter is used to associate each target human body detected in the current frame of image with existing tracking objects; If a target human body is associated with a tracking object, the trackID of the tracking object is assigned to the target human body; If a target human body is not associated with a tracking object, a new unique trackID is assigned to the target human body and a tracking time is recorded, serving as a tracking object; If a tracking object is not associated with a target human body within a preset time length, the trackID of the tracking object is discarded; Key points of each target human body in the current frame of image are updated.

9. The method of claim 8, wherein, The association of each target human body detected in the current frame of image with existing tracking objects by using the AlphaBeta filter comprises: The AlphaBeta filter is used to calculate the association degrees of each tracking object and each target human body, forming an association degree matrix; The Hungarian algorithm is used to find the optimal match based on the association degree matrix, and a minimum association degree threshold is set in the matching process; According to the optimal matching result, each target human body is associated with an existing tracking object; If there is no tracking object at present, a unique trackID and a tracking time are assigned to the latest detected target human body as a tracking object; If one of the tracking objects is not associated with a target human body, an AlphaBeta filter is used for prediction and modeling, and the modeled target human body is associated with the tracking object, and the modeled target human body is updated to the set of detected target human bodies.

10. The method of claim 9, wherein, The association degree is any one of IoU, DIoU, GIoU and CIoU.

11. The method of claim 1, wherein, The association of the detected human head and face with each target human body comprises: The improved intersection over union of each detected human head or face with the detection box of each target human body is calculated respectively; If the improved intersection over union of one human head or face with the detection box of one target human body is greater than a preset first empirical threshold, the human head or face is associated with the target human body; The improved intersection over union is expressed as formula (16): Formula (16); In the formula, A and B represent different detection frame pairs formed by detection frames; The minimum value representing the union of the plurality of detection frame pairs.

12. The method of claim 1, wherein, The association of the detected human head and face with each target human body comprises: If three specified key points of one target human body are covered by the detection box of one face, the face is associated with the target human body, and the specified key points include eyes and nose.

13. The method of claim 11, wherein, The position information of each target human body relative to the blackboard region is determined, comprising: The overlap degree of each target human body with the blackboard region is determined, and the overlap degree is represented by improved intersection over union; If the overlap degree is greater than a preset second empirical threshold, the target human body is marked as being on the podium; Otherwise, the target human body is marked as not being on the podium.

14. The method of claim 1, wherein, The preliminary screening of the teacher identity of each target human body according to the position information and the timing information obtains a teacher target, comprising: Each target human body is traversed, and for the traversed target human body, it is determined whether the target human body is on the podium or not, or whether the length of time that the target human body is on the podium is greater than a preset length threshold according to the recorded timing information; If yes, the target human body is a candidate object, and if the number of candidate objects is greater than zero, the length of time that multiple candidate objects are on the podium is sorted in descending order, and the first one is selected as a teacher target; If multiple candidate objects are in parallel, the one with the highest height is selected as a teacher target.

15. The method of claim 1, wherein, The standing posture classifier is pre-trained according to the following method: An initial model of the standing posture classifier is constructed, and the initial model is established according to any one of SVM mechanism, decision tree mechanism and shallow neural network; Sample data in a plurality of classroom scenes are obtained and sample processing is performed to construct a standing posture data set, wherein the sample processing comprises key point detection on the sample data by a human posture estimation network, and standing posture category labeling on the sample data, wherein the key points include eyes, ears, nose, hands, elbows, shoulders and hips, and the standing posture categories include front, back, left side and right side; Train an initial model of the station posture classifier based on the station posture data set to obtain the station posture classifier.

16. The method of claim 1, wherein, The step of determining at least one spot-check target in a current frame image according to the position information and the time sequence information to form a query graph comprises: If the target human body in the traversal is located on the rostrum or the time length of the target human body on the rostrum is greater than a preset time length threshold according to the recorded time sequence information, the target human body is taken as a spot-check target; The spot-checking frequency of the spot-check target in a preset time length is recorded, and if the spot-checking frequency is greater than or equal to a preset frequency threshold, the spot-check target is abandoned.

17. The method of claim 1, wherein, Based on the query graph, the cascade model is used to perform feature and identity matching between the spot-check target and each comparison object in the candidate pedestrian library to perform identity re-authentication, comprising: Based on the station posture classifier of the cascade model, the spot-check target in the query graph is classified according to the station posture; According to the station posture classification result, the matching degree of the spot-check target and each comparison object in the candidate pedestrian library is calculated; According to the matching degree, the identity of the spot-check target is re-authenticated.

18. The method of claim 17, wherein, According to the station posture classification result, the matching degree of the spot-check target and each comparison object in the candidate pedestrian library is calculated, comprising: For one comparison object, the similarity of the spot-check target and the comparison object in different station posture dimensions is calculated based on the Euclidean distance or the cosine distance; If the similarity is greater than a preset similarity threshold in the station posture dimension of the station posture classification result, the similarity is taken as the matching degree of the spot-check target and the comparison object; If the similarity is not greater than a preset similarity threshold in the station posture dimension of the station posture classification result, the similarities in each station posture dimension are fused by weighting to obtain the matching degree of the spot-check target and the comparison object.

19. The method of claim 18, wherein, The similarity of the spot-check target and the comparison object in different station posture dimensions is calculated based on the Euclidean distance or the cosine distance, comprising: The spot-check ReID feature of the spot-check target is extracted by using the pedestrian ReID feature extraction model of the cascade model; For each station posture dimension, the Euclidean similarity or the cosine similarity of the spot-check ReID feature and the ReID feature of the comparison object at different time points is calculated respectively; The maximum value of the Euclidean similarity or the cosine similarity in each station posture dimension is selected as the similarity in the station posture dimension.

20. The method of claim 17, wherein, According to the matching degree, the identity of the spot-check target is re-authenticated, further comprising: The matching degree of the spot-check target and the comparison object with the same trackID in the candidate pedestrian library is calculated or read, and if the obtained matching degree is within a preset high similarity threshold range, the spot-check ReID feature of the spot-check target is saved in the candidate pedestrian library.

21. A teacher identity authentication and tracking apparatus, characterized by, The device comprises: An acquisition unit is configured to acquire a video image stream of a classroom monitoring device; The target detection unit is configured to perform human body and key point detection on each frame of image based on a human pose estimation network, and perform blackboard, head and face detection on each frame of image by using a three-class target detection network. The tracking unit is configured to track each target human body detected based on an improved multi-target tracking algorithm, and attribute the detected head and face to each target human body. The preliminary screening unit is configured to determine position information and timing information of each target human body relative to a blackboard region, and perform preliminary screening of teacher identity on each target human body according to the position information and the timing information, to obtain a teacher target. The re-authentication unit is configured to perform identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to prevent identity exchange between a teacher and a student. The re-authentication unit is configured to construct and train a pose classifier and a pedestrian ReID feature extraction model to form the cascade model; construct a candidate pedestrian library based on the cascade model, the candidate pedestrian library including ReID features of comparison objects in different pose dimensions, wherein the pose dimensions include facing, back to back, left side and right side; determine at least one spot check target in a current frame of image according to the position information and the timing information, to form a query graph; and perform feature and identity matching between the spot check target and each comparison object in the candidate pedestrian library based on the query graph by using the cascade model, to perform identity re-authentication, including: arranging matching degrees of the spot check target and each comparison object in descending order, and selecting the spot check target and the comparison object with the first order as a comparison group; if the matching degrees of the spot check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and track IDs of the spot check target and the comparison object are the same, it is confirmed that identity re-authentication is successful, and a current track ID of the spot check target is maintained; if the matching degrees of the spot check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and track IDs of the spot check target and the comparison object are different, it is confirmed that identity exchange occurs, and track IDs of target human bodies corresponding to the spot check target and the comparison object are exchanged; and if the matching degrees of the spot check target and the comparison object in the comparison group are less than a preset matching degree threshold, and track IDs of the spot check target and the comparison object are the same, it is marked that the spot check target is not a teacher target.

22. A classroom monitoring device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the teacher identity authentication and tracking method according to any one of claims 1 to 20.

23. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the teacher identity authentication and tracking method according to any one of claims 1 to 20.

Images