Teacher identity authentication and tracking method and device, classroom monitoring equipment and medium

Abstract

The invention discloses a teacher identity authentication and tracking method and device, classroom monitoring equipment and a medium. The method comprises the following steps: acquiring a video image stream of the classroom monitoring equipment; carrying out human body and key point detection on each frame of image based on a human body posture estimation network, and carrying out blackboard, head and face detection on each frame of image by adopting a three-category target detection network; tracking each detected target human body based on an improved multi-target tracking algorithm, and assigning the detected head and face to each target human body; position information and time sequence information of each target human body relative to the blackboard area are determined, and preliminary screening of teacher identities is carried out to obtain teacher targets; and performing identity re-authentication based on a cascade model of pre-trained posture classification and pedestrian ReID feature extraction to prevent identity exchange between teachers and students. According to the method, continuous teacher identity authentication which is non-inductive, low in consumption, simple in algorithm form, high in robustness, high in credibility and extensible is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, biometric identification and education informatization, in particular to a teacher identity authentication and tracking method and device, a classroom monitoring device and a medium. BACKGROUND

[0002] In the scenarios of teaching quality monitoring, teacher attendance, teaching supervision and education big data management, the common problems encountered in the implementation of the prior art are that: (1) The traditional methods of manual roll call, card swiping or mobile terminal check-in entering face or body features need schools to formulate relevant systems under the legal framework, and their legality and effective binding are controversial, there are risks of human or system such as signing on behalf of others, forgetting, resisting, etc., and they go against the "non-intrusive" principle, have poor experience, and are not conducive to implementation.

[0003] (2) The reliability of general static face recognition algorithms significantly decreases in the classroom environment, and teachers frequently turn sideways to the blackboard, bend down to operate experimental equipment or bend over to tutor students, resulting in a low proportion of frontal face visible video frames, which greatly increases the miss rate.

[0004] (3) The actions of students standing up to ask questions, value day students wiping the blackboard, and back row patrols are easily confused with the actions of teachers. SUMMARY

[0005] To solve the above problems, the embodiments of the present application provide a teacher identity authentication and tracking method, device, classroom monitoring device and medium, aiming to provide a non-intrusive, low-consumption, simple algorithm form, high robustness, high credibility and extensible teacher identity continuous authentication method to overcome or at least partially overcome the deficiencies of the prior art.

[0006] The embodiments of the present application adopt the following technical solutions: In a first aspect, the present application provides a teacher identity authentication and tracking method, comprising: obtaining a video image stream of a classroom monitoring device; detecting human bodies and key points in each frame image based on a human pose estimation network, and detecting blackboards, heads and faces in each frame image using a three-class target detection network; tracking each detected target human body based on an improved multi-target tracking algorithm, and attributing the detected head and face to each target human body; determining the position information and time sequence information of each target human body relative to the blackboard area, and preliminarily screening the teacher identity of each target human body according to the position information and the time sequence information, to obtain a teacher target; performing identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to prevent identity exchange between teachers and students.

[0007] In a second aspect, the present application also provides a teacher identity authentication and tracking device, which comprises: an acquisition unit configured to acquire a video image stream of a classroom monitoring device; a target detection unit configured to perform human body and key point detection on each frame of image based on a human pose estimation network, and perform blackboard, head and face detection on each frame of image based on a three-class target detection network; a tracking unit configured to track each target human body detected based on an improved multi-target tracking algorithm, and attribute a detected head and face to each target human body; a preliminary screening unit configured to determine position information and timing information of each target human body relative to a blackboard region, and perform preliminary screening of teacher identity for each target human body based on the position information and the timing information, to obtain a teacher target; a re-authentication unit configured to perform identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to prevent identity exchange between a teacher and a student.

[0008] In a third aspect, the present application also provides a classroom monitoring device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned teacher identity authentication and tracking method when executing the computer program.

[0009] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above-mentioned teacher identity authentication and tracking method when being instructed by a processor.

[0010] The above-mentioned at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects: The present application first acquires a video image stream of a classroom monitoring device; performs human body and key point detection on each frame of image based on a human pose estimation network, and performs blackboard, head and face detection on each frame of image based on a three-class target detection network; then tracks each target human body detected based on an improved multi-target tracking algorithm, and further attributes a detected head and face to each target human body; then performs preliminary screening of teacher target based on the detection and tracking results, specifically determines position information and timing information of each target human body relative to a blackboard region, and performs preliminary screening of teacher identity for each target human body, to obtain a teacher target; then performs identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to effectively avoid identity exchange between a teacher and a student. The present application realizes continuous authentication of teacher identity with no sense, low consumption, simple algorithm form, high robustness, high credibility and scalability.

[0011] First, the present application automatically locates the teacher's position and identifies the teacher's identity throughout the process without manual input of teacher characteristics, which is more intelligent than traditional methods, avoids legal risks and experience barriers of traditional manual or biometric feature input methods, and realizes de-identified feature extraction based on public monitoring equipment through a pure visual passive sensing mechanism without active cooperation, which not only meets the requirements of privacy protection regulations, but also eliminates management loopholes such as proxy signing and forgetting.

[0012] Second, the present application innovatively combines teaching scene semantic understanding and multi-modal biometric recognition technology, preliminarily locates the teacher target through human posture and target space-time correlation analysis, and then uses a cascaded model of pre-trained posture classification and pedestrian ReID feature extraction for identity re-authentication, effectively overcoming the recognition failure problem caused by the teacher's frequent turning back, bowing and student activity interference.

[0013] Third, the present application realizes embedded real-time computing through lightweight design, introduces only a small amount of tolerable delay under the premise of ensuring matching accuracy without loss; at the same time, it has a high degree of flexible algorithm configuration capability, for high-performance computing platforms or video stream processing scenarios, only simple parameter adjustment is needed to seamlessly switch to the traditional ReID full-function mode, adapting to multi-level hardware ecology. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings illustrated herein are used to provide further understanding of the present application, constitute a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a machine vision-based teacher identity authentication and tracking method in a classroom observation according to one embodiment of the present application is shown; Figure 2 A schematic diagram of 17 key points according to one embodiment of the present application is shown; Figure 3 A result schematic diagram of human posture estimation for a frame of image according to one embodiment of the present application is shown; Figure 4 A result schematic diagram of blackboard region detection for a frame of image according to one embodiment of the present application is shown; Figure 5 A result schematic diagram of human head detection for a frame of image according to one embodiment of the present application is shown; Figure 6 A result schematic diagram of human face detection for a frame of image according to one embodiment of the present application is shown; Figure 7 A schematic diagram of merging multiple blackboard regions to obtain the overall blackboard region according to one embodiment of the present application is shown; Figure 8 A schematic diagram of a process for tracking detected human targets based on an improved multi-target tracking algorithm according to an embodiment of this application is shown. Figure 9-a This represents the intersection of two rectangular detection boxes in a set of detection boxes. Figure 9-b This represents the union of two rectangular detection boxes in a set of detection boxes; Figure 10 A schematic diagram of four standing postures of a pedestrian according to an embodiment of this application is shown; Figure 11 A schematic diagram of the comparison objects stored in a candidate pedestrian database according to an embodiment of this application is shown; Figure 12 A schematic diagram illustrating identity re-authentication using a cascaded model according to an embodiment of this application is shown; Figure 13 A schematic diagram illustrating identity re-authentication using a cascaded model according to another embodiment of this application is shown; Figure 14 This diagram illustrates the structure of a teacher identity authentication and tracking method apparatus according to an embodiment of the present application. Figure 15 A schematic diagram of the structure of a classroom monitoring device according to an embodiment of this application is shown. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To enable those skilled in the art to more clearly understand the technical solutions provided in the various embodiments of this application, the technical concept of this application will first be described.

[0017] This application aims to provide a non-intrusive, low-consumption algorithm that can be deployed on low-power embedded platforms, especially with recording and broadcasting equipment equipped with cameras or other forms of AIoT edge devices, to achieve highly robust, highly reliable, and scalable continuous authentication of teacher identity in classroom observation scenarios, providing a reliable foundation for the correlation analysis of teaching behaviors.

[0018] Figure 1 This application illustrates a method for teacher authentication and tracking according to an embodiment of the present application, from... Figure 1 As can be seen, this embodiment includes steps S110 to S150: Step S110, obtaining a video image stream of the classroom monitoring device.

[0019] The present application is mainly applied to a classroom observation scene and is realized based on machine vision technology. The classroom monitoring device can be, but is not limited to, a camera or a camera with a camera function, which is installed at a proper position in a classroom.

[0020] The present application can be used for authentication and tracking of a teacher in a video image stream in a historical time, and can also be used for real-time monitoring of a classroom situation, which can read a real-time video image stream captured by the classroom monitoring device. It is recommended that the resolution of the video image stream can be, but is not limited to, 1280x720.

[0021] Step S120, performing human body and key point detection on each frame image based on a human body pose estimation network, and performing blackboard, head and face detection on each frame image by using a three-class target detection network.

[0022] A human body pose estimation network or model is constructed and trained in advance, which can be based on any network structure, and can simultaneously detect human body detection boxes of multiple human bodies in a single picture and multiple key points of each human body. In some embodiments of the present application, the multiple key points recommended include, but are not limited to, two eyes, two ears, a nose, two hands, two elbows, two shoulders, two hips, two knees, and two feet, a total of 17 key points, which can be referred to in detail in Figure 2 . Figure 2 A schematic diagram of 17 key points according to one embodiment of the present application is shown.

[0023] Specifically, in some embodiments of the present application, for each frame image, it is recommended to use a human body pose estimation model constructed based on YOLOv5-Pose to perform detection, to obtain a plurality of target human body detection boxes and a plurality of key points of each target human body. It should be noted that for the key points that are covered, the human body pose estimation model can be obtained by pre-estimation. The detection box of each target human body and its key points can be represented by a 38-dimensional vector, as shown in formula (1): Formula (1); wherein, are the coordinates of the top left corner of the human body rectangular detection box (which can also be the center point of the human body target box), are the length and width of the rectangular detection box; are the coordinates of the 17 key points.

[0024] The coordinates of the detection box and the key points of each target human body are normalized. The coordinates are normalized by dividing the horizontal value by the image width and the vertical value by the image height, so that the value range is [0, 1).

[0025] A three-class target detection network or model can be constructed and trained in advance, which can detect all blackboard regions and all human heads and faces in a single picture based on any network structure.

[0026] Specifically, in some embodiments of the present application, for a frame of image, a three-class target detection model based on YOLOv5 is recommended to be used for detection to obtain a plurality of detection boxes of blackboard regions, human heads and faces.

[0027] Each blackboard region is represented by the following vector, as shown in equation (2): Equation (2); Each human head detection box is represented by the following vector, as shown in equation (3): Equation (3); Each human face detection box is represented by the following vector, as shown in equation (4): Equation (4); Similarly, x b , y b ; x h , y h ; x f , y f are the coordinates of the top-left corner of the rectangular detection box, w b , h b ; w h , h h ; w f , h f are the length and width of the rectangular detection box.

[0028] Similarly, the coordinate values are normalized to have a value range of [0, 1).

[0029] In each frame of image, the above human pose estimation and three-class target detection processing are performed to obtain the human pose set of the target human body and the target result set.

[0030] The human pose set of a plurality of target human bodies in a frame of video can be represented by equation (5), as shown in Figure 3 , Figure 3This diagram illustrates the result of human pose estimation on a frame of image according to an embodiment of this application. The detected target human body includes X0-X1. k : Equation (5); The set of blackboard regions can be represented by equation (6), please refer to it. Figure 4 , Figure 4 This diagram illustrates the result of blackboard region detection on a frame of image according to an embodiment of this application. Figure 4 The document includes a blackboard area detection frame. The blackboard area includes B0-B1. k : Equation (6); The set of heads can be represented by equation (7), please refer to it. Figure 5 , Figure 5 This diagram illustrates the result of head detection on a frame of image according to an embodiment of this application, where the heads include H0-H0. k : Equation (7); The set of faces can be represented by equation (8), please refer to it. Figure 6 , Figure 6 This diagram illustrates the result of face detection on a frame of image according to an embodiment of this application, where the face includes F0-F1. k : Equation (8); Similarly, the coordinate values ​​of the detection boxes for the blackboard area, the head, and the face are normalized to a range of [0,1].

[0031] It should be noted that k in equations (5) to (8) represents the quantity. k is an independent parameter in each formula, and its value is determined based on the actual detection situation of each frame.

[0032] exist Figure 4 In the scenario shown, there are multiple blackboard areas. If there are multiple blackboard areas, then these multiple blackboard areas are merged to obtain a single, unified blackboard area.

[0033] Specifically, in some embodiments of this application, merging multiple blackboard regions includes: calculating the pixel area of ​​each blackboard region and sorting them in descending order of pixel area; setting an area ratio threshold and removing detection boxes whose pixel area is less than the area ratio threshold of the largest blackboard region; and obtaining bounding boxes for the remaining blackboard regions to form an overall blackboard region.

[0034] The removing the detection frame with a pixel area less than the area proportion threshold of the maximum blackboard area includes: determining an area ratio of each remaining blackboard region to the maximum blackboard region based on the maximum blackboard region; if the area ratio of one remaining blackboard region is less than the area proportion threshold, the remaining blackboard region is removed; otherwise, the remaining blackboard region is retained.

[0035] The bounding box of the retained blackboard region is calculated to form an overall blackboard region, and specifically includes: for each retained blackboard region, the top-left corner and the bottom-right corner are counted; the maximum and minimum values of the top-left corner and the bottom-right corner of the multiple blackboard regions are calculated respectively; the bounding box is constructed based on the minimum value of the top-left corner and the maximum value of the bottom-right corner to form an overall blackboard region.

[0036] In an actual scene, the blackboard may be divided into multiple regions, such as Figure 4 Therefore, the blackboard regions need to be merged, and the following steps can be used to merge the blackboard regions: First, the pixel area of each blackboard region is calculated by formula (9): Formula (9); S i represents the pixel area of the i-th blackboard region.

[0037] Second, the pixel areas are sorted in descending order, as shown in formula (10): Formula (10).

[0038] Third, for each element in the set, the pixel area ratio of the element to the maximum blackboard region is calculated, as shown in formula (11): Formula (11).

[0039] Fourth, the area proportion threshold is set to T B If , the blackboard region corresponding to the condition is removed from the detection result, and the area proportion threshold T B is an empirical value, and the recommended value is 0.20. If the area of the currently detected blackboard region is less than one-fifth of the maximum blackboard region, it is likely to be a false detection, and it needs to be excluded. Therefore, the set of removed false detection regions is obtained, as shown in formula (12): Formula (12); Fifth, for B ​n The upper left and lower right vertex coordinates of the rectangle of each element in the set are calculated, and are expressed as a vector of formula (13): Formula (13).

[0040] Sixth, the maximum and minimum values of are calculated, as listed below: .

[0041] Seventh, a new blackboard area bounding box is constructed using the maximum and minimum values, forming the overall blackboard area B minmax , which can be referenced Figure 7 , Figure 7 A diagram showing the merging of multiple blackboard areas to obtain the overall blackboard area according to one embodiment of the present application is shown in the red rectangular frame in the figure, and the formula is shown in formula (14): Formula (14).

[0042] In step S130, each detected target human body is tracked based on the improved multi-target tracking algorithm, and the detected human head and face are attributed to each target human body.

[0043] Multi-target tracking is a key component in video analysis, which can maintain a unique trackID for each detected target in a sequence of video frames.

[0044] The popular multi-target tracking framework is the SORT model. SORT (Simple Online and Realtime Tracking, ICIP16, A. Bewley et al.) is a mainstream Tracking-by-Detection framework. Its basic idea is to detect first and track second, and it advocates that if the frame rate is high enough and the detection effect is good enough, the intersection over union (IoU) between the detection boxes of each target in the previous and subsequent frames can be used as a strong basis for association.

[0045] In some embodiments of the present application, the SORT model is improved. Mainly reflected in, first, the AlphaBeta filter is used instead of the existing Kalman filter; second, the specific algorithm of the intersection over union is improved; third, in the tracking, not only the detection box of the target is tracked, but also the 17 key points of the target are tracked.

[0046] Specifically, in some embodiments, the improved multi-target tracking algorithm is a SORT model implemented by using an AlphaBeta filter; the improved multi-target tracking algorithm is used to track each detected target human body, including: assigning a unique trackID to each detected target human body and recording a tracking time as a tracking object; using an AlphaBeta filter to associate each detected target human body in a current frame image with existing tracking objects; if a target human body is associated with a tracking object, the trackID of the tracking object is assigned to the target human body; if a target human body is not associated with a tracking object, a new unique trackID is assigned to the target human body and a tracking time is recorded as a tracking object; if a tracking object is not associated with a target human body within a preset time length, the trackID of the tracking object is discarded; and updating the key points of each target human body in the current frame image.

[0047] The tracking idea is: first detection-then association. As described above, if the current frame is an initial frame image, for each detected target human body (denoted as a target human body) and its key points in the set of detected target human bodies in the frame, a unique trackID is assigned to each target human body, and the trackID does not change over time. To this end, the vector representing the target human body is expanded to obtain the vector shown in equation (15). Equation (15); Equation (15); where z represents the trackID of the target human body, and t represents the tracking time.

[0048] Each target human body assigned with a unique trackID is taken as a tracking object.

[0049] As described above, if the current frame is the first frame, five target human bodies are detected, and unique trackIDs are assigned to the five target human bodies, denoted as A1, A2, A3, A4, and A5, that is, the existing tracking objects are A1, A2, A3, A4, and A5.

[0050] Then, an AlphaBeta filter is used to associate each detected target human body in the current frame image with the existing tracking objects.

[0051] Then, the second frame is reached, and it is known through the detection of step S120 that there are several target human bodies in the second frame image, and an AlphaBeta filter is used to associate the target human bodies with the existing tracking objects.

[0052] ​​Specifically, in some embodiments of the present application, an AlphaBeta filter is used to associate each target human body detected in a current frame image with a plurality of existing tracking objects, including: using an AlphaBeta filter to calculate the association degree of each tracking object and each target human body, forming an association degree matrix; using the Hungarian algorithm to find the optimal matching based on the association degree matrix, setting a minimum association degree threshold in the matching process; according to the optimal matching result, associating each target human body with a plurality of existing tracking objects; if there is no tracking object at present, assigning a unique trackID to the latest detected target human body and recording the tracking time as a tracking object; if a tracking object is not associated with a target human body, using an AlphaBeta filter to predict and model, and associating the modeled target human body with the tracking object, and updating the modeled target human body to the set of detected target human bodies.

[0053] Then come to the second frame, through the detection of step S120, calculate the association degree of each existing tracking object and the target human body detected in the frame, form an association degree matrix, assuming that four target human bodies are detected in the second frame image, respectively denoted as B1, B2, B3, B4, then form the association degree matrix shown in Table 1.

[0054] Table 1

[0055] Using the Hungarian algorithm to find the optimal matching based on the association degree matrix, setting a minimum association degree threshold in the matching process.

[0056] Set a minimum association degree threshold, if the association degree of a tracking object and a target human body is less than the minimum association degree threshold, mark that the matching of the two is unacceptable, thereby performing the preliminary screening of matching.

[0057] Then use the Hungarian algorithm to find the optimal matching of tracking objects and target human bodies, and match tracking objects for each target human body. After matching, the matching result is obtained, which usually contains the following several cases.

[0058] If a target human body and a tracking object match successfully, associate the two, and assign the trackID of the tracking object to the target human body, as shown in Table 1, assuming that tracking object A1 and target human body B1 match successfully, and the trackID of tracking object A1 is recorded as ID0, then assign ID0 to target human body B1.

[0059] If a target human body is not associated with a tracking object, a new unique trackID is assigned to it, its tracking time is recorded, and it is also taken as a tracking object. Assuming that there are 5 tracking objects in the first frame and 6 target human bodies in the second frame, and each tracking object in the first frame has been matched with 5 of the target human bodies in the second frame, it is indicated that a new target human body appears in the second frame. In this case, a new trackID is assigned to the target human body that is not associated with the tracking object, that is, a new unique trackID, and its tracking time is recorded, and it is also taken as a tracking object.

[0060] If a tracking object is not associated with a target human body, an AlphaBeta filter is used for prediction and modeling, the modeled target human body is associated with the tracking object, and the modeled target human body is updated to the set of detected target human bodies. As shown in Table 1, assuming that tracking object A1 and target human body B1 are successfully matched, tracking object A2 and target human body B2 are successfully matched, tracking object A3 and target human body B3 are successfully matched, tracking object A4 and target human body B4 are successfully matched, and tracking object A5 is not matched with a target human body. This situation can be caused by many reasons, such as tracking object A5 walking out of the current frame, or tracking object A5 being blocked by other human bodies or objects, but in the next frame, it can return to the current frame or the blockage is removed. In this case, an AlphaBeta filter is used for prediction and modeling, the modeled target human body is associated with tracking object A5, and it is updated to the set of detected target human bodies.

[0061] Specifically, the SORT model can use the Kalman filter to model the target human body in this frame based on the center coordinates, area, aspect ratio, and other measurement values, and use the position information of the target human body to constantly update its state. The modeled target human body is associated with tracking object A5. It is recommended that an interpolation method be used for modeling, that is, the center coordinates, area, aspect ratio, and other measurement values of the target in the first frame and the third frame are used for modeling.

[0062] If a tracking object is not associated with a target human body, it will not be processed by modeling all the time. The modeling method is only for the short absence of the target human body. If a target human body does not appear for a long time, it means that it has left the frame. Therefore, if a tracking object is not associated with a target human body within a preset time period, the trackID of the tracking object is discarded.

[0063] After each association, the key points of each target human body in the current frame image are updated. One aspect in which the present application differs from the multi-target tracking technology of the prior art is that the present application not only tracks the detection box, but also tracks and updates the key points. In addition to the detection box (4 quantities), there are 17 key points (34 quantities) that need to participate in tracking, a total of 38 quantities. Compared with the traditional technology of only tracking the detection box, the present application has a significant advantage. This mechanism can enhance the target appearance expression, improve the ability to distinguish similar targets under occlusion, optimize the data association accuracy, reduce ID switching and track fragmentation, especially in crowded or dynamic scenes such as classrooms, which can improve the tracking robustness and positioning accuracy, and provide more reliable target motion and posture information for downstream tasks such as behavior analysis.

[0064] The above representation of the association degree can be any one of the intersection-over-union IoU or its variants DIoU, GIoU, and CIoU.

[0065] Figure 8 A flowchart showing tracking of each detected target human body based on an improved multi-target tracking algorithm according to an embodiment of the present application is shown. From the flowchart, it can be seen that the present embodiment includes the following steps: Figure 8 As can be seen, the present embodiment includes the following steps: First, a plurality of target human bodies in the current frame image and a plurality of existing tracking targets are obtained.

[0066] Subsequently, it is determined whether the number of current tracking targets is zero. If the number is zero, a unique trackID is assigned to each target human body and the tracking time is recorded as a tracking object, and the next frame is entered.

[0067] If the number is not zero, the association degree of each tracking object and each target human body is calculated to form an association degree matrix, and then the Hungarian algorithm is used to find the optimal match based on the association degree matrix, and each target human body is associated with the existing tracking objects according to the optimal match result.

[0068] Subsequently, for each target human body, it is determined whether it is associated with a tracking object. If it is associated, the trackID of the associated tracking object is assigned to the target human body. If it is not associated, a new unique trackID is assigned to the target human body and used as a new tracking object.

[0069] After the above steps are completed, it is determined whether there is a tracking object that is not associated with a target human body within a predetermined time period. If there is, an AlphaBeta filter is used to predict and model the tracking object that is not associated with a target human body, associate the modeled target human body with the tracking object, and update it to the set of detected target human bodies.

[0070] If not, update the key points of each target human body in the current frame image; finally, enter the next frame to repeat the above process.

[0071] In the above embodiment, the AlphaBeta filter is used to replace the Kalman filter in the original SORT tracker, thereby reducing the computational complexity, which is the preferred recommended technology, and other lightweight filters can also be replaced.

[0072] In order to further improve the accuracy and robustness of detection, the detected human head and face are attributed to each target human body. Specifically, it includes: calculating the improved intersection over union of each detected human head or face and the detection box of each target human body respectively; if the improved intersection over union of a human head or face and a target human body is greater than a first empirical threshold, the human head or face belongs to the target human body; Wherein, the improved intersection over union can be expressed as formula (16): Formula (16); In the formula, A and B represent different detection box pairs; The minimum value of the union of a plurality of detection box pairs.

[0073] In some embodiments of the present application, the existing intersection over union definition is improved, and based on the improved intersection over union, the detected human head and face in the original result are associated with each target human body.

[0074] Please refer to Figure 9-a and Figure 9-b , two rectangular detection boxes form a detection box pair, Figure 9-a represents the intersection of the two rectangular detection boxes in the detection box pair, Figure 9-b represents the union of the two rectangular detection boxes in the detection box pair, and the existing intersection over union can be expressed as formula (17): Formula (17).

[0075] In the embodiments of the present application, the existing intersection over union is improved, and the foregoing formula (16) can be referred to, represents the minimum value in the union of a plurality of detection box pairs.

[0076] For human head H i , for each H i , calculate the improved intersection over union X i of , when , associate it to the X i , so as to extend it into the vector shown in formula (18): Formula (18); wherein, t h is an empirical value, denoted as a first empirical threshold, and is recommended to be 0.90-0.95.

[0077] For a face, the embodiments of the present application can use any one of the following two methods for association.

[0078] The first method can refer to the association method of the head as described above to calculate When , it is associated with X i .

[0079] The second method, if three specified key points of a target human body are covered by a face detection frame, the face is attributed to the target human body. That is, the 0th, 1st, and 2nd key points of the pedestrian (such as Figure 2 ) are used, and if the three key points are completely contained in the face detection frame of a target human body, it is considered that the face belongs to the target human body.

[0080] In step S140, the position information and the timing information of each target human body relative to the blackboard region are determined, and the preliminary screening of the teacher identity is performed on each target human body according to the position information and the timing information, to obtain a teacher target.

[0081] The teacher target is preliminarily confirmed according to the position information and the timing information of the target human body relative to the blackboard region.

[0082] The position information of the target human body relative to the blackboard region can be calculated according to the following method: the overlap degree of each target human body and the blackboard region is determined, and the overlap degree can be represented by the improved intersection-over-union; if the overlap degree is greater than a preset second empirical threshold, the target human body is marked as being on the podium; otherwise, the target human body is marked as not being on the podium.

[0083] For each target human body X i , the overlap degree with the blackboard region is calculated, such as the improved intersection-over-union of the two as a measure, when , it is considered that the target human body is not on the podium, and the vector representing the target human body is extended to the vector shown in Formula (19): Formula (19); wherein b is a mark of whether the pedestrian is on the podium, and the value of 0 indicates that the pedestrian is not on the podium, and the value of 1 indicates that the pedestrian is on the podium.

[0084] Timing information from the vector representing the target human body X i The parameter t in the vector can be known.

[0085] According to the position information and the timing information, a preliminary screening of the teacher identity is performed on each target human body, and a teacher target is obtained. Specifically, in some embodiments of the present application, the following steps are included: traversing each target human body, for a target human body, determining whether the target human body is located on a podium, or according to the recorded timing information, determining whether the time length of the target human body located on the podium is greater than a preset time length threshold, if yes, the target human body is taken as a candidate object, if the number of candidate objects is greater than zero, the time length of the plurality of candidate objects located on the podium is sorted in descending order, and the first one is selected as the teacher; if a plurality of candidate objects are in the first place, the pedestrian detection frame with the minimum Y-axis value is selected as the teacher target.

[0086] traversing each X i , X i The vector is shown in formula (19), and the target human body with b=1 and T s >60 (60 is a preset threshold and can be modified) is screened out, wherein the definition of T s can be understood as the time length of a target human body located on the podium, and if the target human bodies with the same trackID continuously locate on the podium and the time length is greater than the preset 60, the target human bodies are taken as a candidate object.

[0087] If the number of candidate objects is zero, the foregoing steps S110-S130 are returned to continue detection and observation.

[0088] If the number of candidate objects is one, the candidate object is directly taken as the teacher target, and the standard is performed.

[0089] If the number of candidate objects is more than one, the time length of the plurality of candidate objects located on the podium is sorted in descending order, and the first one is selected as the teacher.

[0090] If a plurality of candidate objects are in the first place in the sorting result, that is, the time length of the plurality of candidate objects located on the podium is equal and is the maximum, the detection frame with the highest value is selected as the teacher, and in the embodiments of the present application, the pedestrian detection frame with the minimum Y-axis (y p ) value is selected as the teacher. The specific selection method and the coordinate setting are related, and the core of the present step is to select the detection frame with the highest value, that is, the tallest person as the teacher target.

[0091] The teacher target is marked.

[0092] Step S150, based on the pre-trained posture classification and pedestrian ReID feature extraction cascade model, identity re-authentication is performed to prevent the teacher and the student from identity exchange.

[0093] In actual classroom observation, the teacher does not always stay in the podium area, and the situation of the student writing on the blackboard or solving problems on the podium is also common. At this time, the teacher and the student are prone to identity exchange ID-Switch, that is, when the detection boxes of the teacher and the student overlap, their trackIDs are exchanged with a certain probability, and the corresponding T s Also be exchanged, causing the teacher identity misidentification. Therefore, the present application provides an identity re-identification strategy, which aims to correct the ID-Switch.

[0094] Specifically, in some embodiments of the present application, based on the pre-trained posture classification and pedestrian ReID feature extraction cascade model, the teacher target is re-authenticated, including: constructing and training a posture classifier and a pedestrian ReID feature extraction model to form the cascade model; based on the posture classifier and the pedestrian ReID feature extraction model, a candidate pedestrian library is constructed, and the pedestrian ReID features of the comparison objects in different posture dimensions are included in the candidate pedestrian library, wherein the posture dimensions include: front, back, left side and right side; at least one spot check target is determined in the current frame image according to the position information and the time sequence information, forming a query graph; based on the query graph, the cascade model is used to perform identity matching between the spot check target and each comparison object in the candidate pedestrian library to perform teacher identity re-authentication.

[0095] In the prior art, the pedestrian RelD feature does not distinguish the four postures of the pedestrian. In fact, the features of the pedestrian in the four postures will be different. The present application classifies the RelD features of the comparison objects in the pedestrian candidate library according to the postures, thereby improving the robustness of the re-identification.

[0096] One, construction of the cascade model In the embodiments of the present application, the posture classifier and the pedestrian ReID feature extraction model are pre-trained, and the two are used in cascade to form a cascade network of the posture classifier and the pedestrian ReID feature extraction model, which continuously and re-authenticates the teacher target to prevent the teacher and the student from identity exchange.

[0097] In some embodiments of the present application, the station posture classifier is pre-trained according to the following method: an initial model of the station posture classifier is constructed, which can be established according to any one of an SVM mechanism, a decision tree mechanism, and a shallow neural network; sample data in a classroom scene are obtained and sample processing is performed to construct a station posture data set, wherein the sample processing includes: key point detection on the sample data by using a human pose estimation network, and posture category labeling on the sample data, wherein the key points include: both eyes, both ears, a nose, both hands, both elbows, both shoulders, and both hips, and the posture categories include: facing, back to, left side, and right side; the initial model of the station posture classifier is trained based on the station posture data set to obtain the station posture classifier.

[0098] First, an initial model of the station posture classifier is established based on any one of an SVM mechanism, a decision tree mechanism, and a shallow neural network. In some embodiments of the present application, the SVM mechanism is recommended, which has excellent processing ability for high-dimensional data (such as human key point coordinates), can effectively capture complex posture patterns, can guarantee good generalization in the case of small samples, avoid overfitting, and has strong noise robustness and strong classification stability.

[0099] The station posture classifier is a four-class classifier, and its categories include: facing, back to, left side, and right side, as shown in the schematic diagram Figure 10 , Figure 10 The schematic diagram of four station postures of a pedestrian is shown according to an embodiment of the present application.

[0100] In constructing the station posture data set used for training, a classroom video is collected, 17 key points of a teacher target in the classroom are extracted by using the aforementioned human pose estimation network such as YOLOv5-Pose, and the key points of the legs and feet (such as the key points No. 13, 14, 15, and 16 in Figure 2 ) are removed, and the remaining 13 key points are reserved. The remaining key points are labeled, and the labels correspond to four different station postures of the teacher in the classroom: facing, back to, left side, and right side, to form a set of station posture data set.

[0101] The initial model of the station posture classifier is trained by using the station posture data set to obtain the station posture classifier.

[0102] When the pre-trained station posture classifier is used to classify , after obtaining the prediction result, the is expanded into a vector shown in equation (20): Equation (20); wherein c is the station posture of the pedestrian.

[0103] A pedestrian ReID feature extraction model also needs to be trained in advance. It is recommended that the ReID feature extraction model be constructed based on MobileFaceNet, which is trained using real teacher scene data. The input of the pedestrian ReID feature extraction model is a color picture, such as a 256x256x3 picture, and the output is a multi-dimensional feature vector, such as a 512-dimensional feature vector. Taking a target human body in an image as an example, the ReID feature extraction model can extract multiple groups of features for re-identification, denoted as ReID features, in a very short time at different times. The ReID feature is a highly refined, robust and highly discriminative vector representation, and its core is to cross time and space and view angle changes and accurately capture the uniqueness of individual identity.

[0104] The trained posture classifier and the pedestrian ReID feature extraction model are used jointly to form a cascade model.

[0105] Second, construction of a candidate pedestrian library (Gallery sets) A set of candidate pedestrian libraries (Gallery sets) can be constructed using the above cascade model. Taking a target human body in an image as an example, the posture classifier is used to classify its posture, and the pedestrian ReID feature extraction model is used to extract its pedestrian ReID features under each posture, to obtain a candidate pedestrian library as shown in Figure 11 It can be seen from Figure 11 that for a target, such as pedestrian 0, the candidate pedestrian library contains multiple groups of pedestrian ReID features (feature vector 0, feature vector 1,...) of the target at different times in four different posture dimensions. The same applies.

[0106] It should be noted that the candidate pedestrian library can be updated according to the detection results. When a target is detected and not included in the candidate pedestrian library, it can be added to the candidate pedestrian library at any time.

[0107] The candidate pedestrian library considers posture and has an updating mechanism, which can improve the robustness of the ReID feature on the one hand, and can improve the recall rate on the other hand.

[0108] Third, selecting a spot check target Unlike existing technologies, the present application uses a spot check method when performing identity re-verification. Specifically, at least one spot check target is determined in the current frame image according to the position information and the time sequence information, and a query set (Query set) is formed.

[0109] Specifically, in some embodiments of the present application, the determining at least one spot-check target in the current frame image according to the position information and the time sequence information to form a query graph comprises: traversing each target human body, and if the target human body is located on the podium, or if it is determined according to the recorded time sequence information that the target human body is located on the podium for a time length greater than a preset time length threshold, the target human body is taken as a spot-check target; and recording the number of spot checks of the spot-check target within a preset time length, and if the number of spot checks is greater than or equal to a preset number threshold, the spot-check target is abandoned.

[0110] traversing each of X i , X i The vector of b is shown in formula (19), and the target human body with b = 1 or T s > 60 (60 is a preset threshold and can be modified) is selected as a spot-check target, wherein T s has the same definition as before. That is, the human body located on the podium in the current frame or the human body not located on the podium but staying on the podium for more than a preset time length threshold (such as 60 seconds) is selected as a candidate target to be spot-checked, and is recorded as a spot-check target.

[0111] For a spot-check target, each time it is spot-checked, it is recorded, so that the number of spot checks of the spot-check target within a preset time length (such as 5 seconds) can be obtained, and if the number of spot checks within the preset time length is greater than or equal to a preset number threshold (such as 2 times), the spot-check target is abandoned. In this way, the target with a number of spot checks reaching 2 times within 5 seconds can be filtered out, and the same human body being spot-checked multiple times within a short time is avoided.

[0112] If the number of spot-check targets is still multiple after screening, random spot-checking can be used to further screen them, and finally not less than one spot-check target is retained.

[0113] The present application filters a small number of spot-check targets for feature comparison by setting a spot-check condition, compared with the way of feature comparison of all targets one by one in the prior art, solves the problems of long identity re-authentication time and large overall computing power consumption in the case of a large number of people in a classroom, is more suitable for a classroom supervision which is an embedded platform with low computing power, and is more practical. The mechanism of further screening spot-check targets in candidate targets reduces the number of feature comparisons in single identity re-authentication, controls the identity re-authentication time consumption and saves computing power, thereby avoiding blocking or affecting other algorithm modules.

[0114] Four, identity re-authentication Based on the query graph, the cascade model is used to perform feature and identity matching between the spot check target and each comparison object in the candidate pedestrian library to perform teacher identity re-authentication. Through the above trained cascade model, the query graph is taken as the verification object, and each comparison object in the candidate pedestrian library is taken as the reference, and the identity re-authentication can be performed.

[0115] Specifically, in some embodiments of the present application, based on the query graph, the cascade model is used to perform identity matching between the spot check target and each comparison object in the candidate pedestrian library to perform teacher identity re-authentication, comprising: based on the stance classifier of the cascade model, performing stance classification on the spot check target in the query graph; according to the stance classification result, calculating the matching degree of the spot check target and each comparison object in the candidate pedestrian library; according to the matching degree, performing teacher identity re-authentication on the spot check target.

[0116] First, for a spot check target k, the stance classifier is used to perform stance classification on the spot check target in the query graph, and the obtained stance classification result is one of front, back, left side and right side.

[0117] Then, according to the stance classification result, the matching degree of the spot check target and each comparison object in the candidate pedestrian library is calculated. Specifically, for one comparison object, based on the Euclidean distance or the cosine distance, the similarity of the spot check target and the comparison object in different stance dimensions is calculated; if the similarity in the stance dimension of the stance classification result is greater than a preset similarity threshold, the similarity is taken as the matching degree of the spot check target and the comparison object; if the similarity in the stance dimension of the stance classification result is not greater than the preset similarity threshold, the similarities in each stance dimension are fused by weighting to obtain the matching degree of the spot check target and the comparison object.

[0118] Among them, the similarity of the spot check target and the comparison object in different stance dimensions is calculated based on the Euclidean distance or the cosine distance, comprising: using the pedestrian ReID feature extraction model of the cascade model to extract the spot check ReID feature of the spot check target; for each stance dimension, the Euclidean similarity or the cosine similarity of the spot check ReID feature and the ReID feature of the comparison object at different times is calculated respectively; the maximum value of the Euclidean similarity or the cosine similarity in each stance dimension is selected as the similarity in the stance dimension.

[0119] First, the pedestrian ReID feature extraction model of the cascade model is used to extract the ReID feature of the spot check target, denoted as spot check ReID feature. The spot check ReID feature of the spot check target can be the same as the dimension of the ReID feature stored in the candidate pedestrian library, or it can be different. In some embodiments of the present application, a 128-dimensional feature vector is recommended, such as the spot check ReID feature v of the spot check target, which represents the dimension v = [v0, v1, v2, …, v 127 ]. Take the cosine distance as an example for illustration. For a comparison object, the cosine similarity of the spot check target and the comparison object in four posture dimensions is calculated. In each posture dimension, multiple values are obtained. Specifically, the more groups of ReID features, the more cosine similarities. Assuming that in one posture dimension, 5 groups of ReID features are extracted for the comparison object (in practice, it is much more than 5 groups, which is only used for illustrative purposes here), which are feature vector 0, feature vector 1, feature vector 2, feature vector 3, and feature vector 4. The cosine similarity matrix of one spot check target and one comparison object is shown in Table 2: Table 2

[0120] For each posture dimension, the maximum value (Top1) of the Euclidean similarity or cosine similarity in one posture dimension is selected as the similarity in that posture dimension. Assuming that the value of cosine similarity v1-0 front is the maximum in the front posture dimension, then the cosine similarity v1-0 front is selected as the similarity of the spot check target and the comparison object in the front posture dimension. For a group of spot check targets and comparison objects, the similarity in each posture dimension is obtained under four posture dimensions, forming a matching feature vector D k , which can be represented as formula (21): Formula (21); wherein, d 0~ d 3 represents the similarity of the spot check target and the comparison object in four posture dimensions, and the subscripts 0~3 represent different posture dimensions.

[0121] Then the final matching degree is calculated. Specifically, if the similarity is greater than the preset similarity threshold in the posture dimension of the posture classification result, the similarity is taken as the matching degree of the spot check target and the comparison object; if the similarity is not greater than the preset similarity threshold in the posture dimension of the posture classification result, the similarities in each posture dimension are weighted and fused to obtain the matching degree of the spot check target and the comparison object. Please refer to formula (22): Formula (22); d iwherein different dimensions of the standing posture are represented, and c represents the standing posture classification result.

[0122] The physical meaning of formula (22) is that if the matching degree of the current standing posture exceeds the preset similarity threshold 0.5 (0.5 is an empirical threshold value, which can be set according to actual needs), the similarity of the current standing posture is directly used as the final matching degree, otherwise the average value of the similarities of the four standing postures is used as the final matching degree.

[0123] Finally, according to the matching degree, the teacher identity re-authentication of the spot-check target is performed.

[0124] Specifically, the matching degrees of the spot-check target and each comparison object are arranged in descending order, and the spot-check target and the comparison object with the first order are selected as a comparison group; if the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the trackIDs of the two are the same, it is confirmed that the identity re-authentication is successful, and the current trackID of the spot-check target is maintained; if the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the trackIDs of the two are different, it is confirmed that identity exchange occurs, and the trackIDs of the target human bodies corresponding to the spot-check target and the comparison object are exchanged; if the matching degrees of the spot-check target and the comparison object in the comparison group are less than a preset matching degree threshold, and the trackIDs of the two are the same, the spot-check target is marked as not being a teacher target.

[0125] The matching degrees of the spot-check target and each comparison object are arranged in descending order, and a group of the spot-check target and the comparison object with the first order is selected as a comparison group. The features and identities of the comparison group are matched, mainly including the following three cases: The first case is that the matching degrees of the two are greater than or equal to a preset matching degree threshold (such as 0.5, which is an empirical threshold value, and can be set according to actual needs), and the trackIDs of the two are the same, so it is confirmed that the identity re-authentication is successful, and the current trackID of the spot-check target is maintained.

[0126] Since the spot-check target determined through the foregoing screening has a high probability of being a teacher target, the spot-check target can be the teacher himself or a student who stops on the podium for a moment. By judging whether the spot-check target has identity exchange, the identity exchange between the teacher and the student can be greatly avoided.

[0127] For the comparison group ranked first in matching degree, first determine that the matching degree of both is greater than or equal to a preset matching degree threshold, the preset matching degree threshold is an empirical value, and a recommended value is 0.5. If the matching degree of one spot check target and one comparison object is greater than the matching degree threshold, it is indicated that the physiological characteristics of both are very similar, and they should be the same person. Then, it is judged whether the trackIDs of both are the same. Assuming that the trackID of the spot check target is "pedestrian 0", and the trackID of the comparison object is also "pedestrian 0", if the trackIDs of both are the same, it is indicated that the spot check target with the trackID "pedestrian 0" and the comparison object with the trackID "pedestrian 0" in the candidate pedestrian library are the same person, and the spot check target does not have trackID exchange. At this time, it is confirmed that the identity reauthentication is successful, and the current trackID of the marked spot check target is maintained.

[0128] In the second case, the matching degree of the spot check target and the comparison object in the comparison group is greater than or equal to the preset matching degree threshold, but the trackIDs of both are not the same. It is confirmed that the spot check target has identity exchange, and the trackIDs of the target bodies corresponding to the spot check target and the comparison object are exchanged.

[0129] As above, if the matching degree of one spot check target and one comparison object is greater than the matching degree threshold, it is indicated that the physiological characteristics of both are very similar, and they should be the same person. However, it is known through judgment that the trackIDs of both are not the same. Assuming that the trackID of the spot check target is "pedestrian 0", and the trackID of the comparison object is "pedestrian 1", it is indicated that the spot check target with the trackID "pedestrian 0" and the comparison object with the trackID "pedestrian 0" in the candidate pedestrian library are not the same person, and they are the same person as the comparison object with the trackID "pedestrian 1" in the candidate pedestrian library, that is, the target bodies corresponding to the spot check target and the comparison object have trackID exchange in the current frame. At this time, the trackIDs of both are exchanged, that is, the trackID of the target body corresponding to the comparison object in the current frame is assigned to the spot check target, and the trackID of the spot check target is assigned to the target body corresponding to the comparison object, that is, the trackID of the spot check target is modified to "pedestrian 1", and the trackID of the target body corresponding to the comparison object is modified to "pedestrian 0".

[0130] In the third case, if the matching degree of the spot check target and the comparison object in the comparison group is less than the preset matching degree threshold, the spot check target is marked as not being a teacher target.

[0131] Since the comparison group selects the combination of the highest matching degree of the spot check target and the comparison object, if the matching degree of the spot check target and the comparison object in the comparison group is still less than the preset matching degree threshold, it means that the physiological characteristics of the spot check target are not similar to all comparison objects in the current candidate pedestrian library, which means that it is not saved in the current candidate pedestrian library. In this case, the identity re-authentication fails, and the spot check target is temporarily marked as not a teacher target.

[0132] Further, the spot check target is classified by the cascade model and ReID feature extraction is performed, and the extracted results are stored as the features of the comparison object in the candidate pedestrian library.

[0133] In some embodiments of the present application, whether the spot check target matches the comparison object or not, the matching degree of the spot check target and the comparison object with the same trackID in the candidate pedestrian library is calculated (if the aforementioned has been calculated, the result can be directly used here), if the matching degree is within the preset high similarity threshold range, such as (0.70, 0.90), the ReID feature extracted this time is saved in the candidate pedestrian library, that is, the ReID feature of the spot check target extracted this time is saved in the ReID feature of the corresponding posture of the comparison object in the candidate pedestrian library. Thus, the accuracy of the ReID feature of the comparison object can be improved, which provides guarantee for subsequent re-authentication, and improves the accuracy of re-authentication.

[0134] Figure 12 A schematic diagram of identity re-authentication using a cascade model according to an embodiment of the present application is shown, from Figure 12 It can be seen that the present embodiment includes: First, a spot check target is determined from the current frame, then it is judged whether the spot check times of the spot check target within a preset time period are greater than a preset number threshold, if yes, the spot check target is abandoned.

[0135] If not, the spot check target is classified by the posture classifier, and then the matching degree of the spot check target and each comparison object in the candidate pedestrian library is calculated, and the group with the maximum matching degree is selected as the comparison group.

[0136] Then it is judged whether the matching degree of the comparison group is greater than a preset experience threshold, if not, it is confirmed that the re-authentication fails, and the spot check target is marked as not a teacher; if yes, it is judged whether the trackIDs of the spot check target and the comparison object in the comparison group are consistent, if yes, it is confirmed that the re-authentication succeeds, and the current trackID of the spot check target is maintained; if not, it is confirmed that identity exchange occurs, and the trackIDs of the target bodies corresponding to the selected spot check target and the selected comparison object are exchanged.

[0137] After the above operations are completed, the spot check time of the spot check target is recorded and the accumulated number of spot checks is updated.

[0138] The matching degree of the matching object consistent with the trackID in the candidate pedestrian library is calculated, and if the matching degree is in the range of the high similarity threshold, the ReID feature extracted this time is saved in the candidate pedestrian library, and finally enters the next frame.

[0139] Figure 13 A schematic diagram of identity re-authentication using a cascade model according to another embodiment of the present application is shown, from Figure 13 It can be seen that the present embodiment includes: Two sampling targets, pedestrian 0 and pedestrian 1, are sampled, and the two are matched with the matching objects in the candidate pedestrian library. The matching degree between the sampling target and the matching object is represented by d

[0140] In Figure 13 , taking pedestrian 0 as an example, it is assumed that the n edges connected with the query graph In the similarity maximum edge is d 00 , and the value is greater than 0.5, indicating that among the n edges connected with the query graph, the trackID=0 pedestrian feature in the pedestrian feature library is similar to the trackID=0 pedestrian feature, and is considered to be the same person.

[0141] It is assumed that the n edges connected with the query graph In the similarity maximum edge is not d 00 , it is assumed that d 01 , indicating that the sampling target with trackID=0 is not the same person as the candidate pedestrian library trackID=0 person, at this time if d 01 is greater than 0.5, indicating that the trackID=0 person has ID-Switch, at this time the trackID of the two needs to be exchanged.

[0142] The identity re-authentication of the present application considers the difference between different postures and the updating mechanism of the candidate pedestrian library, which not only improves the robustness of the ReID feature, but also improves the authentication recall rate.

[0143] The algorithm of the present application is simple, suitable for embedded real-time calculation, does not sacrifice matching accuracy, and only uses a small amount of acceptable delay as a price; and the algorithm configuration is flexible, and for platforms with high computing power or video processing, only a small amount of configuration modification is needed to restore the traditional RelD method.

[0144] Figure 14 A structural schematic diagram of a teacher identity authentication and tracking device according to an embodiment of the present application is shown, from Figure 14 It can be seen that the teacher identity authentication and tracking device 200 includes: ​The acquisition unit 210 is configured to acquire a video image stream of a classroom monitoring device. The target detection unit 220 is configured to perform human body and key point detection on each frame of image based on a human pose estimation network, and perform blackboard, head and face detection on each frame of image by using a three-class target detection network. The tracking unit 230 is configured to track each target human body detected based on an improved multi-target tracking algorithm, and attribute the detected head and face to each target human body. The preliminary screening unit 240 is configured to determine position information and timing information of each target human body relative to a blackboard region, and perform preliminary screening of teacher identity for each target human body according to the position information and the timing information, to obtain a teacher target. The re-authentication unit 250 is configured to perform identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to prevent identity exchange between a teacher and a student.

[0145] In some embodiments of the present application, in the above device, the key points include: eyes, ears, nose, hands, elbows, shoulders, hips, knees and feet.

[0146] In some embodiments of the present application, in the above device, the target detection unit 220 is configured to, for a frame of image, perform detection by using a human pose estimation model constructed based on YOLOv5-Pose, to obtain detection boxes of a plurality of target human bodies and a plurality of key points of each target human body; and normalize coordinate values of the detection boxes and the key points of each target human body.

[0147] In some embodiments of the present application, in the above device, the target detection unit 220 is configured to, for a frame of image, perform detection by using a three-class target detection model constructed based on YOLOv5, to obtain detection boxes of a plurality of blackboard regions, heads and faces; normalize coordinate values of the detection boxes of each blackboard region, the head and the face; and if the blackboard regions are multiple, merge the multiple blackboard regions.

[0148] In some embodiments of the present application, in the above device, the target detection unit 220 is configured to calculate pixel areas of each blackboard region, and sort the pixel areas in descending order from large to small; set an area proportion threshold, and remove a blackboard region with a pixel area less than the area proportion threshold of the maximum blackboard region area; and calculate bounding boxes of the remaining blackboard regions, to form an overall blackboard region.

[0149] In some embodiments of the present application, in the above device, the target detection unit 220 is configured to determine an area ratio of each of the remaining blackboard regions to the largest blackboard region based on the largest blackboard region; if an area ratio of one of the remaining blackboard regions to the largest blackboard region is less than the area ratio threshold, the one of the remaining blackboard regions is discarded; otherwise, the one of the remaining blackboard regions is retained.

[0150] In some embodiments of the present application, in the above device, the target detection unit 220 is configured to, for each of the retained blackboard regions, count a top coordinate value of a top-left corner and a bottom-right corner of a detection frame; determine a maximum value and a minimum value of the top-left corner coordinate values and a maximum value and a minimum value of the bottom-right corner coordinate values of the plurality of blackboard regions, respectively; construct a bounding box based on the minimum value of the top-left corner coordinate values and the maximum value of the bottom-right corner coordinate values to form the overall blackboard region.

[0151] In some embodiments of the present application, in the above device, the improved multi-target tracking algorithm is implemented by using an AlphaBeta filter to improve a SORT model; the tracking unit 230 is configured to assign a unique trackID to each of the detected target human bodies and record a tracking time as a tracking object; use the AlphaBeta filter to associate each of the target human bodies detected in the current frame image with a plurality of existing tracking objects; if one of the target human bodies is associated with a tracking object, assign the trackID of the tracking object to the target human body; if one of the target human bodies is not associated with the tracking object, assign a new unique trackID to the target human body; if one of the tracking objects is not associated with a target human body within a preset time length, discard the trackID of the tracking object; and update the key points of each of the target human bodies in the current frame image.

[0152] In some embodiments of the present application, in the above device, the tracking unit 230 is configured to use the AlphaBeta filter to calculate an association degree of each of the tracking objects and each of the target human bodies to form an association degree matrix; use the Hungarian algorithm to find an optimal match based on the association degree matrix, and set a minimum association degree threshold in the matching process; associate each of the target human bodies with a plurality of existing tracking objects according to the optimal matching result; if there is no tracking object currently, assign a unique trackID to the latest detected target human body and record a tracking time as a tracking object; if one of the tracking objects is not associated with a target human body, use the AlphaBeta filter to predict and model the target human body, associate the modeled target human body with the tracking object, and update the modeled target human body to a set of detected target human bodies.

[0153] In some embodiments of the present application, in the above device, the correlation degree is any one of IoU, DIoU, GIoU and CIoU.

[0154] In some embodiments of the present application, in the above device, the tracking unit 230 is configured to calculate an improved intersection over union of each detected human head or face and each detection box of the target human body, respectively; if the improved intersection over union of a human head or face and a target human body is greater than a preset first empirical threshold, the human head or face belongs to the target human body; wherein the improved intersection over union can be represented as formula (16): Formula (16); In the formula, A and B represent different detection box pairs. The minimum value represents the union of a plurality of detection box pairs.

[0155] In some embodiments of the present application, in the above device, the tracking unit 230 is configured to, if three specified key points of a target human body are covered by a detection box of a face, the face belongs to the target human body, and the specified key points include eyes and nose.

[0156] In some embodiments of the present application, in the above device, the preliminary screening unit 240 is configured to determine the overlap degree of each target human body and the blackboard area, and the overlap degree can be represented by an improved intersection over union; if the overlap degree is greater than a second empirical threshold, the target human body is marked as being on the podium; otherwise, the target human body is marked as not being on the podium.

[0157] In some embodiments of the present application, in the above device, the preliminary screening unit 240 is configured to traverse each target human body, for a target human body, determine whether the target human body is on the podium, or according to the recorded time sequence information, determine whether the time length of the target human body on the podium is greater than a preset time length threshold; if yes, the target human body is taken as a candidate object; if the number of candidate objects is greater than zero, the time length of the plurality of candidate objects on the podium is sorted in descending order, and the first one is selected as the teacher; if there are a plurality of candidate objects ranked first, the tallest one is selected as the teacher target.

[0158] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to construct and train a standing posture classifier and a pedestrian ReID feature extraction model to form the cascade model; construct a candidate pedestrian library based on the cascade model, the candidate pedestrian library including ReID features of comparison objects in different standing posture dimensions, wherein the standing posture dimensions include facing, back to, left side and right side; determine at least one spot check target in a current frame image according to the position information and the time sequence information to form a query graph; and perform feature and identity matching on the spot check target and each comparison object in the candidate pedestrian library based on the query graph and the cascade model to perform identity re-authentication.

[0159] In some embodiments of the present application, in the above device, the standing posture classifier is pre-trained according to the following method: constructing an initial model of the standing posture classifier, which can be established according to any one of an SVM mechanism, a decision tree mechanism and a shallow neural network; obtaining sample data in a plurality of classroom scenes and performing sample processing to construct a standing posture data set, wherein the sample processing includes: performing key point detection on the sample data by using a human body posture estimation network, and performing posture category labeling on the sample data, wherein the key points include: eyes, ears, nose, hands, elbows, shoulders and hips, and the posture categories include: facing, back to, left side and right side; and training the initial model of the standing posture classifier based on the standing posture data set to obtain the standing posture classifier.

[0160] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to traverse each of the target human bodies, and if the target human body is located on a podium or the time length of the target human body located on the podium is greater than a preset time length threshold according to the recorded time sequence information, the target human body is taken as a spot check target; record the spot check times of the spot check target within a preset time length, and if the spot check times are greater than or equal to a preset number threshold, the spot check target is abandoned.

[0161] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to perform standing posture classification on the spot check target in the query graph based on the standing posture classifier of the cascade model; calculate the matching degrees of the spot check target and each comparison object in the candidate pedestrian library according to the standing posture classification result; and perform identity re-authentication on the spot check target according to the matching degrees.

[0162] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to calculate the similarity between the spot-check target and the comparison object in different station posture dimensions based on the Euclidean distance or the cosine distance; if the similarity in the station posture dimension of the station posture classification result is greater than a preset similarity threshold, the similarity is taken as the matching degree between the spot-check target and the comparison object; if the similarity in the station posture dimension of the station posture classification result is not greater than the preset similarity threshold, the similarities in each station posture dimension are fused by weighting to obtain the matching degree between the spot-check target and the comparison object.

[0163] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to extract the spot-check ReID feature of the spot-check target by using the pedestrian ReID feature extraction model of the cascade model; for each station posture dimension, the Euclidean similarity or the cosine similarity between the spot-check ReID feature and the ReID feature of the comparison object at different times is calculated respectively; and the maximum value of the Euclidean similarity or the cosine similarity in each station posture dimension is selected as the similarity in the station posture dimension.

[0164] In some embodiments of the present application, in the above device, the re-authentication unit 250 is configured to arrange the matching degrees between the spot-check target and each comparison object in descending order, and select the spot-check target and the comparison object ranked first as a comparison group; if the matching degrees between the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the track IDs of the two are the same, it is confirmed that the identity re-authentication is successful, and the current track ID of the spot-check target is maintained; if the matching degrees between the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the track IDs of the two are different, it is confirmed that identity exchange occurs, and the track IDs of the target human bodies corresponding to the spot-check target and the comparison object are exchanged; if the matching degrees between the spot-check target and the comparison object in the comparison group are less than a preset matching degree threshold, and the track IDs of the two are the same, the spot-check target is marked as not being a teacher target.

[0165] In some embodiments of the present application, in the above device, the re-authentication unit 250 is further configured to calculate or read the matching degree between the spot-check target and the comparison object with the same track ID in the candidate pedestrian library, and if the obtained matching degree is within a preset high similarity threshold range, the spot-check ReID feature of the spot-check target extracted this time is saved in the candidate pedestrian library.

[0166] It should be noted that the above teacher identity authentication and tracking device 200 can implement the above-mentioned teacher identity authentication and tracking method, and details are not repeated here.

[0167] Figure 15 This application shows a schematic diagram of the structure of a classroom monitoring device according to an embodiment of the present application. Figure 15 As shown, the classroom monitoring equipment includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When executed by the processor, the computer program implements the functions or steps of the teacher authentication and tracking method.

[0168] In one embodiment, the classroom monitoring device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the aforementioned teacher identity authentication and tracking method.

[0169] The above is as stated in this application. Figure 14 The method for teacher authentication and tracking disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0170] In one embodiment, a computer readable storage medium is also provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the aforementioned method for teacher identity authentication and tracking based on machine vision in a classroom observation.

[0171] It should be noted that the functions or steps that can be achieved by the classroom monitoring device or the computer readable storage medium described above can be referred to the related description in the aforementioned method embodiments. To avoid repetition, they will not be described one by one here.

[0172] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified. In actual application, the above-mentioned functions can be completed by different functional units or modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-mentioned functions.

[0174] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A teacher identity authentication and tracking method, characterized in that, The method comprises the following steps: acquiring a video image stream of a classroom monitoring device; detecting human bodies and key points in each frame image based on a human pose estimation network, and detecting blackboards, human heads and human faces in each frame image using a three-class target detection network; tracking each detected target human body based on an improved multi-target tracking algorithm, and attributing each detected human head and human face to each target human body; determining position information and timing information of each target human body relative to a blackboard region, and preliminarily screening teacher identities of each target human body based on the position information and the timing information to obtain a teacher target; performing identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction to prevent identity exchange between a teacher and a student.

2. The method of claim 1, wherein, The key points include: both eyes, both ears, a nose, both hands, both elbows, both shoulders, both hips, both knees and both feet.

3. The method of claim 1, wherein, The human pose estimation network detects human bodies and key points in each frame image, which comprises the following steps: for a frame image, a human pose estimation model based on YOLOv5-Pose is used for detection to obtain detection boxes of a plurality of target human bodies and a plurality of key points of each target human body; coordinate values of the detection boxes of each target human body and the key points are normalized.

4. The method of claim 1, wherein, The three-class target detection network detects blackboards, human heads and human faces in each frame image, which comprises the following steps: for a frame image, a three-class target detection model based on YOLOv5 is used for detection to obtain detection boxes of a plurality of blackboard regions, human heads and human faces; coordinate values of the detection boxes of each blackboard region, the human head and the human face are normalized; if the blackboard region is multiple, the plurality of blackboard regions are merged.

5. The method of claim 4, wherein, The plurality of blackboard regions are merged, which comprises the following steps: pixel areas of each blackboard region are calculated, and the pixel areas are sorted in descending order from large to small; an area proportion threshold is set, and a blackboard region with a pixel area less than the area proportion threshold of the maximum blackboard region area is removed; bounding boxes of the remaining blackboard regions are calculated to form an overall blackboard region.

6. The method of claim 5, wherein, The blackboard region with a pixel area less than the area proportion threshold of the maximum blackboard region area is removed, which comprises the following steps: the area ratio of each remaining blackboard region to the maximum blackboard region is determined; if the area ratio of a remaining blackboard region to the maximum blackboard region is less than the area proportion threshold, the remaining blackboard region is removed; otherwise, the remaining blackboard region is retained.

7. The method of claim 5, wherein, The bounding boxes of the remaining blackboard regions are calculated to form an overall blackboard region, which comprises the following steps: for each remaining blackboard region, the vertex coordinate values of the top left corner and the bottom right corner of the detection box are counted; the maximum and minimum values of the top left corner vertex coordinate values and the maximum and minimum values of the bottom right corner vertex coordinate values of the plurality of blackboard regions are calculated respectively; a bounding box is constructed based on the minimum value of the top left corner vertex coordinate values and the maximum value of the bottom right corner vertex coordinate values to form the overall blackboard region.

8. The method of claim 1, wherein, The improved multi-target tracking algorithm is realized by improving the SORT model using an AlphaBeta filter. The improved multi-target tracking algorithm is used to track each detected target human body, comprising: allocating a unique trackID to each detected target human body and recording the tracking time as a tracking object; using the AlphaBeta filter to associate each detected target human body in the current frame image with the existing tracking objects; if a target human body is associated with a tracking object, assigning the trackID of the tracking object to the target human body; if a target human body is not associated with a tracking object, allocating a new unique trackID to it and recording the tracking time as a tracking object; if a tracking object is not associated with a target human body within a preset time length, discarding the trackID of the tracking object; updating the key points of each target human body in the current frame image.

9. The method of claim 8, wherein, The association of each detected target human body in the current frame image with the existing tracking objects using the AlphaBeta filter comprises: using the AlphaBeta filter to calculate the association degree of each tracking object and each target human body respectively, forming an association degree matrix; using the Hungarian algorithm to find the optimal match based on the association degree matrix, and setting a minimum threshold for the association degree in the matching process; associating each target human body with the existing tracking objects according to the optimal matching result; if there is no tracking object at present, allocating a unique trackID to the latest detected target human body and recording the tracking time as a tracking object; if a tracking object is not associated with a target human body, using the AlphaBeta filter to predict and model, associating the modeled target human body with the tracking object, and updating the modeled target human body to the set of detected target human bodies.

10. The method of claim 9, wherein, The association degree is any one of IoU, DIoU, GIoU and CIoU.

11. The method of claim 1, wherein, The association of the detected human head and face with each target human body comprises: calculating the improved intersection over union of each detected human head or face with the detection box of each target human body; if the improved intersection over union of a human head or face with the detection box of a target human body is greater than a preset first empirical threshold, the human head or face is associated with the target human body; wherein the improved intersection over union can be expressed as formula (16): Equation (16); In the formula, A and B represent different detection frame pairs formed by detection frames; The minimum value represents the union of the plurality of detection frame pairs.

12. The method of claim 1, wherein, The association of the detected human head and face with each target human body comprises: if the three specified key points of a target human body are covered by the detection box of a face, the face is associated with the target human body, and the specified key points include eyes and nose.

13. The method of claim 1, wherein, Determining the position information of each target human body relative to the blackboard region comprises: determining the overlap degree of each target human body and the blackboard region, which can be represented by the improved intersection over union; if the overlap degree is greater than a preset second empirical threshold, the target human body is marked as being on the podium; otherwise, the target human body is marked as not being on the podium.

14. The method of claim 1, wherein, The preliminary screening of the teacher identity of each target human body according to the position information and the timing information includes: traversing each target human body, and determining whether the target human body is located on a rostrum or whether the length of time that the target human body is located on the rostrum is greater than a preset length threshold according to the recorded timing information; if yes, the target human body is taken as a candidate object, and if the number of the candidate objects is greater than zero, the lengths of time that the plurality of candidate objects are located on the rostrum are sorted in descending order, and the first one is selected as a teacher target; if there are a plurality of candidate objects that are ranked first, the one with the highest height is selected as the teacher target.

15. The method of claim 1, wherein, The identity re-authentication based on the pre-trained posture classification and pedestrian ReID feature extraction cascade model includes: constructing and training a posture classifier and a pedestrian ReID feature extraction model to form the cascade model; constructing a candidate pedestrian library based on the cascade model, and the candidate pedestrian library includes ReID features of comparison objects in different posture dimensions, wherein the posture dimensions include facing, back to, left side and right side; determining at least one spot check target in a current frame image according to the position information and the timing information to form a query graph; based on the query graph, the cascade model is used to perform feature and identity matching between the spot check target and each comparison object in the candidate pedestrian library to perform identity re-authentication.

16. The method of claim 15, wherein, The posture classifier is pre-trained according to the following method: constructing an initial model of the posture classifier, which can be established according to any one of an SVM mechanism, a decision tree mechanism and a shallow neural network; obtaining sample data in a plurality of classroom scenes and performing sample processing to construct a posture data set, wherein the sample processing includes performing key point detection on the sample data by using a human posture estimation network, and performing posture category labeling on the sample data, wherein the key points include eyes, ears, nose, hands, elbows, shoulders and hips, and the posture categories include facing, back to, left side and right side; training the initial model of the posture classifier based on the posture data set to obtain the posture classifier.

17. The method of claim 15, wherein, The determination of at least one spot check target in a current frame image according to the position information and the timing information to form a query graph includes: traversing each target human body, and if the target human body is located on a rostrum or the length of time that the target human body is located on the rostrum is greater than a preset length threshold according to the recorded timing information, the target human body is taken as a spot check target; recording the number of times of spot checks of the spot check target within a preset length of time, and if the number of times of spot checks is greater than or equal to a preset number threshold, the spot check target is abandoned.

18. The method of claim 15, wherein, Based on the query graph, the cascade model is used to perform feature and identity matching between the spot check target and each comparison object in the candidate pedestrian library to perform identity re-authentication, which includes: performing posture classification on the spot check target in the query graph based on the posture classifier of the cascade model; According to the station posture classification result, a matching degree of the spot-check target and each comparison object in the candidate pedestrian library is calculated; According to the matching degree, identity re-authentication is performed on the spot-check target.

19. The method of claim 18, wherein, According to the station posture classification result, a matching degree of the spot-check target and each comparison object in the candidate pedestrian library is calculated, including: For one comparison object, a similarity of the spot-check target and the comparison object in different station posture dimensions is calculated based on a Euclidean distance or a cosine distance; If the similarity in the station posture dimension of the station posture classification result is greater than a preset similarity threshold, the similarity is taken as the matching degree of the spot-check target and the comparison object; If the similarity in the station posture dimension of the station posture classification result is not greater than the preset similarity threshold, the similarities in each station posture dimension are fused by weighting to obtain the matching degree of the spot-check target and the comparison object.

20. The method of claim 19, wherein, The similarity of the spot-check target and the comparison object in different station posture dimensions is calculated based on a Euclidean distance or a cosine distance, including: A spot-check ReID feature of the spot-check target is extracted by using a pedestrian ReID feature extraction model of the cascade model; For each station posture dimension, a Euclidean similarity or a cosine similarity of the spot-check ReID feature and ReID features of the comparison object at different time points is calculated respectively; The maximum value of the Euclidean similarity or the cosine similarity in each station posture dimension is selected as the similarity in the station posture dimension.

21. The method of claim 18, wherein, According to the matching degree, identity re-authentication is performed on the spot-check target, including: The matching degrees of the spot-check target and each comparison object are arranged in descending order, and the spot-check target and the comparison object with the first order are selected as a comparison group; If the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the trackIDs of the two are the same, it is confirmed that the identity re-authentication is successful, and the trackID of the spot-check target is maintained; If the matching degrees of the spot-check target and the comparison object in the comparison group are greater than or equal to a preset matching degree threshold, and the trackIDs of the two are different, it is confirmed that identity exchange occurs, and the trackIDs of the target bodies corresponding to the spot-check target and the comparison object are exchanged; If the matching degrees of the spot-check target and the comparison object in the comparison group are less than a preset matching degree threshold, and the trackIDs of the two are the same, it is marked that the spot-check target is not a teacher target.

22. The method of claim 21, wherein, According to the matching degree, identity re-authentication is performed on the spot-check target, further including: The matching degrees of the spot-check target and the comparison object with the same trackID in the candidate pedestrian library are calculated or read, and if the obtained matching degrees are within a preset high similarity threshold range, the spot-check ReID feature of the spot-check target extracted this time is saved in the candidate pedestrian library.

23. A teacher identity authentication and tracking apparatus, characterized by, The device includes: An acquisition unit is configured to acquire a video image stream of a classroom monitoring device; The target detection unit is configured to perform human body and key point detection on each frame of image based on a human pose estimation network, and perform blackboard, head and face detection on each frame of image by using a three-class target detection network. The tracking unit is configured to track each detected target human body based on an improved multi-target tracking algorithm, and attribute the detected head and face to each target human body. The preliminary screening unit is configured to determine position information and time sequence information of each target human body relative to a blackboard region, and perform preliminary screening of teacher identity on each target human body according to the position information and the time sequence information, to obtain a teacher target. The re-authentication unit is configured to perform identity re-authentication based on a pre-trained cascade model of pose classification and pedestrian ReID feature extraction, to prevent identity exchange between a teacher and a student.

24. A classroom monitoring device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the teacher identity authentication and tracking method according to any one of claims 1 to 22.

25. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the teacher identity authentication and tracking method according to any one of claims 1 to 22.

Images