A personnel behavior monitoring method for laboratory safety management

Abstract

The application provides a personnel behavior monitoring method for laboratory safety management. The application is based on laboratory monitoring video, and simultaneously calculates face recognition results and human body position information in video frames; based on the geometric relationship between the face recognition result rectangular frame and the human body position information rectangular frame, the face recognition result is transmitted to the human body; the center position of the feet of the human body in each frame of the video is recorded, and the motion trajectory of the human body in the video is extracted; a plurality of key points are calibrated in a laboratory plan, corresponding positions in the video sequence are searched and the same marks are made, a two-dimensional mapping relationship between the monitoring picture and the laboratory plan is established, the motion trajectory of the human body in the video is mapped into the laboratory plan, and in the case that a plurality of monitoring cameras exist in the laboratory, the trajectory is fused, and the action route of the person is analyzed based on the trajectory in the plan. The application can detect personnel information entering the laboratory, and record the activity of the personnel in the laboratory.

Description

Technical Field

[0001] This invention mainly relates to the fields of deep learning, computer vision, and camera calibration, specifically a method for monitoring personnel behavior in laboratory safety management. Background Technology

[0002] Laboratory safety management is fundamental to ensuring the smooth operation of experiments. Therefore, using artificial intelligence technologies such as sensors and cameras to assist in laboratory safety management is one of the effective means to ensure laboratory safety.

[0003] Currently, several mature deep learning models exist that can track people appearing in video scenes based on video surveillance and obtain their trajectory maps within the video; facial recognition technology has also been widely used in many access control systems. However, due to the unpredictability of people's clothing and accessories every day, current human body recognition and tracking methods cannot extract effective features sufficient to identify individuals, i.e., they cannot obtain human ID information. Furthermore, the extraction of human trajectory mostly involves extracting the trajectory of a person within the video frame, but due to information loss when converting three-dimensional world coordinates to two-dimensional pixel coordinates and the barrel distortion phenomenon of wide-angle cameras, the trajectory within the video frame cannot accurately represent a person's movement route. Moreover, if the laboratory area is too large or contains too many items, single-camera monitoring can easily lead to blind spots, requiring multiple cameras to work together to obtain accurate movement routes.

[0004] Face recognition is an effective method for determining human identity, but it fails when a person is facing away from the camera or looking down. Human tracking and re-identification are unaffected by orientation and posture, but when a person disappears from the field of view and then reappears, it's difficult to determine whether the person re-entering the camera's field of view is the same person who was previously in the field of view. The main challenges of combining face recognition with human tracking and re-identification to obtain the tracking result and identity of the human body in each frame are: how to accurately transmit the identified face identity to the human body to obtain its identity; how to retain the human identity information in these video frames when the face orientation does not meet the basic requirements for recognition; and how to correct the re-identification failure using face recognition results when the human body reappears after leaving the field of view and is mistakenly identified as a different person by the algorithm.

[0005] Human body tracking is an effective way to determine a person's activity status within the camera's field of view. Based on human body tracking results, the trajectory of a person on a laboratory floor plan can more accurately reflect the person's movement route within the camera's field of view compared to the trajectory of a person within a video frame.

[0006] In laboratory settings, designing a reasonable method combining facial recognition and human tracking, along with a camera calibration scheme, based on surveillance video, to obtain a person's location and ID information, accurately identifies personnel entering the laboratory, and obtains their precise movement routes, is an effective way to determine whether laboratory personnel have entered dangerous areas and whether their operating procedures are correct. This has positive significance for reducing laboratory safety hazards and ensuring the safety of laboratory personnel. Summary of the Invention

[0007] To address the problem that existing technologies cannot accurately identify the ID information of personnel entering the laboratory in real time, thus failing to obtain the movement routes of laboratory personnel, this invention provides a personnel behavior monitoring method for laboratory safety management. It constructs an HFRN (Human and Face Recognition Network) that can simultaneously perform face recognition and human body tracking, acquiring real-time location and ID information of laboratory personnel, and extracting their trajectories within the laboratory floor plan to achieve real-time monitoring of their movement routes. The specific steps of this invention are as follows: 1) Based on laboratory surveillance video and facial images of personnel authorized to enter the laboratory, acquire the human body tracking results and face recognition results from the video frames, and transmit the face ID to the human body.

[0008] Specifically, this invention proposes an HFRN (Human and Face Recognition Network) for human face detection and human ID recognition. This network takes the original video frame as input, constructs a backbone network based on dilated convolution and feature pyramids to extract multi-scale features from the video frame, and constructs two detection branches to detect the location and number of the human body and the location and ID of the face, respectively. Then, based on the spatial distribution relationship between the detection results, the face ID is passed to the human body.

[0009] For the branch involving human detection and tracking, the spatiotemporal relationship between the current frame and all previous frames is first established based on Kalman filtering for human tracking and re-identification. Kalman filtering is used to assess the motion trend of the human body in all previous frames. Then, the detection results of the current frame are cascaded and matched with human bodies within a reasonable range of motion trends. If a match is found, the target was identified as having appeared in the previous frame; otherwise, it is identified as a new target. During this stage, people are numbered according to the order in which they enter the video field of view. When a person reappears after disappearing from the field of view, their original number is retained.

[0010] For the branch detecting face location and ID, the network employs a triplet loss function during training to ensure small differences in encoding between two images of the same person and large differences between encodings of two images of different people. Additional supervision information is added, enabling this branch to not only output bounding boxes representing face location information but also extract five key points for each face: the center of the left eye, the center of the right eye, the center of the nose, the left corner of the mouth, and the right corner of the mouth. This results in more robust detection accuracy for dense or blurry faces. After outputting the detection results for the current frame, the network maps the face images of lab admission personnel and the detected face images in the video frame to Euclidean space and compares the spatial distance between these images to obtain the similarity between faces in the video frame and those on the lab admission list. The ID information of those on the list is then passed to similar faces in the video frame.

[0011] Furthermore, the specific steps for passing the face ID to the human body are as follows: The bounding box of the face detection result for each frame is represented by (x... 1i ,y 1i ) and (x 2i ,y 2i ) represents, i∈n, where n is the number of faces detected in the current frame, and i is the i-th face in the current frame, (x 1i ,y 1i (x) represents the coordinates of the top-left corner of the face bounding box. 2i ,y 2i (X) represents the coordinates of the bottom right corner of the face bounding box. The human tracking results for each frame are represented by (X... 1j ,Y 1j ) and (X 2j ,Y 2j ) represents j∈m, where m is the number of human bodies detected in the current frame, and j is the j-th human body in the current frame, (X 1j ,Y 1j (X) represents the coordinates of the top-left corner of the human figure frame. 2j ,Y 2j (x) represents the coordinates of the bottom right corner of the human body bounding box. 1i ,y 1i ), (x 2i ,y 2i ), (X 1j ,Y 1j ), (X 2j ,Y 2j When the coordinates satisfy the positional relationship shown in Formula 1, the ID information of face i is passed to body j.

[0012]

[0013] The positional relationships implied in Formula 1 that satisfy the id passing condition are as follows:Figure 3 As shown.

[0014] 2) Capture and store facial images of people who have entered the laboratory but are not on the laboratory's access list.

[0015] Specifically, when the similarity between the face in the video and the face of a person admitted to the laboratory is below a certain threshold, there are two possible explanations for this phenomenon. First, the face is too small or is a profile view with a large tilt angle, resulting in limited facial features that are insufficient to identify the face's ID information. Second, someone not on the laboratory's admission list entered the laboratory.

[0016] To ensure that all captured facial images with low similarity are of non-accessible personnel, this invention sets two restrictions to filter faces with similarity below a threshold. The threshold is set as follows: After extracting features from the facial images of the video and the faces of laboratory access personnel through a backbone network, each image is transformed into a 10×10×3 matrix, which represents the facial features. The similarity between two faces is calculated based on cosine similarity. In this invention, the similarity threshold is set to 90%, that is, comparing whether the cosine similarity of the feature matrices of two faces is less than 90%.

[0017] Limitation 1: Face images with a resolution less than 30*30 pixels are not cropped; Limitation 2: Face images that are not frontal are not cropped. This invention determines whether a face is frontal based on five key points: the center of the left eye, the center of the right eye, the center of the nose, the left corner of the mouth, and the right corner of the mouth. First, the center of the left eye, the center of the right eye, the right corner of the mouth, and the left corner of the mouth are connected sequentially to form a quadrilateral region; then, the aspect ratio of this region is fixed, and the area is reduced to 1 / 4 of the original area to obtain a sub-region located at the same center position as the original region (e.g., ...). Figure 2 (As shown); Finally, it is determined whether the coordinates corresponding to the center of the nose are located within this sub-region. If they are, it is a frontal view; otherwise, it is a side view or a head-down position. This invention expands the frontal view of a face with a resolution greater than 30*30 by 1.5 times, crops the image of this region, stores it in the warning folder, and names the image with the current timestamp.

[0018] 3) Mark several key points on the laboratory floor plan and mark the corresponding positions in the camera's field of view. Based on the coordinates of these points, establish a two-dimensional mapping relationship between the video scene and the laboratory floor plan, and obtain the transformation matrix.

[0019] Specifically, the conversion relationship between the laboratory floor plan and the laboratory video scene coordinates is shown in Formula 2.

[0020]

[0021] In the formula, a, b, c, d, c, f, g, h are the parameters of the transformation matrix, X', Y' are the set of coordinate points marked on the laboratory floor plan, x′, y′ are the corresponding coordinates of the laboratory video scene, and W is the weight of the transformation matrix. According to the matrix multiplication rule, W can be transformed into Formula 3.

[0022] W=gx′+hy′+1, (3)

[0023] Based on Equations 2 and 3, X' and Y' can be represented in matrix form as Equation 4.

[0024]

[0025] Mark more than four key points on the laboratory floor plan and find their corresponding coordinates in the laboratory video scene. Substitute these coordinates into Formula 4 to derive Formula 5. Calculate the transformation matrix parameters using the least squares method. Let p represent the number of key points on the laboratory floor plan, then:

[0026]

[0027] Based on the transformation matrix parameters, a unique coordinate point can be obtained on the laboratory floor plan for any point on the ground within the laboratory's video field of view.

[0028] 4) For a single camera in the laboratory, obtain the center coordinates of the human foot based on the human tracking results, and map the coordinates onto the laboratory floor plan according to the mapping function to obtain the trajectory of the person entering the laboratory on the floor plan.

[0029] Specifically, this invention establishes a two-dimensional mapping relationship between the laboratory floor plan and the ground in the video surveillance. Only by finding the coordinates of the center of the human foot on the ground from the surveillance video can the corresponding position of the human body be obtained in the laboratory floor plan. The corresponding coordinates of the upper left and lower right corners of the human body tracking frame are (x... 1i ,y 1i ) and (x 2i ,y 2i For situations where the entire human body enters the monitoring field of view, this invention selects (1 / 2 × x) 1i +1 / 2×x 2i ,y 2iThe coordinates of the foot center are used as the coordinates. For cases where the foot area is not within the monitoring video, this invention predicts the position of the foot center based on the position and width of the human tracking frame. Analysis of 1632 monitoring video images shows that when the human tracking frame accurately covers the entire human body, its aspect ratio is between 2.63 and 2.95. Even if the legs are not within the monitoring field of view, the width w of the tracking frame is accurate. Therefore, in this case, this invention predicts the foot center coordinates based on the average aspect ratio of the tracking frame and represents these coordinates as (1 / 2 × x). 1i +1 / 2×x 2i ,y 1i +2.79×(x 2i -x 1i )).

[0030] Within a single camera, the coordinates of a person's feet in each frame are determined based on the two-dimensional mapping relationship between the center coordinates of each person's feet and the laboratory floor plan. The coordinates of the same person are then connected in the time sequence of the video frames to form a trajectory within the floor plan.

[0031] 5) Based on multiple cameras placed at different locations in the same laboratory, multiple trajectories collected by multiple cameras are merged to form a more accurate trajectory of laboratory personnel on the floor plan.

[0032] Specifically, because a single camera's capture will leave visual blind spots in the laboratory, and a single wide-angle camera will experience more severe barrel distortion at the edges, reducing the accuracy of camera calibration, this invention designs a trajectory fusion scheme based on multiple cameras when multiple monitoring cameras are located in multiple directions in the laboratory.

[0033] Furthermore, assuming there are r surveillance cameras in the same laboratory, for the same human body at the same time, r camera coordinates can be detected, resulting in r predicted location points for that human body on the laboratory floor plan. Theoretically, these r points should completely overlap on the floor plan, but due to camera distortion, calibration errors, tracking errors, etc., they may be scattered within a local sub-region of the floor plan. This invention first uses a local anomaly detection algorithm to determine the local reachability density between sample points. The local reachability density is the reciprocal of the average reachable distance between the r predicted location points. The higher the sample point density, the lower the probability that the sample point is an anomaly. After multiple experiments, this invention sets the local reachability density threshold to 30. If the local reachability density from a sample point to other points is less than 30, then that point is an outlier; otherwise, it is a non-outlier. Outliers among the r points are removed using this method. Then, based on the remaining r′ coordinate points, the coordinates of the human body on the floor plan at the current time are calculated. The calculation formula is shown in Formula 6.

[0034]

[0035] By sorting the coordinates of the same person obtained based on Formula 6 in chronological order, a more robust trajectory within the laboratory floor plan can be obtained compared to a single camera.

[0036] 6) Based on the trajectory of laboratory personnel within the floor plan and the location of various areas and items in the laboratory, analyze the movement routes of people within the floor plan.

[0037] Specifically, this invention delineates an adjacent area for each functional area and fixed facility within the floor plan, serving as the basis for determining whether laboratory personnel have traversed that area. Based on the extracted trajectory within the floor plan and the extent of each sub-area, the time and order in which laboratory personnel pass through each area and facility can be determined, thereby identifying any irregularities in the workflow. For example, in a biological laboratory, sterile areas and experimental areas are divided. This invention can delineate the extent of sterile and experimental areas within the floor plan. Personnel entering the laboratory must first enter the sterile area to change into sterile clothing and caps before entering the experimental area. If, in this scenario, the extracted trajectory of a person does not pass through the area encompassed by the sterile area, it is determined that the laboratory personnel have engaged in irregularities in their operational procedures.

[0038] Beneficial effects

[0039] Based on image data transmitted from surveillance cameras within laboratories, this invention proposes a method for human detection and movement route analysis for laboratory safety management. Compared to traditional methods, this invention can obtain real-time tracking results with human IDs and more practically valuable human movement routes. This invention utilizes a combination of facial recognition and human tracking to obtain accurate human IDs. Even when a person is facing away from the camera or obscured for a period of time, the human ID information is still retained. A camera calibration method is used to obtain the correspondence between the video scene and the laboratory floor plan, and a multi-camera fusion strategy is designed to robustly and comprehensively extract the trajectory of people within the laboratory floor plan and generate their movement routes within the laboratory. This invention only requires one calibration based on the cameras and floor plan within each laboratory to analyze the movement routes of laboratory personnel over a long period, making it applicable to various laboratories such as biological, chemical, and information technology rooms. Attached Figure Description

[0040] Figure 1 Overall Flowchart of the Invention

[0041] Figure 2 An illustration for determining whether a face is a frontal view.

[0042] Figure 3 Explanation of transferring human ID to face

[0043] Figure 4 Camera calibration diagram

[0044] Figure 5 A screenshot showing the effect of multi-target human body tracking and obtaining human body IDs.

[0045] Figure 6 Image of human movement path extraction in a floor plan Detailed Implementation

[0046] The invention will now be further described with reference to the accompanying drawings.

[0047] 1. Model Framework

[0048] The overall framework for human detection and route analysis methods for laboratory safety management is attached. Figure 1 As shown. The underlying data required by the model includes frontal facial images of personnel allowed to enter the laboratory, one or more surveillance cameras, and a floor plan of the laboratory. This invention first obtains facial recognition and human tracking results based on the surveillance video; then, it passes the facial ID to the human body and captures facial images of those not on the access list; next, based on the laboratory floor plan and video footage, it marks several key points in both the floor plan and the video footage, establishing a mapping relationship between the floor plan and the video footage; finally, based on the tracking results and the mapping relationship, it obtains the trajectory of personnel entering the laboratory within the floor plan and analyzes their movement routes.

[0049] The following is a detailed description:

[0050] (1) In this invention, all frames in the video sequence are converted into RGB images and used as input to the proposed HFRN (Human and Face Recognition Network). The backbone of this network integrates dilated convolutions and a spatial pyramid structure. The feature pyramid contains three network layers with strides of 32, 16, and 8, respectively. The feature map output by the network layer with a stride of 32 corresponds to a 32×32 receptive field in the original image, which can be used to detect larger areas such as faces and human bodies. Similarly, the receptive fields corresponding to strides of 16 and 8 have a strong detection effect on small targets that are far from the camera. The feature maps output by the above three network layers are combined as the output features of the backbone network.

[0051] This invention uses the output of the backbone network as input to both the human detection branch and the face recognition branch. In the human detection branch, a simple, low-computation convolutional layer is used to extract human appearance features. These features are represented as low-dimensional vectors. After detection and tracking in each frame, the appearance features are extracted and saved. For each subsequent frame of tracking, a similarity calculation is performed between the current frame's human appearance features and the previously stored appearance features. This similarity score serves as the basis for determining the human's ID and achieving face recognition. In this branch, the time to obtain the human tracking result is approximately 0.08 seconds per frame.

[0052] In the face recognition branch, the features output from the backbone network are first input into the GoogleNet Style Inception model. This model encodes each face in the surveillance video, representing each face as a 128-dimensional feature vector. Then, the frontal images of the captured laboratory access personnel are input into the same model and encoded into 128-dimensional vectors in the same way. When the laboratory access personnel remain unchanged, only the frontal images of the captured personnel need to be encoded once, and the encoded information is stored in a specific file. Next, based on the L2 paradigm, the image encoding of the faces in the surveillance video is compared with the frontal image encoding of the access personnel. The face with the highest similarity (above 90%) is identified as the same person. If the similarity between the face and all the frontal faces of the access personnel is below 90%, the face captured from the video frame is considered an unfamiliar person or the capture quality is considered poor. This branch performs face recognition in approximately 0.06 seconds per frame.

[0053] Typically, video frames per second (fps) are around 30. Performing face recognition and body tracking on every frame would be impossible in real-time. In laboratory settings, vigorous movement by personnel is rare, so inter-frame deformation is not significant within adjacent time intervals. Therefore, this invention uses a frame-skipping approach to achieve real-time performance, performing body tracking every 3 frames and face recognition every 20 frames. The reason for significantly fewer face recognition attempts compared to body tracking attempts is that face ID information only needs to be transmitted to the body in critical video frames, as the acquired body ID information is automatically passed to the next frame during body tracking. Performing face recognition periodically allows for timely identification and transmission of the ID to the body when new personnel enter the laboratory, effectively correcting for failed body tracking attempts.

[0054] (2) When the detected face is a profile view, the extraction of its facial features may have significant errors, affecting the accuracy of obtaining the human ID and behavioral trajectory. Therefore, this invention designs constraints to determine whether the detected face is a frontal view based on five key points of the face. (See attached diagram) Figure 2As shown, the area indicated by the white dashed box is a quadrilateral closed area formed by sequentially connecting the center of the left eye, the center of the right eye, the right end of the corner of the mouth, and the left end of the corner of the mouth; the area indicated by the black dashed box is a sub-region with the same center as the white dashed box and an area of ​​1 / 4 of its size. When the coordinate point representing the center of the nose is located within the area contained in the black dashed box, the face is identified as a frontal face; otherwise, it is identified as a side face. The determination of a frontal face can reduce the error transmission phenomenon when transmitting the face ID to the human body and improve the accuracy of determining whether non-laboratory access personnel have entered the laboratory. (3) When the face ID is transmitted to the human body, if the face ID is not recognized correctly, it will lead to an incorrect human ID and cause error accumulation. This invention sets four restrictions. When the face position coordinates meet all four restrictions simultaneously, the face ID is transmitted to the human body. Appendix Figure 3 The larger solid black box represents the human body tracking result, the dashed box represents the possible range of the face obtained based on the human body tracking result and four constraints, and the smaller solid black box represents the face recognition result. When the entire area of ​​the face recognition result is within the dashed box, the face recognition ID is passed to the human body. If the face recognition result of the current frame is unknown, i.e., its ID information is displayed as "Unknown", the predicted ID of the human body in the previous frame is retained first. If the human body is a silhouette when it first enters the camera's field of view, there is no face recognition result for it at this time, and the human body ID at this time is the number of the order in which the human body entered the laboratory.

[0055] (4) The camera calibration method of the present invention is as follows: Figure 4 As shown, attached Figure 4 The left-middle image shows the calibration results within the laboratory floor plan. To achieve better calibration results, key points should be marked in prominent and fixed locations within the laboratory, and their two-dimensional coordinates on the floor plan should be recorded. (Attached image) Figure 4 The right-hand image shows the calibration results within the video scene. Points with the same symbol in both images correspond to the points in the planar view and the video scene, respectively. To solve for the eight parameters in the mapping matrix, at least four sets of key points should be calibrated. To reduce calibration errors, this invention uses more than four sets of key points and obtains the matrix parameters using the least squares method. (Appendix) Figure 4 The image shows the camera calibration results at 5 key points.

[0056] (5) The human body tracking results of the present invention are as follows: Figure 5 As shown in the image, two human bodies are detected. The letter above each tracking box represents the name of the person being tracked, and this name serves as the tracking ID. If the entire area of ​​one human body is within the field of view, the center of its foot is indicated by the gray pentagram. If only a portion of the other human body is within the field of view, the center of its foot is indicated by the coordinate relationship between the rectangles: "(1 / 2 × x 1i +1 / 2×x 2i ,y 1i +2.79×(x2i - x 1i ))” to obtain.

[0057] (6) Based on the coordinates of the human feet and the mapping matrix, obtain the movement trajectory of the person on the laboratory floor plan, as shown in the attached figure. Figure 6 The diagram is shown as a gray scatter plot. This plan contains 8 lab benches, numbered 1-8, and the elliptical areas circumscribed around each bench represent the adjacent areas. The arrows next to the trajectory plot indicate the direction of movement. Analysis of this trajectory plot shows that the lab worker with the ID "example" moved from the aisle between the two rows of lab benches, turned into the gap between lab benches 4 and 6, and finally stopped near lab bench 6.

Claims

1. A personnel behavior monitoring method for laboratory safety management, characterized by The method comprises the following steps: (1) Based on the laboratory monitoring video and the face image of the laboratory access personnel, the human body tracking result and the face recognition result in the video frame are obtained, and the face id is transmitted to the human body; (2) The face image of the person entering the laboratory and not in the laboratory access list is intercepted and stored; (3) A plurality of key points are calibrated in the laboratory plan, and corresponding marks are made at the same position in the camera field of view, and a two-dimensional mapping relationship between the video scene and the laboratory plan is established according to the coordinates of the above-mentioned points, and a transformation matrix is obtained; (4) For a single camera in the laboratory, the center coordinates of the human foot are obtained based on the human body tracking result, and the coordinates are mapped into the laboratory plan according to the mapping function, and the trajectory of the person entering the laboratory in the plan is obtained; (5) Based on a plurality of cameras arranged at different positions in the same laboratory, a plurality of trajectories collected by the plurality of cameras are fused to form a more accurate trajectory of the laboratory personnel in the plan; (6) Based on the trajectory of the laboratory personnel in the plan and the positions of each region and object in the laboratory, the action route of the person in the plan is analyzed; An HFRN network for human face detection and human id recognition is proposed; the network first takes the original video frame as input, constructs a backbone network based on a hollow convolution and a feature pyramid to extract multi-scale features of the video frame, and constructs two detection branches for detecting the position and number of the human body and the position and id of the face, and then transmits the face id to the human body based on the spatial distribution relationship between the detection results; For the branch for detecting and tracking the human body, firstly, the spatio-temporal relationship between the current frame and all previous frames is established based on Kalman filtering for human body tracking and re-identification, wherein the Kalman filtering is used for the motion trend of the human body in all previous frames; then the detection results of the current frame and the human body within a reasonable range of the motion trend are cascaded and matched, if matched, it is a target that has appeared in the previous frame, if not matched, it is a new target; in this stage, the personnel entering the video field in the order of their appearance is numbered, and when the person disappears and reappears in the field of view, the original number of the human body is still retained; For the branch for detecting the face position and id, the network uses a tripletloss function in the training stage to ensure that the encoding difference of two portraits from the same person is small, and the encoding difference of two portraits from different people is large, and additional supervision information is added, so that the branch can not only output a rectangular frame that can express the face position information, but also extract 5 key points for each face image, respectively corresponding to the center of the left eye, the center of the right eye, the center of the nose, the left end of the mouth corner and the right end of the mouth corner, and the detection accuracy is more robust for dense or blurred faces; after outputting the detection results of the faces in the current frame, the network maps the face images of the laboratory access personnel and the detected face images in the video frame to the Euclidean space, and compares the spatial distance between the face images of the laboratory access personnel and the face images in the video frame, to obtain the similarity between the faces in the video frame and the faces in the laboratory access list, and transmit the id information of the personnel in the list to the similar faces in the video frame; The specific steps of transmitting the id of the face to the human body are as follows: the face detection result rectangular frame of each frame is represented by (x 1i ,y 1i ) and (x 2i ,y 2i ), i∈n, n is the number of faces detected in the current frame, i is the ith face in the current frame, (x 1i ,y 1i ) is the upper left corner coordinate of the face frame, and (x 2i ,y 2i ) is the right lower corner coordinate of the face frame; the human body tracking result of each frame is represented by (X 1j ,Y 1j ) and (X 2j ,Y 2j ), j∈m, m is the number of human bodies detected in the current frame, j is the jth human body in the current frame, (X 1j ,Y 1j ) is the upper left corner coordinate of the human body frame, and (X 2j ,Y 2j ) is the right lower corner coordinate of the human body frame; when the coordinates (x 1i ,y 1i ), (x 2i ,y 2i ), (X 1j ,Y 1j ), (X 2j ,Y 2j ) satisfy the positional relationship as shown in formula 1, the id information of the face i is transmitted to the human body j; The two-dimensional mapping relationship between the laboratory plan and the ground in video monitoring is established, and only the coordinates of the foot center of the human body on the ground in the monitoring video can obtain the corresponding position of the human body in the laboratory plan; The corresponding coordinates of the upper left corner and the lower right corner of the human body tracking frame are (x 1i ,y 1i ) and (x 2i ,y 2i ), respectively. For the case that the whole human body enters the monitoring field of view, (1 / 2x 1i +1 / 2x 2i ,y 2i ) is selected as the foot center coordinate of the human body. For the case that the foot region of the human body does not enter the monitoring video, the position of the foot center is predicted according to the position and width of the human body tracking frame. Through analysis of 1632 monitoring video images, it can be obtained that when the human body tracking frame accurately covers the whole region of the human body, the aspect ratio of the tracking frame is between 2.63 and 2.

95. Even if the legs of the human body do not enter the monitoring field of view, the width w of the tracking frame is accurate. Therefore, in this case, the foot center coordinate is predicted based on the average value of the aspect ratio of the tracking frame, and the coordinate is expressed as (1 / 2x 1i +1 / 2x 2i ,y 1i +2.79x 2i -x 1i ). In a single camera, according to the foot center coordinates of each human body in each frame and the two-dimensional mapping relationship of the laboratory plan, the coordinates of the person in each frame are determined, and the coordinates of the same person are connected in time sequence to form a trajectory in the plan. Since the shooting of a single camera will leave a visual dead angle in the laboratory, and the more edge the single wide-angle camera is, the more serious the barrel distortion will be, which reduces the accuracy of camera calibration. Therefore, for the case of multiple monitoring cameras in multiple directions in the laboratory, a trajectory fusion scheme based on multiple cameras is designed. Assuming there are r monitoring cameras in the same laboratory, for the same person at the same time, r camera internal coordinates can be detected, and r predicted position points of the person in the laboratory plan are obtained; in theory, the above r points should completely coincide in the plan, but due to camera distortion, calibration error, tracking error and other problems, they will be scattered in a local sub-area of the plan; first, the local reachable density between sample points is judged by a local anomaly factor detection algorithm, the local reachable density being the inverse of the average reachable distance between the r predicted position points, the higher the sample point density, the smaller the probability that the sample point is an anomaly factor; the local reachable density threshold is set to 30, if the local reachable density of a sample point to other points is less than 30, the point is an outlier, otherwise it is a non-outlier; remove the outliers in the r points according to this method, and then calculate the coordinates of the person in the plan at the current time according to the remaining r' coordinate points The calculation formula is shown as formula 6; The same person coordinates obtained based on formula 6 are sorted in time sequence, and more robust trajectories in the laboratory plan can be obtained compared with single camera.

2. The personnel behavior monitoring method for laboratory safety management according to claim 1, characterized in that: When the similarity between the face in the video and the face of the laboratory access personnel is below a certain threshold, there are two reasons to explain this phenomenon; Reason one, the face is too small or the face is a side face with a large tilt angle, resulting in limited face features extracted, which is not enough to distinguish the id information of the face; Reason two, a person not on the laboratory access list enters the laboratory; The threshold setting method is as follows: after extracting the features of the faces in the video and the faces of the laboratory access personnel by the backbone network, each image is converted into a matrix with dimensions of 10*10*3, which represents the features of the face. According to the cosine similarity, the similarity between the two faces is calculated, and the similarity threshold is set to 90%, that is, whether the cosine similarity of the feature matrix of the two faces is less than 90% is compared; To ensure that all face pictures with low similarity are non-access personnel, two restrictions are set to filter faces with similarity below the threshold; Limit one, do not intercept face images with resolution less than 30*30; Limit two, do not intercept non-frontal face images; Based on the five key points of the center of the left eye, the center of the right eye, the center of the nose, the left corner of the mouth and the right corner of the mouth, it is judged whether the face is a frontal face; First, connect the left eye center, right eye center, right corner of the mouth, left corner of the mouth in order to form a quadrilateral region; Then lock the aspect ratio of the region, reduce the area to 1 / 4 of the original area, and get a sub-region with the same center position as the original region; Finally, it is judged whether the coordinates corresponding to the nose center are located in the sub-region, if yes, it is a frontal face, if not, it is a side face or a low head state; The face with resolution greater than 30*30 and frontal face is expanded by 1.5 times, and the image of the region is intercepted and stored in the warning folder, and the image is named with the current timestamp.

3. The personnel behavior monitoring method for laboratory safety management according to claim 1, wherein: The conversion relationship between the laboratory plan and the laboratory video scene coordinates is shown in formula 2, In the formula, a, b, c, d, c, f, g, h are parameters of the transformation matrix, X', Y' are a set of calibration coordinate points in the laboratory plan, x', y' are corresponding laboratory video scene coordinates, and W is a weight of the transformation matrix. According to the matrix multiplication rule, W can be converted into formula 3, W = gx' + hy' + 1, (3) Based on formula 2 and formula 3, X', Y' are expressed in the form of a matrix as formula 4, More than 4 key points are calibrated in the laboratory plan, and corresponding coordinate points in the laboratory video scene are found, formula 5 is obtained by bringing formula 4, and the transformation matrix parameters are calculated based on the least square method; the number of key points in the laboratory plan is represented as p, and formula 5 is obtained. According to the transformation matrix parameters, any point on the ground in the laboratory video field of view can obtain a unique coordinate point corresponding to it in the laboratory plan.

4. The personnel behavior monitoring method for laboratory safety management according to claim 1, characterized in that: Each functional area and fixed facility in the plan is divided into an area adjacent thereto as a basis for judging whether the laboratory personnel pass through the area; based on the extracted trajectory in the plan and the range of each sub-area, the time and sequence of the laboratory personnel passing through each area and facility are judged, and whether the laboratory personnel have non-standard process phenomena is further judged.

Images