Experimenter monitoring method and device based on behavior analysis and medium

Abstract

The invention relates to the technical field of laboratory safety management. The experimenter monitoring method based on behavior analysis comprises the steps of performing human body detection and tracking on video information through an RCNN algorithm to obtain experimenter position information in a laboratory, performing face recognition verification on the experimenter information through a DLIB library to obtain experimenter identity information, and performing face recognition verification on the experimenter identity information through a DLIB library. If the identity information of the experimenter is successfully matched, carrying out behavior analysis on the experimenter in the laboratory through a behavior analysis model to obtain behavior state information, and judging whether the operation behavior of the experimenter accords with a preset laboratory safety management rule or not based on the behavior state information of the experimenter; and if the operation behavior of the experimenter does not conform to the preset laboratory safety management rule, safety early warning information is generated, and early warning levels are divided according to the safety early warning information. The method has the effect of improving the safety management of the laboratory.

Description

Technical Field

[0001] The present invention relates to the technical field of laboratory safety management, and in particular to a method, device and medium for monitoring laboratory personnel based on behavior analysis. Background Art

[0002] Laboratory safety management is an indispensable part of the modern scientific research environment. With the rapid development of science and technology, laboratories, as important places for scientific research, play a vital role in promoting scientific and technological innovation and social progress. However, due to the complexity and diversity of the experimental environment, involving a variety of hazardous chemicals, precision instruments and high-voltage equipment, how to effectively ensure the safety of laboratory personnel has become a key issue that needs to be solved urgently. Although the traditional management model has improved safety to a certain extent, it is still difficult to meet the growing safety needs.

[0003] Most existing intelligent monitoring systems only have basic video surveillance and personnel identification functions, and are unable to accurately analyze the behavior of laboratory personnel, and lack an automated safety warning mechanism. For example, existing human detection technology can identify the presence of personnel, but cannot determine whether the laboratory personnel are operating the laboratory equipment correctly. Face recognition technology can verify the identity of laboratory personnel, but cannot monitor violations during the experiment. Therefore, existing technologies are difficult to meet the high-precision and high-security management needs of laboratories. Summary of the invention

[0004] In order to improve the safety management of laboratories, the present application provides a method, device and medium for monitoring laboratory personnel based on behavior analysis.

[0005] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A method for monitoring experimenters based on behavior analysis, the method comprising: Obtaining informed consent information from the experimenter, and after the informed consent information is passed, obtaining video information in the laboratory; Performing human body detection and tracking on the video information by using the RCNN algorithm to obtain the location information of the experimenter in the laboratory; Perform face recognition verification on the experimenter information through the DLIB library to obtain the experimenter identity information; If the identity information of the experimenter matches successfully, the behavior analysis of the experimenter in the laboratory is performed through the behavior analysis model to obtain behavior status information; Based on the behavior status information of the experimenter, determine whether the operation behavior of the experimenter complies with the preset laboratory safety management rules, and if the operation behavior of the experimenter does not comply with the preset laboratory safety management rules, generate safety warning information; According to the said security warning information, divide the warning levels and take corresponding safety management measures based on the said warning levels.

[0006] By adopting the above technical solution, human body detection and tracking are carried out through the RCNN algorithm to ensure the real-time update of the position information of the experimental personnel. Combined with the DLIB library for face recognition verification, the identity of the experimental personnel is accurately recognized, preventing unauthorized personnel from entering the laboratory and improving the safety of the laboratory. The operation behaviors of the experimental personnel are intelligently analyzed by using the behavior analysis model to judge whether they conform to the laboratory safety management rules, which can effectively identify illegal operations and reduce potential safety hazards. For the detected illegal behaviors, the system will automatically generate security warning information and conduct warning processing according to different levels to ensure that corresponding safety management measures are taken in a timely manner and avoid the occurrence of safety accidents.

[0007] In a preferred example of the present application, it can be further configured that: the RCNN algorithm is used to perform human body detection and tracking on the video information to obtain the position information of the experimental personnel in the laboratory, including: Perform image denoising, brightness adjustment and resolution optimization on the video information to obtain preprocessed video information; Use the RCNN algorithm to perform target detection on the preprocessed video information to identify the experimental personnel in the laboratory and generate corresponding bounding box information; According to the corresponding bounding box information, extract the feature information of the experimental personnel from consecutive multiple frames of images of the video information, and establish the time series feature of the experimental personnel based on the feature information; Based on the time series feature, use the optical flow algorithm to match the movement trajectories of the experimental personnel in the consecutive multiple frames of images to obtain the experimental personnel information.

[0008] By adopting the above technical solution, performing image denoising, brightness adjustment and resolution optimization on the video information can ensure the quality of video data and improve the accuracy of subsequent target detection and recognition. Using the RCNN algorithm for target detection can accurately identify the experimental personnel in the laboratory and generate bounding box information to ensure that the system can locate the position of the experimental personnel in real time. Based on the bounding box information, extract the features of the experimental personnel and combine time series modeling to construct the time series feature data of the experimental personnel, so as to realize the continuous monitoring of the state of the experimental personnel. Using the optical flow algorithm for movement trajectory matching enables the system to accurately track the movement trajectory of the experimental personnel and ensure stable target tracking ability in complex scenarios.

[0009] In a preferred example, the present application can be further configured as follows: the DLIB library is used to perform face recognition verification on the information of the experimental personnel to obtain the identity information of the experimental personnel, including: According to the information of the experimental personnel, the facial area of the experimental personnel is cropped to obtain face feature information; The face key point detection model in the DLIB library is used to extract key points from the face feature information to obtain face geometric feature information; The face geometric feature information is compared with the face templates in the laboratory authorized face information library through the DNN model in the DLIB library to calculate the similarity score; If the similarity score is higher than the preset face recognition threshold, it is determined that the identity of the experimental personnel is successfully matched, and the identity information of the experimental personnel is obtained.

[0010] By adopting the above technical solution, according to the information of the experimental personnel, the facial area is cropped to ensure that only valid face information is extracted, improving the calculation efficiency of subsequent recognition. Then, the face key point detection model in the DLIB library is used for feature extraction to generate high-precision face geometric feature information, avoiding recognition errors caused by changes in light, angle or expression. Next, through the DNN model in the DLIB library, the extracted face geometric features are compared with the laboratory authorized face information library to calculate the similarity score, ensuring the accuracy of identity verification. Finally, if the similarity score exceeds the set threshold, the match is successful, and the system associates the identified identity information with the experimental activity to achieve identity tracking and permission management of the experimental personnel.

[0011] In a preferred example, the present application can be further configured as follows: if the identity information of the experimental personnel is successfully matched, the behavior analysis model is used to analyze the behavior of the experimental personnel in the laboratory to obtain behavior state information, and further includes: According to the identity information of the experimental personnel, the action change information of the experimental personnel is obtained; the behavior analysis model extracts features from the action change information to obtain a behavior feature vector, and performs trajectory analysis on the behavior feature vector to identify the change trend of the action; Perform pattern recognition on the change trend of the action, extract behavior features, classify the behavior features, identify the operation behavior of the experimental personnel, and obtain behavior state information.

[0012] By adopting the above technical solution, based on the identity information of the experimenter, the action change information is extracted to ensure the individual accuracy of behavior analysis. Then, a behavior analysis model is used for feature extraction to construct a behavior feature vector, and the change trend of the action is identified through trajectory analysis to capture the movement pattern of the experimenter. Subsequently, pattern recognition and classification are performed on the extracted behavior features to accurately judge the specific operation behavior of the experimenter, and finally, behavior status information is generated to provide a basis for laboratory safety monitoring.

[0013] In a preferred example, the present application can be further configured as follows: based on the behavior status information of the experimenter, it is determined whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, a safety warning information is generated, including: Based on the behavior status information of the experimenter, the behavior category information, operation duration information, and activity area information of the experimenter are extracted, and the behavior category information, operation duration information, and activity area information are respectively matched with the preset laboratory safety management rules; If any one of the behavior category information, operation duration information, or activity area information does not conform to the laboratory safety management rules, the safety warning information is generated.

[0014] By adopting the above technical solution, based on the behavior status information of the experimenter, key data such as behavior category, operation duration, and activity area are extracted to ensure the full - range monitoring of experimental operations. Then, the extracted information is matched with the preset laboratory safety management rules to achieve automatic compliance detection. If any one of the behavior category, operation duration, or activity area does not conform to the safety management rules, the system will automatically generate safety warning information and provide a decision - making basis for subsequent safety management and intervention.

[0015] In a preferred example, the present application can be further configured as follows: based on the behavior status information of the experimenter, it is determined whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, a safety warning information is generated, and it further includes: If the experimenter has corrected their behavior according to the laboratory safety management requirements after receiving the safety warning information, the real - time gesture information of the experimenter is obtained; The real - time gesture information is matched with the preset safety confirmation gesture. After successful matching, the behavior status information of the experimenter is updated, and the safety warning information is cancelled.

[0016] By adopting the above technical solution, after receiving the safety warning information, the experimenter obtains the rectification situation of the experimenter based on behavior monitoring to ensure that the operation complies with the laboratory safety management requirements. Then, the real-time gesture information of the experimenter is extracted, and the feature vector is calculated through gesture recognition technology and matched with the preset safety confirmation gesture. When the matching is successful, the behavior status information of the experimenter is updated, and the safety warning is cancelled to avoid continuously triggering the alarm after rectification and affecting the normal progress of the experimental activities.

[0017] In a preferred example of the present application, it can be further configured that: according to the safety warning information, the warning level is divided, and corresponding safety management measures are taken based on the warning level, including: According to the safety warning information, the violation behavior category, violation influence range, violation duration and violation times of the experimenter are extracted and respectively matched with the preset laboratory safety management rules to calculate the comprehensive violation score; Based on the comprehensive violation score, according to the preset warning level threshold, the safety warning information is divided into low-level warning, medium-level warning and high-level warning; If the safety warning information is divided into low-level warning, a prompt warning information is generated, a safety reminder is sent to the experimenter, and the violation behavior information is recorded; If the safety warning information is divided into medium-level warning, a warning warning information is generated and a violation notice is sent to the laboratory management personnel; If the safety warning information is divided into high-level warning, an emergency warning information is generated to trigger the alarm of the laboratory monitoring terminal.

[0018] By adopting the above technical solution, the violation behavior category, influence range, duration and violation times are extracted and matched with the laboratory safety management rules to calculate the comprehensive violation score, ensuring that the quantitative evaluation of violation behaviors is more accurate. Based on the comprehensive violation score, according to the preset warning level threshold, the safety warning information is divided into low-level, medium-level and high-level warnings, so that different severity violation behaviors can be reasonably handled. For low-level warnings, a safety reminder is sent and the violation behavior information is recorded to ensure that the experimenter is aware of the potential violation risks. For medium-level warnings, a violation notice is sent to the laboratory management personnel for timely intervention and supervision. For high-level warnings, an emergency warning information is generated and the alarm of the laboratory monitoring terminal is triggered to ensure that serious violation behaviors can be quickly discovered and emergency treatment can be taken.

[0019] In a preferred example of the present application, it can be further configured that: the method for monitoring experimenters based on behavior analysis further includes: During the information transmission process, an end-to-end encryption protocol is adopted to encrypt the video information, the experimental personnel's location information, the experimental personnel's identity information, and the behavior status information; During the process of storing and using the information, only the information for laboratory safety management is retained. For the high-risk sensitive information in the experimental personnel's identity information and the behavior status information, desensitization or anonymization processing is adopted; During the behavior analysis process, by processing the experimental personnel's identity information, only the characteristic information for laboratory safety monitoring is retained, and the complete experimental personnel's identity information is not stored.

[0020] By adopting the above technical solutions, the security of laboratory monitoring information can be effectively enhanced, the privacy of experimental personnel can be ensured, and at the same time, the efficiency and compliance of laboratory management can be improved. During the information transmission process, an end-to-end encryption protocol is adopted, so that the video information, the experimental personnel's location information, identity information, and behavior status information are always in an encrypted state in the network, preventing unauthorized access and information tampering, thereby reducing the risk of information leakage. During the information storage and use stage, only the information required for laboratory safety management is retained. For the high-risk sensitive information related to the identity of experimental personnel, through desensitization or anonymization processing, even if the information is leaked, the identity of experimental personnel cannot be directly identified or the complete behavior trajectory cannot be restored, thereby reducing the possibility of information abuse and meeting the requirements of international information privacy protection regulations. In addition, during the behavior analysis process, the complete experimental personnel's identity information is not stored, but only the characteristic information for laboratory safety monitoring is retained, such as behavior patterns, violation records, activity categories, etc., enabling the system to accurately identify and analyze the safety behavior of experimental personnel, improving the accuracy of monitoring and management efficiency, and at the same time avoiding monitoring blind spots caused by identity recognition errors.

[0021] The above-mentioned second invention object of the present application is achieved through the following technical solutions: An experimental personnel monitoring device based on behavior analysis, the experimental personnel monitoring device based on behavior analysis includes: A video acquisition module, which is used to obtain the informed consent information of experimental personnel, and after the informed consent information passes, obtain the video information in the laboratory; A human body detection and tracking module, which is used to perform human body detection and tracking on the video information through the RCNN algorithm to obtain the experimental personnel's location information in the laboratory; A face recognition module, which is used to perform face recognition verification on the experimental personnel information through the DLIB library to obtain the experimental personnel's identity information; A behavior analysis module, configured to, if the identity information of the experimental personnel is successfully matched, perform behavior analysis on the experimental personnel in the laboratory through a behavior analysis model to obtain behavior status information; A behavior compliance determination module, configured to, based on the behavior status information of the experimental personnel, determine whether the operation behavior of the experimental personnel complies with the preset laboratory safety management rules. If the operation behavior of the experimental personnel does not comply with the preset laboratory safety management rules, generate a safety warning information; A warning management module, configured to divide warning levels according to the safety warning information and take corresponding safety management measures based on the warning levels.

[0022] By adopting the above technical solution, human body detection and tracking are performed through the RCNN algorithm to ensure real-time update of the position information of the experimental personnel. In combination with the DLIB library for face recognition verification, the identity of the experimental personnel is accurately recognized, preventing unauthorized personnel from entering the laboratory and improving the safety of the laboratory. The operation behavior of the experimental personnel is intelligently analyzed by using the behavior analysis model to determine whether it complies with the laboratory safety management rules, which can effectively identify illegal operations and reduce potential safety hazards. For the detected illegal behaviors, the system will automatically generate safety warning information and perform warning processing according to different levels to ensure timely adoption of corresponding safety management measures and avoid the occurrence of safety accidents.

[0023] The above object three of the present application is achieved through the following technical solution: A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above experimental personnel monitoring method based on behavior analysis are implemented.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: 1. Human body detection and tracking are performed through the RCNN algorithm to ensure real-time update of the position information of the experimental personnel. In combination with the DLIB library for face recognition verification, the identity of the experimental personnel is accurately recognized, preventing unauthorized personnel from entering the laboratory and improving the safety of the laboratory. The operation behavior of the experimental personnel is intelligently analyzed by using the behavior analysis model to determine whether it complies with the laboratory safety management rules, which can effectively identify illegal operations and reduce potential safety hazards. For the detected illegal behaviors, the system will automatically generate safety warning information and perform warning processing according to different levels to ensure timely adoption of corresponding safety management measures and avoid the occurrence of safety accidents; 2. After receiving the safety warning information, the experimenter obtains the rectification situation of the experimenter based on behavior monitoring to ensure that the operations comply with the requirements of laboratory safety management. Then, the real-time gesture information of the experimenter is extracted, and the feature vector is calculated through gesture recognition technology and matched with the preset safety confirmation gesture. When the matching is successful, the behavior status information of the experimenter is updated, and the safety warning is cancelled to avoid continuous triggering of alarms after rectification, which affects the normal progress of the experiment; 3. The types, scopes, durations, and frequencies of violations are extracted and matched with the laboratory safety management rules to calculate the comprehensive violation score, ensuring a more accurate quantitative assessment of violations. Based on the comprehensive violation score, according to the preset warning level thresholds, the safety warning information is divided into low-level, medium-level, and high-level warnings, so that violations of different severities can be reasonably handled. For low-level warnings, safety reminders are sent, and the violation information is recorded to ensure that the experimenter is aware of the potential violation risks. For medium-level warnings, violation notices are sent to the laboratory management personnel for timely intervention and supervision. For high-level warnings, emergency warning information is generated, and the laboratory monitoring terminal alarm is triggered to ensure that serious violations can be quickly detected and emergency measures can be taken. Brief Description of the Drawings

[0025] Figure 1 is a flowchart of a method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 2 is an implementation flowchart of step S20 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 3 is an implementation flowchart of step S30 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 4 is an implementation flowchart of step S40 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 5 is an implementation flowchart of step S50 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 6 is an implementation flowchart after step S50 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 7 is an implementation flowchart of step S60 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 8 is an implementation flowchart after step S60 in the method for monitoring experimenters based on behavior analysis in an embodiment of the present application; Figure 9It is a principle block diagram of an experimental personnel monitoring device based on behavior analysis in an embodiment of the present application. Detailed implementation manners

[0026] The present application will be further described in detail below with reference to the accompanying drawings.

[0027] In one embodiment, as Figure 1 shown, the present application discloses a method for monitoring experimental personnel based on behavior analysis, which specifically includes the following steps: S10: Obtain the informed consent information of the experimental personnel. After the informed consent information passes, obtain the video information in the laboratory.

[0028] In this embodiment, the informed consent information refers to the confirmation information in which the experimental personnel clearly authorize the collection, storage, and use of personal information, video information, and behavior analysis data before information collection. This confirmation information can be obtained through electronic signature, paper signature, or system pop-up prompt, and includes the experimental personnel's knowledge and consent to the data usage, storage duration, access rights, and data protection measures. After the confirmation passes, the video information in the laboratory will be collected to ensure that the data processing process complies with privacy protection requirements and laboratory safety management specifications.

[0029] Specifically, before the experimental personnel enter the laboratory, obtain the informed consent information of the experimental personnel through an electronic signature system, a laboratory management platform, or a physical paper document. The electronic signature system can display the informed consent agreement through a mobile application, a web page, or a laboratory access control terminal. The experimental personnel can confirm by clicking to confirm, entering identity information, or electronic signature. If a paper agreement is used, it will be collected by the laboratory management personnel and entered into the system for archiving to ensure that all experimental personnel have clearly understood the video monitoring-related information before entering the laboratory. After the informed consent information passes, the system automatically updates the experimental personnel's permissions and triggers the video collection device to start working to ensure that the video information is obtained only under the premise of obtaining legal authorization.

[0030] S20: Perform human detection and tracking on the video information through the RCNN algorithm to obtain the position information of the experimental personnel in the laboratory.

[0031] Specifically, preprocess the acquired video information, including video frame segmentation, background modeling, and denoising processing, to improve the detection accuracy. In the preprocessed video frames, use the RCNN algorithm for human target detection. Extract the image features in the video frames through a convolutional neural network, and use the selective search algorithm to determine the regions that may contain the experimental personnel during the candidate region generation process. After determining the candidate regions, screen the candidate regions to filter out non-human targets. After obtaining the position information of the experimental personnel, use the Kalman filter to predict the trajectory of the detection target to reduce recognition errors caused by occlusion or temporary loss. At the same time, calculate the movement direction and speed of the experimental personnel to ensure stable and accurate tracking of the experimental personnel, thereby obtaining the real-time position information of the experimental personnel.

[0032] S30: Perform face recognition verification on the experimental personnel information through the DLIB library to obtain the identity information of the experimental personnel.

[0033] Specifically, after detecting the experimental personnel, first use the target box cropping technique to extract the facial region from the detected experimental personnel region, and use the bicubic interpolation algorithm to normalize the cropped face image to meet the input size requirements of the DLIB face recognition model. Then, use the face key point detection model in DLIB to extract 68 key points of the face, and perform alignment operations on the key points through geometric transformation methods to eliminate the influence of different shooting angles. Subsequently, use the face deep neural network (DNN) model in DLIB to extract the face feature vector, and calculate the Euclidean distance with the face template in the laboratory authorized face information database. If the similarity score exceeds the set threshold, it is determined that the identity of the experimental personnel matches successfully, and the identity information is recorded.

[0034] S40: If the identity information of the experimental personnel matches successfully, perform behavior analysis on the experimental personnel in the laboratory through the behavior analysis model to obtain the behavior state information.

[0035] Specifically, after the identity of the experimental personnel matches successfully, extract the key point information of the experimental personnel from the video frames, use the pose estimation algorithm to detect the joint points of the head, upper limbs, and lower limbs of the experimental personnel, and calculate the changes in the joint point coordinates over time. Calculate the Euclidean distance between adjacent key points in consecutive frames, use time series analysis methods to construct the action state vector, calculate the action change rate, trajectory curvature, and acceleration change, perform Fourier transform on the time series data, extract frequency features, classify the extracted motion patterns, use the trained behavior analysis model to classify the state vector, and determine the behavior category to which the current frame belongs. Associate the classification results with the time series to determine the complete behavior sequence and output the behavior state information of the experimental personnel.

[0036] S50: Based on the behavior status information of the experimenter, determine whether the operation behavior of the experimenter complies with the preset laboratory safety management rules. If the operation behavior of the experimenter does not comply with the preset laboratory safety management rules, generate safety warning information.

[0037] Specifically, after obtaining the behavioral status information of the experimenter, the operation category, operation time and operation area of ​​the experimenter are extracted, the mapping relationship between the operation category and the category in the preset experimental rules is calculated, the activity area of ​​the experimenter is analyzed, the minimum Euclidean distance between the experimenter's trajectory point and the boundary of the safe area is calculated, and it is judged whether it crosses the boundary. The stability of the operation behavior is calculated using the timing analysis method, and it is judged whether the operation behavior complies with the laboratory standard operation process. If the operation category does not match, the operation area crosses the boundary, or the operation time exceeds the preset threshold, a safety warning information is generated.

[0038] S60: According to the safety warning information, the warning level is divided, and corresponding safety management measures are taken based on the warning level.

[0039] Specifically, the violation category, violation impact range and violation occurrence time are extracted from the security warning information, and the comprehensive risk score of the violation is calculated. The preset weight coefficient is used to calculate, and risk coefficients are first assigned to different violations, and the number of violations and the duration of violations are normalized. All influencing factors are weighted and summed, and the calculated score is compared with the warning level threshold of the laboratory safety management rules. Low-level warnings, medium-level warnings and high-level warnings are divided according to the score range. Low-level warnings trigger safety reminders and record violations. Medium-level warnings send notifications to laboratory managers. High-level warnings trigger alarms and link the access control system to limit the activity permissions of experimental personnel.

[0040] In one embodiment, if Figure 2 As shown, in step S20, the human body is detected and tracked in the video information by using the RCNN algorithm to obtain the position information of the experimenter in the laboratory, including: S201: performing image denoising, brightness adjustment and resolution optimization on video information to obtain pre-processed video information.

[0041] Specifically, perform frame segmentation on the acquired video information, split the continuous video stream into independent image frames, and perform Gaussian filtering on each frame of the image to calculate the weighted mean of the pixel neighborhood, removing high-frequency noise. Perform brightness adjustment on the filtered image, calculate the pixel gray-level distribution through histogram equalization, and redistribute the gray values to make the gray-level distribution tend to be uniform. Optimize the resolution of the image with adjusted brightness, use the bilinear interpolation method to calculate the pixel values at the target resolution. When calculating, perform weighted averaging on the four neighborhoods of the original pixels, and calculate the gray value of the target pixel point through the interpolation formula to make the resolution match the input requirements of the subsequent processing flow, and finally output the preprocessed video information.

[0042] S202: Use the RCNN algorithm to perform object detection on the preprocessed video information, identify the experimenters in the laboratory, and generate the corresponding bounding box information.

[0043] Specifically, perform frame-by-frame parsing on the preprocessed video information, input each frame of the image into the RCNN model for object detection. The RCNN model first performs convolution operations on the input image, uses multiple convolutional kernels to extract the local features of the image, and reduces the feature dimension through pooling operations. The extracted feature map is input into the region proposal network. The region proposal network performs sliding window scanning on the feature map, generates multiple candidate regions, and calculates the confidence scores of the candidate regions. The candidate regions are screened by the non-maximum suppression method to remove overlapping regions and retain the target regions with the highest confidence. Calculate the feature vectors of the screened candidate regions and input them into the fully connected layer for object classification. The object classification network calculates the class probabilities of each candidate region and filters the experimenter targets according to the classification results. For the target regions classified as experimenters, use the bounding box regression method to calculate the exact bounding box coordinates of the targets, calculate the center point coordinates, width, and height of the bounding box, and optimize the coordinate offsets of the bounding box based on the regression loss function, and finally output the bounding box information of the experimenters.

[0044] S203: According to the corresponding bounding box information, extract the feature information of the experimenters from multiple consecutive frames of images in the video information, and establish the time series features of the experimenters based on the feature information.

[0045] Specifically, image data of the area where the experimenter is located is extracted from the detected bounding box information, and feature calculations are performed on the extracted image data. First, a convolutional neural network is used to perform convolutional operations on the bounding box area to extract the local texture features, contour information, and color histogram information of the experimenter. The extracted feature vectors are L2-normalized to standardize the feature scale, and the normalized feature vectors are stored. In consecutive multiple frames of images, the cosine similarity is calculated for the feature vectors of the experimenter in adjacent frames to match the same target in adjacent frames. Based on the matching results, the movement trajectories of the experimenter are associated to construct the time series data of the experimenter. The time series data includes the position information, pose features, and movement direction of the experimenter at different time points. The Kalman filtering method is used to perform state estimation on the time series data, predict the target position at the next moment, and perform trajectory compensation for short-term target loss situations, finally forming the time series features of the experimenter.

[0046] S204: Based on the time series features, use the optical flow algorithm to match the movement trajectories of the experimenter in consecutive multiple frames of images to obtain the experimenter information.

[0047] Specifically, in consecutive multiple frames of images, feature points of the experimenter in adjacent frames are extracted based on the time series features, and the optical flow information of the feature points is calculated. First, within the bounding box area of the experimenter in the current frame and the previous frame, the Shi-Tomasi corner detection method is used to select feature points. The selection of feature points is sorted according to the image gradient response value, and the feature point with the highest response value is selected as the tracking target. Subsequently, the optical flow algorithm is used to calculate the displacement of the feature points in adjacent frames. During the calculation process, the motion components of the feature points in the horizontal and vertical directions are solved based on the optical flow constraint equation. After obtaining the displacement vectors of the feature points, the motion information of all feature points is clustered, and the overall movement trajectory of the experimenter is calculated. In multiple frames of images, based on the time series information of the trajectory, the Kalman filter is used to predict the position of the experimenter, calculate the optimal matching relationship between the current frame and the predicted frame, and update the motion state of the experimenter. If the match is successful, a unique target ID is assigned, and the position information, movement direction, and speed of the experimenter are stored. Finally, the experimenter information is obtained. The experimenter information includes the target identity identifier, trajectory point coordinates, motion vector, and timestamp.

[0048] In one embodiment, as Figure 3 shown, in step S30, the face recognition verification of the experimenter information is performed through the DLIB library to obtain the experimenter identity information, including: S301: According to the experimenter information, crop the image of the face area of the experimenter to obtain the face feature information.

[0049] Specifically, based on the bounding box coordinates in the experimenter information, determine the position of the experimenter in the image frame, and use a face detection algorithm to locate the facial region. First, within the experimenter's bounding box, use a multi-level Haar feature classifier or the deep learning-based MTCNN model to detect the face region. After detecting the face region, extract the bounding box coordinates of the face, and calculate the cropping range of the face region based on the bounding box coordinates. Use the bilinear interpolation method to normalize the size of the cropped image so that the face image adapts to the input size of the subsequent feature extraction model. Perform a color space conversion on the normalized image, converting the RGB image to a grayscale image to reduce the computational complexity, and perform histogram equalization on the grayscale image to enhance the local contrast. Finally, output the processed face image as face feature information.

[0050] S302: Use the face keypoint detection model in the DLIB library to extract keypoints from the face feature information to obtain face geometric feature information.

[0051] Specifically, input the cropped and normalized face image into the face keypoint detection model of the DLIB library, and use the pre-trained deep learning model to detect feature points in the face region. First, use a convolutional neural network to extract features from the input face image, calculate multi-level feature maps, and predict the positions of face keypoints through a regression model. During the feature extraction process, the model uses a 68-point keypoint detection algorithm to locate the coordinate points of the eyes, eyebrows, nose, mouth, and jaw contour of the face respectively. After detecting the keypoints, convert the keypoint coordinates to a relative coordinate system, and calculate the Euclidean distance between the keypoints. At the same time, calculate the facial geometric feature parameters based on the keypoint coordinates, including the inter-ocular distance, inter-brow distance, nose width, mouth width, and jaw angle, etc. Finally, output the face geometric feature information, including the coordinate data of the keypoints and the facial geometric ratio information.

[0052] S303: Compare the face geometric feature information with the face templates in the laboratory authorized face information library through the DNN model in the DLIB library to calculate the similarity score.

[0053] Specifically, the extracted human face geometric feature information is input into the DNN face recognition model in the DLIB library. First, the coordinates of the key points of the human face are normalized to keep the feature data in the same scale. After normalization, the human face image is input into the deep neural network, and feature extraction is performed through multiple convolutional layers, and a 128-dimensional feature vector is generated in the fully connected layer. The extracted feature vector undergoes L2 normalization to ensure that the numerical range of the feature vector is consistent. After obtaining the feature vector, the stored human face template data is loaded from the laboratory authorized human face information database, and the 128-dimensional feature vector of each template in the database is extracted using the same DNN model. The cosine similarity between the input human face feature vector and the human face template feature vector in the database is calculated. The similarity calculation formula is the dot product of the two vectors divided by their respective L2 norms. The calculated similarity score is used to measure the similarity degree of human face matching and is compared with the set matching threshold, and finally the similarity score is output.

[0054] S304: If the similarity score is higher than the preset human face recognition threshold, it is determined that the identity of the experimental personnel matches successfully, and the identity information of the experimental personnel is obtained.

[0055] Specifically, after the calculated similarity score is obtained, it is compared with the preset human face recognition threshold. First, the recognition threshold is set as the determination criterion for the identity matching of the experimental personnel. If the similarity score is greater than or equal to the recognition threshold, it is determined that the input human face feature matches successfully with a certain stored template in the laboratory authorized human face information database. After successful matching, the identity information of the experimental personnel, including name, job number, permission level, etc., is extracted from the matching human face template, and the identity information of the experimental personnel is associated with the current timestamp, and the successfully matched identity data is recorded, and finally the identity information of the experimental personnel is output.

[0056] In one embodiment, as Figure 4 shown, in step S40, that is, if the identity information of the experimental personnel matches successfully, the behavior of the experimental personnel in the laboratory is analyzed through the behavior analysis model to obtain the behavior state information, and it also includes: S401: According to the identity information of the experimental personnel, the action change information of the experimental personnel is obtained; the behavior analysis model extracts features from the action change information to obtain the behavior feature vector, and performs trajectory analysis on the behavior feature vector to identify the change trend of the action.

[0057] Specifically, based on the identity information of the experimenter, retrieve the position information of the corresponding experimenter in the video frame, and extract the limb key point data of the experimenter from consecutive multiple frames of images. First, use the pose estimation algorithm to detect the key points of the experimenter's head, torso, limbs, etc., and calculate the position changes of the key points in adjacent frames to construct the action change information of the experimenter. After obtaining the key point data, use the time series modeling method to extract features from the action data, convert the position information of the key points into a behavior feature vector, and the behavior feature vector consists of data such as the joint angle, displacement, acceleration, and movement direction of the experimenter. The extracted behavior feature vector is analyzed by a time series neural network such as LSTM (Long Short-Term Memory Network) or Transformer. During the analysis process, calculate the change rate of the behavior feature vector in the time dimension, and eliminate abnormal data through the trajectory smoothing algorithm to finally obtain the movement trend of the experimenter.

[0058] S402: Perform pattern recognition on the change trend of the action, extract the behavior features, classify the behavior features, identify the operation behavior of the experimenter, and obtain the behavior state information.

[0059] Specifically, based on the change trend of the action, extract the behavior features of the experimenter from the time series data. First, perform standardization processing on the extracted key point trajectory, convert parameters such as the movement amplitude, speed, and direction of the experimenter into a feature vector of a fixed scale, use Fourier transform or wavelet transform to analyze the movement frequency, and calculate the action period and posture change pattern of the experimenter. After obtaining the standardized behavior features, input the feature vector into a trained deep learning model for classification. The classification model is based on a convolutional neural network (CNN) and a long short-term memory network (LSTM), analyzes the changes of the behavior features in the time dimension, and classifies the operation behavior of the experimenter into preset operation types. The operation categories output by the classification correspond to the behavior labels in the laboratory safety management rules. After the operation category recognition is completed, combine additional information such as the behavior duration and operation location of the experimenter to generate the complete behavior state information.

[0060] In one embodiment, as Figure 5 shown, in step S50, that is, based on the behavior state information of the experimenter, determine whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, generate a safety warning information, including: S501: Based on the behavior state information of the experimenter, extract the behavior category information, operation duration information, and activity area information of the experimenter, and match the behavior category information, operation duration information, and activity area information with the preset laboratory safety management rules respectively.

[0061] Specifically, the behavior category, operation time, and activity area are parsed from the behavior status information of the experimenter. First, the operation behavior category of the experimenter is obtained from the output result of the classification model, and the duration of the operation behavior is calculated through time series data. The calculation method of the operation duration is based on the time stamps of behavior classification, recording the start and end time points of the behavior, and calculating the time difference between the two to obtain the operation duration information of the experimenter. At the same time, the activity area data is extracted from the position information trajectory of the experimenter, the movement trajectory of the experimenter is calculated, and it is matched with the laboratory area division data to determine the current experimental operation area where the experimenter is located. After obtaining the behavior category, operation duration, and activity area, based on the laboratory safety management rule library, the behavior category of the experimenter is sequentially matched to see if it conforms to the operation specifications, the operation duration is calculated to see if it is within the specified time range, and it is judged whether the activity area is within the allowable range, and finally the matching result is output.

[0062] S502: If any one of the behavior category information, operation duration information, or activity area information does not conform to the laboratory safety management rules, a safety warning message is generated.

[0063] Specifically, first, check whether the behavior category of the experimenter belongs to the operation types allowed in the laboratory. If the behavior category is not in the allowed operation list, it is marked as a violation operation. Second, calculate the operation duration of the experimenter and compare it with the preset time threshold. If the operation duration exceeds the upper limit set by the safety management rules or is lower than the minimum requirement, it is determined that the operation duration is abnormal. Finally, based on the position information of the experimenter, the activity range of the experimenter is compared with the laboratory area division data, and the overlap degree between the trajectory points of the experimenter and the allowed operation area is calculated. If the experimenter enters a restricted area or operates in an unauthorized area, it is determined that the activity area is in violation. If any of the above detection results does not conform to the safety management rules, a safety warning message is generated.

[0064] In one embodiment, as Figure 6 shown, after step S50, that is, based on the behavior status information of the experimenter, it is judged whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, a safety warning message is generated, and it further includes: S503: If the experimenter has corrected the behavior according to the laboratory safety management requirements after receiving the safety warning message, obtain the real-time gesture information of the experimenter.

[0065] Specifically, after the experimenter receives the safety warning information, the system continuously monitors the experimenter's behavior status and determines whether there is any adjustment to the violation behavior. First, the experimenter's latest operation behavior is detected through the behavior analysis model, and compared with the behavior category before the violation. If it is detected that the behavior category has switched from the violation category to the operation category that complies with the laboratory safety management rules, the gesture recognition module is triggered, and the video frame parsing method is called to extract the upper limb area of ​​the experimenter from the real-time video stream. The hand key point detection is performed on the extracted image area, and the MediaPipeHands or OpenPoseHand model is used to extract the experimenter's hand key point coordinates, including the spatial positions of the fingertips, palms and joints. The key point data is normalized and converted into a feature vector, and the feature vector is input into the pre-trained gesture classification model. The model identifies whether the experimenter has performed the preset safety confirmation gesture based on the relative position and movement pattern of the hand joints. If the recognition result matches the safety confirmation gesture set by the laboratory management system, the gesture recognition result is stored and the subsequent safety warning release process is entered.

[0066] S504: Match the real-time gesture information with the preset safety confirmation gesture. After the match is successful, update the behavior status information of the experimenter and cancel the safety warning information.

[0067] Specifically, based on the real-time gesture information of the experimenter, the gesture feature vector is input into the gesture matching module and compared with the safety confirmation gesture database preset in the laboratory. First, the real-time gesture features are normalized, and the Euclidean distance or cosine similarity is used to calculate the matching degree between the real-time gesture and the preset gesture template. During the calculation process, a spatial feature description is established based on key point data such as fingertips, palms, and joint angles, and the similarity between the skeletal angle changes of the real-time gesture and the standard gesture template is calculated. If the matching degree exceeds the preset gesture matching threshold, the gesture matching is determined to be successful. After the matching is successful, the behavior status information of the experimenter is updated, the violation mark is removed, and the current behavior category of the experimenter is adjusted to a state that meets the laboratory safety management requirements. At the same time, the gesture confirmation time and recognition results are recorded in the database, and then the safety warning release mechanism is triggered, and the warning management module is called to mark the experimenter's safety warning information as released, remove the experimenter's violation record, and restore the experimenter's normal experimental authority.

[0068] In one embodiment, if Figure 7 As shown, in step S60, the warning level is divided according to the security warning information, and corresponding safety management measures are taken based on the warning level, including: S601: Extract the violation behavior category, violation impact scope, violation duration, and violation frequency of the experimental personnel according to the security warning information, and respectively match them with the preset laboratory safety management rules to calculate the comprehensive violation score.

[0069] Specifically, obtain the violation records of the experimental personnel from the security warning information database, extract the violation behavior category, and determine the specific type of violation according to the attributes of the violation behavior, such as not wearing safety equipment, operating equipment without authorization, entering restricted areas in violation, etc. Calculate the violation impact scope, analyze the impact scope of the violation behavior on the equipment, other experimental personnel, and experimental procedures within the experimental area based on the location information trajectory of the experimental personnel, calculate the violation duration, record the start time and end time of the violation occurrence, and calculate the length of the violation duration. Count the violation frequency, query the historical violation records of the experimental personnel, and calculate the cumulative number of violations within a certain period in the past. Compare the above violation parameters with the standard violation data in the laboratory safety management rule library respectively, match the classification of the violation severity set by the laboratory, set the weight coefficients, perform weighted calculations on the violation behavior category, violation impact scope, violation duration, and violation frequency respectively, and perform normalization processing to calculate the comprehensive violation score.

[0070] S602: Based on the comprehensive violation score, divide the security warning information into low-level warnings, medium-level warnings, and high-level warnings according to the preset warning level thresholds.

[0071] Specifically, based on the comprehensive violation score, call the warning level division standard in the laboratory safety management rule library, and map the violation score to the corresponding warning level according to the set warning level thresholds. First, set the warning level threshold range, divided into low-level warnings, medium-level warnings, and high-level warnings. After calculating the comprehensive violation score, compare the score value with the thresholds of each warning level. If the score is lower than the upper threshold of the low-level warning, it is marked as a low-level warning. If the score is between the upper threshold of the low-level warning and the upper threshold of the medium-level warning, it is marked as a medium-level warning. If the score is higher than the upper threshold of the medium-level warning, it is marked as a high-level warning.

[0072] S603: If the security warning information is divided into low-level warnings, generate a prompt warning message, send a safety reminder to the experimental personnel, and record the violation behavior information.

[0073] Specifically, after determining that the level of the safety warning information is a low-level warning, a prompt warning information is generated. The prompt warning information includes the identity of the laboratory personnel, the type of violation, the time of violation occurrence, and the location of violation. A notice is sent to the terminal device of the laboratory personnel through the laboratory management system, and the violation information and the proposed rectification measures are displayed. The violation behavior of the laboratory personnel is recorded in the laboratory management database, and the time of violation occurrence, the type of violation, and the warning level are stored.

[0074] S604: If the safety warning information is classified as a medium-level warning, a warning warning information is generated and a violation notice is sent to the laboratory management personnel.

[0075] Specifically, after determining that the level of the safety warning information is a medium-level warning, a warning warning information is generated. The warning warning information includes the identity of the laboratory personnel, the type of violation, the time of violation occurrence, the scope of violation impact, and the duration of violation. The laboratory management personnel are notified through email, text message, or the laboratory management system, and the laboratory management personnel are informed to check the violation situation and take appropriate measures.

[0076] S605: If the safety warning information is classified as a high-level warning, an emergency warning information is generated and the laboratory monitoring terminal alarm is triggered.

[0077] Specifically, after determining that the level of the safety warning information is a high-level warning, an emergency warning information is generated. The emergency warning information includes the identity of the laboratory personnel, the type of violation, the location of violation, the time of violation occurrence, and the possible laboratory safety risks caused. An emergency notice is sent to the relevant laboratory responsible persons, the laboratory emergency management team, and the safety supervisor through the laboratory safety management system. At the same time, the laboratory monitoring terminal alarm is triggered, and the laboratory alarm system is called to activate the sound alarm, the flashing warning light, or the video monitoring prompt.

[0078] In one embodiment, as Figure 8 shown, after step S60, that is, the experimental personnel monitoring method based on behavior analysis, further includes: S70: During the information transmission process, an end-to-end encryption protocol is used to encrypt the video information, the experimental personnel location information, the experimental personnel identity information, and the behavior status information.

[0079] In this embodiment, the information refers to the data collected, transmitted, and stored during the operation of the laboratory monitoring system, including but not limited to the high-definition video stream in the laboratory, the real-time location information of the experimental personnel, the identity authentication data (such as face feature vectors, identity codes), the behavior status analysis results (such as behavior classification, violation behavior recognition results, warning information), etc.; Specifically, when transmitting video information, to prevent unauthorized interception or tampering, an encrypted transmission method is adopted. The video data is encrypted during the transmission process so that even if the data is intercepted in the network, the video content cannot be directly viewed. When transmitting the location information and identity information of the experimental personnel, an encryption algorithm is used to transform the data so that it cannot be directly read during the transmission process. At the same time, before the data reaches the receiving end, all information remains encrypted, and only authorized devices can decrypt it. In addition, a unique security key is generated before each data transmission to ensure that even if the same information is transmitted multiple times, the encryption results are different each time, thereby improving data security and preventing attackers from cracking the information by repeatedly analyzing the data stream.

[0080] S80: During the process of storing and using information, only the information used for laboratory safety management is retained. For the high-risk sensitive information in the identity information and behavior status information of the experimental personnel, desensitization or anonymization processing is adopted.

[0081] In this embodiment, the high-risk sensitive information refers to the information that may directly identify the personal identity of the experimental personnel or expose the core security information of the laboratory, including but not limited to the complete face image of the experimental personnel, ID number, fingerprint information, specific operation records of the experimental project, operation logs of the experimental equipment, and detailed behavior information of special experiments involving dangerous chemicals, biological experiments, confidential scientific research tasks, etc.

[0082] Specifically, when transmitting video information, to prevent unauthorized interception or tampering, an encrypted transmission method is adopted. The video data is encrypted during the transmission process so that even if the data is intercepted in the network, the video content cannot be directly viewed. When transmitting the location information and identity information of the experimental personnel, an encryption algorithm is used to transform the data so that it cannot be directly read during the transmission process. At the same time, before the data reaches the receiving end, all information remains encrypted, and only authorized devices can decrypt it. In addition, a unique security key is generated before each data transmission to ensure that even if the same information is transmitted multiple times, the encryption results are different each time, thereby improving data security and preventing attackers from cracking the information by repeatedly analyzing the data stream.

[0083] S90: During the behavior analysis process, by processing the identity information of the experimental personnel, only the characteristic information used for laboratory safety monitoring is retained, and the complete identity information of the experimental personnel is not stored.

[0084] In this embodiment, the characteristic information of laboratory safety monitoring refers to behavior analysis information used only for the purpose of laboratory safety management, including but not limited to the behavior patterns of experimental personnel (such as standing, walking, operating equipment), the detection results of violation behaviors (such as not wearing safety protection equipment, entering restricted areas), the characteristics of movement trajectories (such as the heat map of activities in key experimental areas), the usage of laboratory equipment (such as records of frequently operated equipment), etc., and does not include the complete identity information of experimental personnel (such as name, photo, ID number, etc.).

[0085] Specifically, during behavior analysis, the action characteristics of experimental personnel are extracted, such as gestures, walking paths, equipment operation behaviors, etc., without storing the complete face information. For example, when identifying whether an experimental personnel is using the experimental equipment correctly, only the hand movements are analyzed to see if they conform to the operation specifications, without recording the specific identity of the experimental personnel. For violation behavior records, only the type and time of the violation are retained, such as "illegal entry into a dangerous area" and "staying in an irrelevant area for a long time", without storing the complete behavior details or video footage. This can not only meet the needs of laboratory safety management but also ensure that personal identity information unrelated to safety is not stored, reducing the privacy risk of experimental personnel.

[0086] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0087] In one embodiment, a monitoring device for experimental personnel based on behavior analysis is provided. The monitoring device for experimental personnel based on behavior analysis corresponds one-to-one with the method for monitoring experimental personnel based on behavior analysis in the above embodiment. As Figure 9 shown, the monitoring device for experimental personnel based on behavior analysis includes a video acquisition module, a human body detection and tracking module, a face recognition module, a behavior analysis module, a behavior compliance determination module, and an early warning management module. The detailed description of each functional module is as follows: The video acquisition module is used to obtain the informed consent information of experimental personnel. After the informed consent information passes, it obtains the video information in the laboratory; The human body detection and tracking module is used to perform human body detection and tracking on the video information through the RCNN algorithm to obtain the position information of experimental personnel in the laboratory; The face recognition module is used to perform face recognition verification on the experimental personnel information through the DLIB library to obtain the identity information of experimental personnel; The behavior analysis module is used to, if the identity information of the experimental personnel matches successfully, perform behavior analysis on the experimental personnel in the laboratory through the behavior analysis model to obtain the behavior status information; The behavior compliance determination module is used to determine whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules based on the behavior status information of the experimenter. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, a safety warning message will be generated; The warning management module is used to divide the warning level according to the safety warning message and take corresponding safety management measures based on the warning level.

[0088] Optionally, the human body detection and tracking module includes: The video preprocessing sub-module is used to perform image denoising, brightness adjustment and resolution optimization on the video information to obtain the preprocessed video information; The target detection sub-module is used to perform target detection on the preprocessed video information using the RCNN algorithm, identify the experimenters in the laboratory, and generate corresponding bounding box information; The feature extraction and time series construction sub-module is used to extract the feature information of the experimenters from the continuous multi-frame images of the video information according to the corresponding bounding box information, and establish the time series features of the experimenters based on the feature information; The trajectory matching and motion analysis sub-module is used to match the motion trajectories of the experimenters in the continuous multi-frame images using the optical flow algorithm based on the time series features to obtain the experimenter information.

[0089] Optionally, the face recognition module includes: The face region extraction sub-module is used to crop the facial region of the experimenter according to the experimenter information to obtain the face feature information; The face key point detection sub-module is used to extract the key points of the face feature information using the face key point detection model in the DLIB library to obtain the face geometric feature information; The face feature matching sub-module is used to compare the face geometric feature information with the face templates in the laboratory authorized face information database through the DNN model in the DLIB library to calculate the similarity score; The identity verification sub-module is used to determine that the experimenter identity match is successful and obtain the experimenter identity information if the similarity score is higher than the preset face recognition threshold.

[0090] Optionally, the behavior analysis module includes: The action analysis sub-module is used to obtain the action change information of the experimenter according to the experimenter identity information; the behavior analysis model extracts features from the action change information to obtain the behavior feature vector, and performs trajectory analysis on the behavior feature vector to identify the change trend of the action; The behavior recognition submodule is used to perform pattern recognition on the changing trend of actions, extract behavior features, classify behavior features, identify the operating behavior of the experimenter, and obtain behavior status information.

[0091] Optionally, the behavior compliance determination module includes: The action analysis submodule is used to extract the behavior category information, operation time information and activity area information of the experimenter based on the behavior state information of the experimenter, and match the behavior category information, operation time information and activity area information with the preset laboratory safety management rules respectively; The behavior recognition submodule is used to generate safety warning information if any of the behavior category information, operation duration information or activity area information does not comply with the laboratory safety management rules.

[0092] Optionally, the behavior compliance determination module includes: The gesture detection submodule is used to obtain the real-time gesture information of the experimenter if the experimenter has corrected his behavior in accordance with the laboratory safety management requirements after receiving the safety warning information; The gesture matching and warning cancellation submodule is used to match the real-time gesture information with the preset safety confirmation gesture. After the match is successful, the behavior status information of the experimenter is updated and the safety warning information is cancelled.

[0093] Optionally, the early warning management module includes: The violation analysis and scoring submodule is used to extract the violation category, violation impact range, violation duration and number of violations of the experimenter based on the safety warning information, and match them with the preset laboratory safety management rules to calculate the comprehensive violation score; The warning level classification submodule is used to classify the security warning information into low-level warning, medium-level warning and high-level warning based on the comprehensive violation score and the preset warning level threshold; The low-level warning processing submodule is used to generate a warning message if the safety warning information is classified as a low-level warning, send a safety reminder to the experimenter, and record the violation information; The medium-level warning processing submodule is used to generate warning information if the safety warning information is classified as a medium-level warning, and send a violation notification to the laboratory manager; The high-level warning processing submodule is used to generate emergency warning information and trigger the laboratory monitoring terminal alarm if the safety warning information is classified as a high-level warning.

[0094] Optionally, the early warning management module includes: A secure transmission module, which is used to encrypt the video information, the location information of the experimental personnel, the identity information of the experimental personnel, and the behavior status information by using an end-to-end encryption protocol during the information transmission process; A privacy protection storage module, which is used to only retain the necessary information for laboratory safety management during the process of storing and using the information. For the highly risky sensitive information in the identity information of the experimental personnel and the behavior status information, desensitization or anonymization processing is adopted; A characterized behavior analysis module, which is used to only retain the characteristic information for laboratory safety monitoring by processing the identity information of the experimental personnel during the behavior analysis process, without storing the complete identity information of the experimental personnel.

[0095] For the specific limitations of the experimental personnel monitoring device based on behavior analysis, reference can be made to the limitations of the experimental personnel monitoring method based on behavior analysis in the above text, which will not be elaborated here. Each module in the above experimental personnel monitoring device based on behavior analysis can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: Obtain the informed consent information of the experimental personnel. After the informed consent information passes, obtain the video information in the laboratory; Perform human detection and tracking on the video information through the RCNN algorithm to obtain the location information of the experimental personnel in the laboratory; Perform face recognition verification on the experimental personnel information through the DLIB library to obtain the identity information of the experimental personnel; If the identity information of the experimental personnel matches successfully, perform behavior analysis on the experimental personnel in the laboratory through a behavior analysis model to obtain the behavior status information; Based on the behavior status information of the experimental personnel, judge whether the operation behavior of the experimental personnel conforms to the preset laboratory safety management rules. If the operation behavior of the experimental personnel does not conform to the preset laboratory safety management rules, generate a safety warning information; According to the safety warning information, divide the warning level and take corresponding safety management measures based on the warning level.

[0097] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0098] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0099] The above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An experimental personnel monitoring method based on behavior analysis, characterized in that The experimental personnel monitoring method based on behavior analysis includes: Obtain the informed consent information of the experimental personnel. After the informed consent information passes, obtain the video information in the laboratory; Perform human detection and tracking on the video information through the RCNN algorithm to obtain the position information of the experimental personnel in the laboratory; Perform face recognition verification on the experimental personnel information through the DLIB library to obtain the experimental personnel identity information; If the experimental personnel identity information matches successfully, perform behavior analysis on the experimental personnel in the laboratory through the behavior analysis model to obtain the behavior status information; Based on the behavior status information of the experimental personnel, determine whether the operation behavior of the experimental personnel conforms to the preset laboratory safety management rules. If the operation behavior of the experimental personnel does not conform to the preset laboratory safety management rules, generate a safety warning information; According to the safety warning information, divide the warning level and take corresponding safety management measures based on the warning level.

2. The experimental personnel monitoring method based on behavior analysis according to claim 1, wherein The performing human detection and tracking on the video information through the RCNN algorithm to obtain the position information of the experimental personnel in the laboratory includes: Perform image denoising, brightness adjustment, and resolution optimization on the video information to obtain the preprocessed video information; Use the RCNN algorithm to perform target detection on the preprocessed video information, identify the experimental personnel in the laboratory, and generate the corresponding bounding box information; According to the corresponding bounding box information, extract the feature information of the experimental personnel from the consecutive multi-frame images of the video information, and establish the time series feature of the experimental personnel based on the feature information; Based on the time series feature, use the optical flow algorithm to match the movement trajectory of the experimental personnel in the consecutive multi-frame images to obtain the experimental personnel information.

3. The experimental personnel monitoring method based on behavior analysis according to claim 1, characterized in that The performing face recognition verification on the experimental personnel information through the DLIB library to obtain the experimental personnel identity information includes: According to the experimental personnel information, crop the face area of the experimental personnel to obtain the face feature information; Use the face key point detection model in the DLIB library to extract the key points of the face feature information to obtain the face geometric feature information; Compare the face geometric feature information with the face template in the laboratory authorized face information library through the DNN model in the DLIB library to calculate the similarity score; If the similarity score is higher than the preset face recognition threshold, determine that the experimental personnel identity matches successfully and obtain the experimental personnel identity information.

4. The experimental personnel monitoring method based on behavior analysis according to claim 1, characterized in that, The if the experimental personnel identity information matches successfully, performing behavior analysis on the experimental personnel in the laboratory through the behavior analysis model to obtain the behavior status information further includes: According to the experimental personnel identity information, obtain the action change information of the experimental personnel; the behavior analysis model extracts features from the action change information to obtain the behavior feature vector, and performs trajectory analysis on the behavior feature vector to identify the change trend of the action. Perform pattern recognition on the changing trend of the actions, extract behavioral features, classify the behavioral features, identify the operation behaviors of the experimenter, and obtain behavioral status information.

5. The experimental personnel monitoring method based on behavior analysis according to claim 1, wherein, Based on the behavioral status information of the experimenter, determine whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, generate safety warning information, including: Based on the behavioral status information of the experimenter, extract the behavioral category information, operation duration information, and activity area information of the experimenter, and respectively match the behavioral category information, the operation duration information, and the activity area information with the preset laboratory safety management rules; If any one of the behavioral category information, the operation duration information, or the activity area information does not conform to the laboratory safety management rules, generate the safety warning information.

6. The experimental personnel monitoring method based on behavior analysis according to claim 1, characterized in that, Based on the behavioral status information of the experimenter, determine whether the operation behavior of the experimenter conforms to the preset laboratory safety management rules. If the operation behavior of the experimenter does not conform to the preset laboratory safety management rules, generate safety warning information, and further include: If the experimenter has corrected their behavior according to the laboratory safety management requirements after receiving the safety warning information, obtain the real-time gesture information of the experimenter; Match the real-time gesture information with the preset safety confirmation gesture. After successful matching, update the behavioral status information of the experimenter and cancel the safety warning information.

7. The method for monitoring experimenters based on behavior analysis according to claim 1, wherein According to the safety warning information, divide the warning level and take corresponding safety management measures based on the warning level, including: According to the safety warning information, extract the category of the experimenter's violation behavior, the scope of the violation impact, the duration of the violation, and the number of violations, and respectively match them with the preset laboratory safety management rules to calculate the comprehensive violation score; Based on the comprehensive violation score, according to the preset warning level threshold, divide the safety warning information into low-level warning, medium-level warning, and high-level warning; If the safety warning information is divided into a low-level warning, generate a prompt warning information, send a safety reminder to the experimenter, and record the violation behavior information; If the safety warning information is divided into a medium-level warning, generate a warning warning information and send a violation notice to the laboratory management personnel; If the safety warning information is divided into a high-level warning, generate an emergency warning information and trigger the alarm of the laboratory monitoring terminal.

8. The method for monitoring experimental personnel based on behavior analysis according to claim 1, characterized in that, The method for monitoring experimenters based on behavior analysis further includes: During the information transmission process, use an end-to-end encryption protocol to encrypt the video information, the experimenter's location information, the experimenter's identity information, and the behavioral status information; During the process of storing and using the information, only retain the information for laboratory safety management. For the high-risk sensitive information in the experimenter's identity information and behavioral status information, perform desensitization or anonymization processing; During the behavior analysis process, by processing the identity information of the experimental personnel, only the feature information for laboratory safety monitoring is retained, and the complete identity information of the experimental personnel is not stored.

9. An experimental personnel monitoring device based on behavior analysis, characterized in that The experimental personnel monitoring device based on behavior analysis includes: A video acquisition module, which is used to obtain the informed consent information of the experimental personnel. After the informed consent information passes, it acquires the video information in the laboratory. A human body detection and tracking module, which is used to perform human body detection and tracking on the video information through the RCNN algorithm to obtain the position information of the experimental personnel in the laboratory. A face recognition module, which is used to perform face recognition verification on the experimental personnel information through the DLIB library to obtain the identity information of the experimental personnel. A behavior analysis module, which is used to, if the identity information of the experimental personnel matches successfully, perform behavior analysis on the experimental personnel in the laboratory through a behavior analysis model to obtain behavior status information. A behavior compliance determination module, which is used to, based on the behavior status information of the experimental personnel, determine whether the operation behavior of the experimental personnel conforms to the preset laboratory safety management rules. If the operation behavior of the experimental personnel does not conform to the preset laboratory safety management rules, a safety warning information is generated. A warning management module, which is used to, according to the safety warning information, divide the warning level and take corresponding safety management measures based on the warning level.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the experimental personnel monitoring method based on behavior analysis according to any one of claims 1 to 8.

Images