Laboratory digital management method and system

Abstract

The invention relates to the field of digital twinning, in particular to a laboratory digital management method and system. The method comprises the following steps: acquiring a real-time panoramic monitoring video of a laboratory; performing multi-object visual identification and regional layout analysis on the laboratory real-time panoramic monitoring video, and constructing a laboratory panoramic structure diagram; identifying laboratory equipment state parameters based on multiple sensors; performing equipment position registration on the laboratory panoramic structure diagram according to the laboratory equipment state parameters, and constructing a laboratory panoramic digital model; performing time sequence inter-frame delay optimization on the laboratory real-time panoramic monitoring video, performing dynamic optical flow tracking, and generating a time sequence moving track and an operation behavior track sequence of each person; and performing full-period dynamic behavior trajectory evolution according to the time sequence moving trajectory and the operation behavior trajectory sequence of each person, so as to generate a dynamic trajectory rendering twinborn model. According to the invention, the management efficiency of the laboratory is improved, and the safety of the laboratory is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of digital twins, and particularly to a method and system for digital management of laboratories. Background Art

[0002] With the continuous progress of scientific research and technological development, the work processes and management models of laboratories are also facing increasingly complex challenges. Traditional laboratory management methods rely on manual operations and paper records, suffering from problems such as lagging data transmission, information asymmetry, and low management efficiency. With the continuous increase in laboratory scientific research activities and the growing diversification of experimental projects, traditional management means are no longer sufficient to meet the requirements of efficient and precise laboratory operations. Especially in aspects such as experimental data collection, equipment monitoring, environmental control, and personnel management, manual operations not only have the potential for errors but also cannot provide real-time and comprehensive monitoring and analysis.

[0003] In recent years, with the rapid development of information technology, the Internet of Things, artificial intelligence, and big data technology, more and more intelligent devices and systems have been applied to laboratory management, providing new solutions for the automation and intelligence of laboratory management. Against this background, the digital full-cycle management of laboratories has become the key to modern laboratory management. By comprehensively monitoring and performing real-time data analysis on all aspects of the laboratory through digital means, the management efficiency of the laboratory is greatly improved, the accuracy and security of experimental data are ensured, and at the same time, the efficient operation of laboratory scientific research activities is promoted.

[0004] However, the full-cycle management of laboratories not only involves the real-time monitoring of equipment operation status but also includes multiple aspects such as the intelligent analysis of experimental data, the dynamic adjustment of the experimental environment, and the management of the behavior trajectories of experimental personnel. This requires the management system to be able to, based on real-time monitoring, intelligently identify and analyze the dynamic changes of various elements such as equipment, environment, and personnel, timely discover potential risks and anomalies, and thus make precise management decisions. In addition, existing digital management systems mostly focus on a single aspect (such as equipment monitoring or data collection) and lack comprehensive solutions for the full-cycle management of laboratories. Therefore, in order to address the increasingly complex management requirements of modern laboratories, it is particularly urgent to propose a full-cycle management method based on digital technology. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method and system for digital management of laboratories to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a method for digital management of laboratories, including the following steps:

[0007] Step S1: Obtain the real-time panoramic monitoring video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic monitoring video of the laboratory to construct a panoramic structure diagram of the laboratory;

[0008] Step S2: Identify the status parameters of laboratory equipment based on multiple sensors; perform equipment position registration on the panoramic structure diagram of the laboratory according to the status parameters of laboratory equipment, and conduct trend mapping to construct a panoramic digital model of the laboratory;

[0009] Step S3: Optimize the time-sequence frame delay of the real-time panoramic monitoring video of the laboratory, and conduct dynamic optical flow tracking to generate the time-sequence movement trajectory and operation behavior trajectory sequence of each person;

[0010] Step S4: Conduct full-cycle dynamic behavior trajectory evolution based on the time-sequence movement trajectory and operation behavior trajectory sequence of each person, and perform real-time rendering of the behavior trajectory on the panoramic digital model of the laboratory, so as to generate a dynamic trajectory rendering twin model;

[0011] Step S5: Conduct in-depth behavior semantic analysis and behavior safety risk analysis on the dynamic trajectory rendering twin model to generate behavior safety warning signals;

[0012] Step S6: Make risk warning decisions according to the behavior safety warning signals, and perform real-time visualization processing in combination with the dynamic trajectory rendering twin model, so as to execute the full-cycle digital management operation of the laboratory.

[0013] The present invention provides a complete view of the laboratory environment through real-time panoramic monitoring videos, ensuring real-time monitoring and analysis of various objects, personnel, equipment, etc. inside the laboratory. Through multi-object visual recognition, various types of equipment, items, and personnel in the laboratory can be accurately recognized, avoiding omissions in manual monitoring. At the same time, regional layout analysis can help construct a detailed laboratory structure diagram, facilitating the management and scheduling of resources in each area of the laboratory. Multi-sensor technologies (such as temperature and humidity sensors, pressure sensors, cameras, etc.) accurately capture the real-time states of various equipment in the laboratory (such as temperature, operating status, fault information, etc.). Registering the equipment status information with the panoramic structure diagram accurately displays the current position and status of each equipment, providing strong support for the remote monitoring and management of the equipment. The trend mapping function helps discover the laws of equipment performance changes, and then make preventive maintenance decisions to avoid the impact of equipment failures on the laboratory. The constructed digital model is convenient for data analysis and optimization. Through the optimization of the time-series frame interval delay, the image blur caused by video transmission delay is reduced, thereby improving the accuracy of real-time monitoring. The dynamic optical flow tracking technology can accurately track the movement trajectories and operation behaviors of each person, not only helping to understand the activities of people in the laboratory, but also analyzing whether the operation behaviors comply with the safety specifications of the laboratory. It helps to better manage the activities of laboratory personnel and discover potential unsafe behaviors or operations. Through the dynamic trajectory evolution of the movement trajectories and operation behaviors of people, the behavior patterns and their changing trends of people in the laboratory can be comprehensively analyzed. This evolutionary analysis helps to identify the behavior habits of people, discover potential dangerous behaviors, and make real-time adjustments or reminders to them. Through the instant rendering of the behavior trajectories, the formed dynamic trajectory rendering twin model visualizes all the dynamic information of people, which provides intuitive real-time data support for managers and enables more accurate decision-making. Through the in-depth analysis of the people's behavior trajectories in the twin model, abnormal behaviors or potential safety hazards are identified, such as people entering dangerous areas, improper equipment operations, etc. Combining with behavioral safety risk analysis, potential risk sources are identified in advance and safety warning signals are generated, timely reminding laboratory managers to take emergency measures to reduce the occurrence of safety accidents. This is crucial for ensuring the safety of laboratory operations. Making risk warning decisions based on safety warning signals helps laboratory managers respond to potential safety problems in a timely manner. Combining with the dynamic trajectory rendering twin model, managers can see the safety status of all personnel and equipment in real time, realizing the full-process monitoring of the laboratory environment. This visual real-time processing system not only improves the management efficiency, but also enhances the safety of the laboratory, ensuring that all tasks are carried out in accordance with the established specifications.

[0014] In this specification, a laboratory digital management system is provided for implementing the laboratory digital management method as described above, including:

[0015] The object visual recognition module is used to obtain the real-time panoramic monitoring video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic monitoring video of the laboratory, and construct a panoramic structure diagram of the laboratory;

[0016] The trend mapping module is used to identify the state parameters of laboratory equipment based on multiple sensors; perform equipment position registration on the panoramic structure diagram of the laboratory according to the state parameters of laboratory equipment, and perform trend mapping to construct a panoramic digital model of the laboratory;

[0017] The dynamic optical flow tracking module is used to optimize the time-series frame delay of the real-time panoramic monitoring video of the laboratory, and perform dynamic optical flow tracking to generate the time-series movement trajectory and operation behavior trajectory sequence of each person;

[0018] The instant rendering module is used to perform full-cycle dynamic behavior trajectory evolution according to the time-series movement trajectory and operation behavior trajectory sequence of each person, and perform instant rendering of the behavior trajectory on the panoramic digital model of the laboratory, so as to generate a dynamic trajectory rendering twin model;

[0019] The behavior semantic analysis module is used to perform in-depth behavior semantic analysis and behavior safety risk analysis on the dynamic trajectory rendering twin model, and generate behavior safety warning signals;

[0020] The real-time visualization module is used to make risk warning decisions according to the behavior safety warning signals, and perform real-time visualization processing in combination with the dynamic trajectory rendering twin model, so as to execute the full-cycle digital management operation of the laboratory.

[0021] By obtaining the panoramic monitoring video of the laboratory in real time, the present invention can comprehensively monitor the dynamics of the laboratory and improve the monitoring ability of the laboratory. Through multi-object visual recognition and regional layout analysis, it can accurately identify different areas, equipment, personnel, etc. in the laboratory and construct a panoramic structure diagram of the laboratory. Using a variety of sensor data to monitor the status of laboratory equipment in real time (such as temperature, pressure, working status, etc.), it can accurately grasp the health status of the equipment. Registering the equipment status with the panoramic structure diagram of the laboratory improves the visualization of equipment management and the accuracy of spatial layout. The use of trend mapping can obtain the trend of equipment status changes in real time, providing data support for predictive maintenance and laboratory operation to prevent potential failures. The optimization of the time-series frame delay effectively reduces the delay in the video processing process and improves the response speed and smoothness of the real-time monitoring video. The dynamic optical flow tracking technology can track the real-time movement trajectory and operation behavior trajectory of each person in the laboratory. Through this technology, not only the position of personnel is dynamically tracked, but also their behaviors and operation processes are recorded, providing data support for subsequent behavior analysis and safety detection. The extraction of the personnel behavior trajectory sequence helps to better understand the activity patterns of personnel in the laboratory, thus making reasonable judgments on the safety and operation behaviors of the laboratory. Through the full-cycle dynamic behavior trajectory evolution of the time-series movement trajectory and operation behavior trajectory sequence of personnel, the long-term behavior patterns of personnel can be simulated and predicted, improving the management predictability of the laboratory. Rendering the behavior trajectory immediately into the panoramic digital model of the laboratory to form a dynamic trajectory rendering twin model. This twin model can display the real-time changes in personnel activities in a visual way, helping managers to immediately understand the dynamics in the laboratory. It provides an immersive monitoring method for managers, enabling them to quickly identify potential problems in a visual environment. Deep behavior semantic parsing extracts meaningful safety hazards from complex behavior trajectories to help identify potential safety problems. Through the safety risk analysis of behavior trajectories, dangerous behavior patterns and abnormal behaviors are accurately identified, generating safety warning signals to improve the safety of the laboratory. The probability of laboratory accidents is effectively reduced, and intervention and correction are carried out in a timely manner to ensure the safety of personnel and equipment. According to the behavior safety warning signal, risk warning decisions are made, and corresponding measures can be taken before safety hazards occur to avoid potential hazards. The combination of the dynamic trajectory rendering twin model and real-time visualization processing can, when a safety risk occurs, display the safety status in the laboratory in real time, helping laboratory managers to quickly respond and make decisions. It realizes the full-cycle digital management operation, making the management of the laboratory more intelligent and automated, thereby reducing manual intervention and improving work efficiency and management quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the step flow of a laboratory digital management method of the present invention;

[0023] Figure 2 It is a schematic diagram of the detailed implementation steps of step S1;

[0024] Figure 3 It is a schematic diagram of the detailed implementation steps of step S2;

[0025] Figure 4 It is a schematic diagram of the detailed implementation steps of step S3. Specific implementation manners

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] The embodiments of the present application provide a laboratory digital management method and system. The execution subjects of the laboratory digital management method and system include, but are not limited to, the following that are equipped with this system: mechanical equipment, data processing platforms, cloud server nodes, network uploading devices, etc., which can be regarded as general computing nodes of the present application. The data processing platform includes, but is not limited to, at least one of an audio and image management system, an information management system, and a cloud data management system.

[0028] Please refer to Figures 1 to 4 , the present invention provides a laboratory digital management method, and the laboratory digital management method includes the following steps:

[0029] Step S1: Obtain the real-time panoramic monitoring video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic monitoring video of the laboratory, and construct a laboratory panoramic structure diagram;

[0030] Step S2: Identify the laboratory equipment status parameters based on multiple sensors; perform equipment position registration on the laboratory panoramic structure diagram according to the laboratory equipment status parameters, and perform trend mapping to construct a laboratory panoramic digital model;

[0031] Step S3: Optimize the time-sequence frame-to-frame delay of the real-time panoramic monitoring video of the laboratory, and perform dynamic optical flow tracking to generate the time-sequence movement trajectory and operation behavior trajectory sequence of each person;

[0032] Step S4: Perform full-cycle dynamic behavior trajectory evolution according to the time-sequence movement trajectory and operation behavior trajectory sequence of each person, and perform behavior trajectory instant rendering on the laboratory panoramic digital model, so as to generate a dynamic trajectory rendering twin model;

[0033] Step S5: Perform in-depth behavior semantic parsing and behavior safety risk analysis on the dynamic trajectory rendering twin model to generate a behavior safety warning signal;

[0034] Step S6: Make a risk warning decision based on the behavioral safety warning signal, and perform real-time visualization processing in combination with the dynamic trajectory rendering twin model, so as to execute the full-cycle digital management operation of the laboratory.

[0035] The present invention provides a complete view of the laboratory environment through real-time panoramic monitoring video, ensuring real-time monitoring and analysis of various objects, personnel, equipment, etc. inside the laboratory. Through multi-object visual recognition, various types of equipment, items, and personnel in the laboratory can be accurately recognized, avoiding omissions in manual monitoring. At the same time, the regional layout analysis can help construct a detailed laboratory structure diagram, facilitating the management and scheduling of resources in each area of the laboratory. Multi-sensor technologies (such as temperature and humidity sensors, pressure sensors, cameras, etc.) accurately capture the real-time states of various equipment in the laboratory (such as temperature, operating status, fault information, etc.). Registering the equipment status information with the panoramic structure diagram accurately displays the current position and status of each equipment, providing strong support for the remote monitoring and management of the equipment. The trend mapping function helps discover the laws of equipment performance changes, and then make preventive maintenance decisions to avoid the impact of equipment failures on the laboratory. The constructed digital model is convenient for data analysis and optimization. Through the optimization of the time series frame interval delay, the image blur caused by video transmission delay is reduced, thereby improving the accuracy of real-time monitoring. The dynamic optical flow tracking technology can accurately track the movement trajectories and operation behaviors of each person, not only helping to understand the activities of personnel in the laboratory, but also analyzing whether the operation behaviors comply with the safety specifications of the laboratory. It helps to better manage the activities of laboratory personnel and discover potential unsafe behaviors or operations. By performing dynamic trajectory evolution on the movement trajectories and operation behaviors of personnel, the behavioral patterns and their changing trends of personnel in the laboratory can be comprehensively analyzed. This evolutionary analysis helps to identify the behavioral habits of personnel, discover potential dangerous behaviors, and make real-time adjustments or reminders to them. Through the instant rendering of the behavioral trajectory, the formed dynamic trajectory rendering twin model visualizes all the dynamic information of personnel, which provides intuitive real-time data support for managers and enables more accurate decision-making. By deeply analyzing the personnel behavior trajectories in the twin model, abnormal behaviors or potential safety hazards are identified, such as personnel entering dangerous areas, improper equipment operation, etc. Combining behavioral safety risk analysis, potential risk sources are identified in advance and safety warning signals are generated, timely reminding laboratory managers to take emergency measures and reducing the occurrence of safety accidents. This is crucial for ensuring the safety of laboratory operations. Making a risk warning decision based on the safety warning signal helps laboratory managers respond to potential safety problems in a timely manner. Combining with the dynamic trajectory rendering twin model, managers can see the safety status of all personnel and equipment in real time, realizing the full-process monitoring of the laboratory environment. This visual real-time processing system not only improves the management efficiency, but also enhances the safety of the laboratory, ensuring that all tasks are carried out in accordance with the established specifications.

[0036] In an embodiment of the present invention, refer to Figure 1 , which is a schematic diagram of the step flow of a laboratory digital management method of the present invention. In this example, the steps of the laboratory digital management method include:

[0037] Step S1: Obtain the real-time panoramic monitoring video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic monitoring video of the laboratory, and construct a laboratory panoramic structure diagram;

[0038] In this embodiment, first select appropriate monitoring devices, such as high-resolution cameras (e.g., 1920x1080 resolution), which can support real-time video recording at 30 FPS. These cameras should have a wide-angle field of view to cover the main areas of the laboratory. Install cameras at key positions in the laboratory (such as entrances, equipment areas, and around workbenches) to ensure there are no blind spots. The installation height of the cameras should be between 2.5 - 3 meters to obtain the best monitoring effect. Use network cameras or IP cameras to send the video stream to the central processing unit via Wi-Fi or a wired network. Use a protocol such as RTSP (Real-Time Streaming Protocol) for video transmission to ensure low latency and high-quality video. Configure video capture software (such as OpenCV or FFmpeg) to receive the real-time video stream.

[0039] import cv2

[0040] # Connect to the monitoring camera

[0041] cap = cv2.VideoCapture('rtsp: / / username:password@ip_address:port')

[0042] Implement real-time processing of the video stream. Read video frames in a loop and save each frame to memory for subsequent processing. Set the reading frequency to 30 frames per second to ensure a smooth video is captured. Store the obtained video stream in real-time on the local hard drive or cloud storage, fourcc = cv2.VideoWriter_fourcc('XVID')

[0043] out = cv2.VideoWriter('output.avi', fourcc, 30.0, (640, 480))

[0044] while cap.isOpened():

[0045] ret, frame = cap.read()

[0046] if ret:

[0047] out.write(frame)

[0048] cv2.imshow('Video Stream', frame)

[0049] if cv2.waitKey(1) & 0xFF == ord('q'):

[0050] break

[0051] Select a deep learning-based object detection model, such as YOLOv5 or Faster R-CNN. These models can identify multiple objects in the video in real time, such as personnel, equipment, and other important elements. Train the model using a labeled dataset (such as COCO or a custom laboratory dataset) to ensure that it can effectively identify specific object types in the laboratory. Set the training parameters, for example, the learning rate is 0.001, the batch size is 16, and the number of training epochs is 50. Extract each frame from the acquired real-time video stream and pass it into the trained object detection model for object recognition.

[0052] import torch

[0053] model = torch.hub.load('ultralytics / yolov5', 'yolov5s', pretrained = True)

[0054] while cap.isOpened():

[0055] ret, frame = cap.read()

[0056] if ret:

[0057] results = model(frame)

[0058] results.render() # Render the recognition results on the frame

[0059] cv2.imshow('Detected Objects', frame)

[0060] Analyze the regional layout of the laboratory based on the recognized object information. Using a spatial coordinate system, map the position information of each object to the panoramic structure diagram of the laboratory. Utilize the recognized object and area information to construct the panoramic structure diagram of the laboratory using a graphics drawing library (such as Matplotlib or Graphviz). Each object and area should be clearly labeled in the diagram. Save the generated panoramic structure diagram in a picture format (such as PNG or SVG) and display it in the laboratory management system for subsequent management and analysis.

[0061] Step S2: Based on multi-sensor recognition of the laboratory equipment status parameters; perform equipment position registration on the panoramic structure diagram of the laboratory according to the laboratory equipment status parameters, and conduct trend mapping to construct a panoramic digital model of the laboratory;

[0062] In this embodiment, suitable sensors are selected to monitor the state parameters of laboratory equipment, including temperature sensors, humidity sensors, pressure sensors, current sensors, etc. Ensure that the accuracy and response time of the sensors meet the laboratory requirements. For example, select a temperature sensor with a measurement range of -40°C to 125°C and an accuracy of ±0.5°C. Install sensors on each key device to ensure real-time monitoring of the device status. For example, deploy corresponding sensors on refrigeration equipment, reactors, and other key instruments in the laboratory. The sensors should be connected to the data acquisition system by wired or wireless means. Use a microcontroller (such as Arduino or Raspberry Pi) to build a data acquisition system. Set the data acquisition frequency to once per second to ensure that changes in the device status can be captured. Send the collected status parameters to the central database via Wi-Fi or a wired network. Use the MQTT protocol or HTTP RESTful API for data transmission to ensure the reliability and real-time nature of the data. Write scripts in Python to regularly read the device status parameters from the database, perform data cleaning and processing to ensure the integrity and accuracy of the data. Use the pandas library for data analysis and processing. Integrate the device status parameters with the device location information in the laboratory panoramic structure diagram. Use GPS coordinates or a relative coordinate system to ensure that the status parameters of each device correspond one-to-one with its actual location. Use a graphics processing tool (such as OpenCV or Matplotlib) to mark the location of each device in the laboratory panoramic structure diagram. Use coordinate transformation methods to map the real-time status parameters of the device onto the panoramic diagram. Conduct trend analysis on the collected device status parameters, and use time series analysis methods to calculate the status change trend of each device. The seasonal_decompose() function in the statsmodels library can be used for trend decomposition. Visualize the analyzed trend data, and use Matplotlib to generate a trend chart. For example, plot a line chart showing the changes in the temperature, pressure, and humidity of the device over time to facilitate observing the operating status of the device. Combine the trend mapping with the device location information to build a panoramic digital model of the laboratory. Use 3D modeling software (such as Blender or SketchUp) to integrate the device status and location information into the digital model to form an interactive laboratory digital model. Export the built panoramic digital model to a visualization format (such as STL or OBJ) for display in the laboratory management system. Ensure that the model can be updated in real time to reflect the device status changes and location registration information.

[0063] Step S3: Optimize the temporal inter-frame delay of the real-time panoramic monitoring video of the laboratory, and perform dynamic optical flow tracking to generate a temporal movement trajectory and an operation behavior trajectory sequence for each person;

[0064] In this embodiment, the inter-frame delay data of the real-time panoramic monitoring video in the laboratory is collected. OpenCV is used to read the video frames, and the timestamp of each frame is recorded to calculate the delay between each frame. A threshold (e.g., 100 milliseconds) is set to identify frames with excessive delay. An interpolation algorithm (such as linear interpolation or spline interpolation) is used to optimize the frames with larger delays. By interpolating the pixel values of adjacent frames, intermediate frames with smooth transitions are generated to reduce the impact of the delay. The OpenCV library is used to read the real-time video stream, and the reading frequency is set to 30 frames per second to ensure that smooth video content can be captured.

[0065] import cv2

[0066] cap = cv2.VideoCapture('video_source')

[0067] fps = cap.get(cv2.CAP_PROP_FPS)

[0068] In the loop, each frame is read, its timestamp is recorded, and the delay between adjacent frames is calculated.

[0069] import time

[0070] frame_times = []

[0071] while cap.isOpened():

[0072] ret, frame = cap.read()

[0073] if ret:

[0074] current_time = time.time()

[0075] frame_times.append(current_time)

[0076] # Calculate the delay

[0077] if len(frame_times) > 1:

[0078] delay = frame_times[-1] - frame_times[-2]

[0079] if delay > 0.1: # Exceed 100 milliseconds

[0080] # Perform interpolation optimization

[0081] For frames with excessive delays, use interpolation algorithms to generate intermediate frames. Use the cv2.addWeighted() function in OpenCV for simple linear interpolation.

[0082] if delay > 0.1:

[0083] # Generate intermediate frames

[0084] mid_frame = cv2.addWeighted(previous_frame, 0.5, current_frame, 0.5, 0)

[0085] # Add mid_frame to the video stream

[0086] Select the Lucas-Kanade optical flow algorithm, which is suitable for small-scale motion tracking and can calculate the motion trajectories of multiple points in real time. Use the cv2.goodFeaturesToTrack() function in OpenCV to detect feature points in each frame for subsequent optical flow tracking. Set parameters such as a minimum distance of 10 pixels and 100 feature points.

[0087] feature_params = dict(maxCorners = 100, qualityLevel = 0.3, minDistance = 10, blockSize = 7)

[0088] p0 = cv2.goodFeaturesToTrack(previous_frame_gray, mask = None, feature_params)

[0089] In each frame, calculate the optical flow of the feature points through the cv2.calcOpticalFlowPyrLK() function to track the motion of the feature points in consecutive frames. Record the motion trajectories of each feature point and convert them into actual time-series movement trajectories. Use a list or dictionary to store the position information of each feature point.

[0090]

[0091] Combine the movement trajectories with the device status information and use machine learning models (such as decision trees or support vector machines) to identify operation behaviors. For example, set specific motion patterns (such as "approach the device" or "move away from the device") for marking. Store the generated time-series movement trajectories and operation behavior trajectory sequences of each person in the database, ensuring consistent data formats for subsequent analysis and visualization.

[0092] Step S4: Perform full-cycle dynamic behavior trajectory evolution based on the time-series movement trajectories and operation behavior trajectory sequences of each person, and perform real-time rendering of the behavior trajectories on the panoramic digital model of the laboratory, so as to generate a dynamic trajectory rendering twin model;

[0093] In this embodiment, the Markov Chain or Hidden Markov Model (HMM) is selected as the basis for dynamic behavior trajectory evolution. These models can learn the state transition probabilities based on historical behavior data, so as to predict future behavior trajectories. Obtain the time-series movement trajectories and operation behavior trajectory sequences of each person from the previous steps. Integrate this data to ensure that each trajectory contains a timestamp, spatial coordinates, and behavior type. Use the pandas library to store the data in DataFrame format for subsequent processing. Extract the key features of each trajectory, such as movement speed, stay time, and direction change. According to the integrated trajectory data, construct a state transition matrix. Each state represents a behavior or location, and the transition probabilities between states are calculated by counting frequencies. For example, if the number of times state A transfers to state B is 5 times and the number of times it transfers to state C is 3 times, then the state transition probability is: P(A→B) = 5 / (5 + 3) = 0.625. Use the Markov model to perform trajectory evolution simulation and generate the next state based on the current state. Set the number of simulation steps (such as 50 time steps) to generate future trajectories. Prepare the panoramic digital model of the laboratory to ensure that the model contains the accurate locations of each device and area. Use 3D modeling software (such as Blender or Unity) to build the model and export it in a visualizable format (such as FBX or OBJ). Load the panoramic digital model in a 3D engine (such as Unity) and perform real-time rendering according to the evolved trajectory data. Use scripts to control the drawing of the trajectories, and set the trajectory color and width to facilitate the distinction of the behavior trajectories of different people. Set the system to update the trajectory rendering regularly to ensure that as time goes by, the dynamic behavior trajectories of people are accurately reflected in the digital model. For example, update the trajectory every 1 second. Store the evolved trajectory data and rendering information in a database for subsequent analysis and management. Ensure that the data formats are consistent for quick query and visualization.

[0094] Step S5: Perform in-depth behavior semantic parsing and behavior safety risk analysis on the dynamic trajectory rendering twin model to generate behavior safety warning signals;

[0095] In this embodiment, a behavior recognition model based on deep learning, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), is selected for behavior semantic parsing. These models can extract different behavior features from dynamic trajectories and classify them. The behavior trajectory data of each person is extracted from the dynamic trajectory rendering twin model, including timestamps, spatial coordinates, and behavior types. These data are stored in a structured format, such as CSV or a database, for subsequent processing. The trajectory data is labeled according to the laboratory safety operating procedures. Behavior categories are set, such as "normal operation", "abnormal approach to equipment", "long stay", etc., to ensure that each trajectory sample has a corresponding label. A suitable deep learning framework (such as TensorFlow or PyTorch) is selected for model construction. The model is trained using the labeled trajectory dataset, and training parameters are set, such as a learning rate of 0.001, a batch size of 32, and 100 epochs for training. During the model training process, CNN is used to extract features from the trajectory data, and the features are sent to a fully connected layer for classification. The performance of the model is evaluated on the validation set, and accuracy and F1 score are used as metrics to ensure that the model can accurately identify different behavior types. A risk scoring model is adopted, combined with logistic regression or random forest algorithms, to conduct a safety risk assessment on the parsed behaviors. Multiple risk factors are set according to the laboratory safety specifications, such as "time of approaching high-risk equipment", "long stay in sensitive areas", etc., and corresponding weights are assigned to each factor. The parsed behaviors are combined with the preset risk factors to calculate the comprehensive risk score. An early warning threshold is set according to the calculated risk score. For example, when the risk score exceeds 0.7, it is marked as "high risk" and an early warning signal is generated. The generated early warning signal and related information (such as person ID, behavior type, risk score, etc.) are sent to the laboratory monitoring system through a message queue or API interface for real-time monitoring and response. Each generated early warning signal is recorded in the database for subsequent analysis and improvement of safety management measures. A regular review mechanism is set to evaluate the accuracy and effectiveness of the early warning signals.

[0096] Step S6: Make a risk warning decision based on the behavior safety warning signal, and perform real-time visualization processing in combination with the dynamic trajectory rendering twin model, so as to execute the full-cycle digital management operation of the laboratory.

[0097] In this embodiment, a rule-based risk warning decision framework is established. According to the safety standards of the laboratory, different risk levels (such as "low risk", "medium risk", "high risk") are set, and corresponding response measures are defined. For example, when the risk score is greater than 0.7, the high-risk response mechanism is triggered. An interface for the system to receive warning signals is set to ensure that warning information from the behavioral safety analysis module can be received in real time. A message queue (such as RabbitMQ or Kafka) is used to implement asynchronous processing of warning signals, improving the response speed and stability of the system. After receiving a warning signal, the risk level is first evaluated. According to the set threshold and risk score, the corresponding response measures are determined.

[0098]

[0099]

[0100] According to the evaluation results, the corresponding response measures are executed. In the case of high risk, the system automatically sends an alarm to notify the laboratory management personnel and activates the emergency plan. A suitable visualization tool or platform (such as Unity, Blender, or WebGL) is selected for dynamic trajectory rendering to ensure that the movement trajectories and behavioral information of personnel can be displayed in real time. The behavioral trajectories and risk assessment results are transmitted to the visualization platform through real-time data streams (such as WebSocket). The data transmission frequency is set to 1 second to ensure that the visualization content can be updated in a timely manner. In the visualization platform, the trajectories and behavioral information of each person are dynamically rendered. Different colors and icons are used to represent different risk levels. For example, the trajectories of high-risk behaviors are displayed in red, and low-risk behaviors are displayed in green.

[0101]

[0102] The risk warning decision-making and dynamic trajectory visualization are combined to form a complete laboratory management system. Ensure that the management personnel can view the real-time status, risk assessment, and personnel behavior trajectories through a unified interface. Design a user-friendly interface to enable the management personnel to easily access the real-time data and historical records of the laboratory. The interface should include risk assessment results, dynamic trajectory visualization, and related operation buttons (such as "activate the emergency plan", "view detailed records", etc.). A decision feedback mechanism is established to allow the management personnel to adjust laboratory operations in a timely manner according to the visualization results and warning signals. Regularly review the accuracy and effectiveness of risk warnings to improve the decision-making process.

[0103] In this embodiment, refer to Figure 2 , which is a schematic diagram of the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0104] Step S11: Obtain the real-time panoramic monitoring video of the laboratory;

[0105] Step S12: Perform multi-object visual recognition on the real-time panoramic monitoring video of the laboratory, and extract all laboratory object nodes;

[0106] Step S13: Perform three-dimensional space positioning on each laboratory object node one by one to obtain the three-dimensional position coordinates of each object;

[0107] Step S14: Mine the topological associations between objects for all laboratory object nodes to generate object topological structure data;

[0108] Step S15: Perform regional layout analysis on the real-time panoramic monitoring video of the laboratory according to the object topological structure data to generate regional layout structure features;

[0109] Step S16: Perform panoramic structure reconstruction on the regional layout structure features according to the three-dimensional position coordinates of each object to construct a panoramic structure diagram of the laboratory.

[0110] In this embodiment, a panoramic camera with high resolution (such as a 360-degree camera) is selected for laboratory monitoring to ensure that every corner of the laboratory can be covered. Devices such as the Insta360 One X2 or GoPro Max are recommended, as these devices can provide clear panoramic videos. The panoramic camera is installed at an appropriate position in the laboratory to ensure that the camera's field of view can cover all important areas. The height is set to 2.5 meters to obtain the best viewing angle. The camera is connected to the monitoring system via Wi-Fi or a wired network to ensure stable real-time data transmission. The monitoring system is configured to obtain a real-time video stream, and the Real-Time Streaming Protocol (RTSP) is used for video stream transmission to ensure low latency and high image quality. The video resolution is set to 1920x1080 (1080p), and the frame rate is set to 30 FPS to ensure video smoothness and details. A multi-object detection model based on deep learning, such as YOLOv5 or Faster R-CNN, is selected. These models can detect multiple objects in the video in real time. A publicly available dataset containing laboratory objects (such as COCO or a custom dataset) is used for model training to ensure that the detection model can identify specific object categories in the laboratory, such as experimental equipment, test tubes, books, etc. The cv2.VideoCapture() function of OpenCV is used to extract the real-time video stream frame by frame for object detection. Each frame of the image is input into the trained object detection model to obtain the category and position (bounding box) of each object. The model output includes object category, confidence, and position information. The bounding boxes and labels of the recognized objects are drawn on each frame of the image, and the cv2.rectangle() and cv2.putText() functions of OpenCV are used for marking for subsequent analysis. Stereo vision or monocular depth estimation methods are used for three-dimensional positioning, and the cv2.reprojectImageTo3D() function of OpenCV is used for point cloud generation. Ensure that the internal and external parameters of the camera are known, and the focal length is set to 800 pixels, and the principal point position is at the center of the image for accurate three-dimensional reconstruction. For each detected object, its pixel coordinates in the video frame are obtained. The three-dimensional coordinates of the object are calculated using the depth information (obtained through a stereo camera or depth sensor). Using the perspective projection formula, the pixel coordinates are mapped to three-dimensional space to calculate the three-dimensional position coordinates (X, Y, Z) of each object, ensuring a unified coordinate system. Define the topological relationships between objects, considering the relative positions, contact relationships, and functional relationships of the objects. Nodes and edges in graph theory are used to represent the objects and their relationships. Set a threshold, such as a distance between objects less than 0.5 meters is considered an adjacency relationship, in order to establish the topological structure between objects. A graph data structure is used to store the object nodes and their topological relationships. Each object is a node in the graph, and the relationships between objects are stored as edges.Traverse all object nodes, identify and record the topological relationships between each pair of objects, and generate topological structure data, including object IDs, position coordinates, and connection relationships. Use clustering algorithms (such as K-Means or DBSCAN) to partition the objects into regions and identify the spatial distribution characteristics of the objects. Set clustering parameters, such as setting the number of clusters in K-Means to 5 and the minimum number of samples in DBSCAN to 3, for effective region partitioning. Analyze the three-dimensional coordinates of the objects using the clustering algorithm to identify the characteristics of different regions and determine the center points and boundaries of each region. Generate layout structure characteristics for each region, including the number of objects within the region, spatial position distribution, and functional characteristics, providing a basis for subsequent layout optimization. Adopt point cloud reconstruction and meshing techniques to fuse the three-dimensional position coordinates of the objects and the layout structure characteristics to construct a panoramic structure diagram of the laboratory. Set the voxel size of the point cloud to 0.01 meters to ensure the fineness of the reconstruction. Generate point cloud data based on the three-dimensional position coordinates of the objects and perform point cloud processing using the PointCloud method of the Open3D library. Apply the Poisson SurfaceReconstruction method to the generated point cloud to generate a three-dimensional mesh model, thereby constructing a panoramic structure diagram of the laboratory. Export the final panoramic structure diagram in a 3D file format (such as OBJ or FBX) and display it using visualization software to ensure the integrity and readability of the model.

[0111] In this embodiment, refer to Figure 3 , which is a schematic diagram of the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0112] Step S21: Identify the state parameters of laboratory equipment based on multiple sensors;

[0113] Step S22: Perform time-series operation analysis on the state parameters of laboratory equipment at multiple time points to extract the equipment operation characteristics at multiple time points;

[0114] Step S23: Perform adjacent-time differential detection on the equipment operation characteristics at multiple time points to generate the operation difference change data for each equipment;

[0115] Step S24: Perform real-time trend fitting on the operation difference change data for each equipment to construct a real-time state trend graph for each equipment;

[0116] Step S25: Perform equipment position registration on the panoramic structure diagram of the laboratory according to the real-time state trend graph of each equipment and perform trend mapping to construct a panoramic digital model of the laboratory.

[0117] In this embodiment, multiple sensors are selected to obtain the status parameters of laboratory equipment, including temperature sensors, humidity sensors, pressure sensors, and current sensors. It is recommended to use sensors such as DHT22 (temperature and humidity sensor), BMP180 (barometric pressure sensor), and ACS712 (current sensor), which can provide accurate real-time data. The sensors are installed near various equipment in the laboratory to ensure that each piece of equipment can be monitored by the corresponding sensor. The sensors should be fixed at a reasonable position about 1 meter away from the equipment to avoid interference. Use a control center for data acquisition such as Arduino or Raspberry Pi, connect all sensors, and achieve timed data acquisition through programming. Set the acquisition frequency to sample once per minute to ensure sufficient time series data is obtained. The acquired equipment status parameters are stored in a local database (such as SQLite) or a cloud database in real time for subsequent analysis. Ensure the stability of data storage and transmission. Clean the collected status parameters to remove outliers and missing values. Use the Pandas library in Python for data processing, set a threshold, and remove data points that exceed ±3 standard deviations. Add timestamps to each data point for time series analysis. Ensure that the timestamp format is unified (such as ISO8601 format) for subsequent analysis. Conduct time series analysis on the status parameters of each device, extract key features such as mean, standard deviation, maximum value, and minimum value. Use the NumPy library in Python for calculations to obtain the operating characteristics of the device at each time point. Store the extracted operating characteristics in a new data table for convenient subsequent analysis and comparison. Each table should include the device ID, timestamp, and the extracted feature values. Select the sliding window method for detecting differences between adjacent times. Set the window size to 2, that is, compare the changes in feature values between two adjacent time points. Set a difference threshold (such as 10%) to judge significant changes in the device operating status. If the change in feature values between adjacent time points exceeds this threshold, it is recorded as a change. Conduct adjacent time point difference detection on the extracted features of each device, and calculate the difference ratio between feature values. Use the Pandas library in Python for data processing to generate a difference change data table. Record the detected operating difference change data of each device in a new data table, including device ID, timestamp, feature value change, and change type (increase or decrease). Select a linear regression or polynomial regression model for real-time trend fitting. For simple trends, linear regression is sufficient, and for more complex trends, polynomial regression can be used. Set the polynomial order of the fitting (such as 2) to capture the non-linear changes in the device operating status. Use the Scikit-learn library in Python to fit the operating difference change data of each device, calculate the fitting function and its parameters. Draw a real-time status trend graph for each device according to the fitting results, and use the Matplotlib library for visualization to show the change trend of the device status over time.Adopt a feature-based registration method, and perform position registration by combining the three-dimensional position coordinates of the device and the real-time status trend graph. Use algorithms such as ICP (Iterative Closest Point) to achieve high-precision registration. Set the convergence conditions for registration, such as the maximum number of iterations being 50 times and the registration error threshold being 0.01 meters. Match the three-dimensional position coordinates of the device with the panoramic structure diagram, and use the ICP algorithm to optimize the device position to ensure the accuracy of the device position in the panoramic structure diagram. According to the real-time status trend graph of the device, map the trend information to the panoramic structure diagram, and use colors or annotations to represent the status trends of different devices for easy visualization. Export the registered panoramic digital model as a 3D file format (such as FBX or GLTF) for display and analysis on the visualization platform. Use visualization software (such as Blender or Unity) to load and display the panoramic digital model to ensure good interactivity and visualization effects of the model.

[0118] In this embodiment, refer to Figure 4 , which is a schematic diagram of the detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0119] Step S31: Optimize the temporal inter-frame delay of the real-time panoramic monitoring video of the laboratory to obtain an inter-frame smoothed optimized video;

[0120] Step S32: Perform frame-by-frame human visual recognition on the inter-frame smoothed optimized video to mark the personnel in the laboratory;

[0121] Step S33: Perform dynamic optical flow tracking on the personnel in the laboratory to obtain the dynamic optical flow trajectories of the personnel;

[0122] Step S34: Perform multi-period position movement calculations on the dynamic optical flow trajectories of the personnel to extract the temporal movement trajectories of each person;

[0123] Step S35: Mine the device operation behaviors from the dynamic optical flow trajectories of the personnel to extract the device operation behavior trajectories;

[0124] Step S36: Perform temporal logic fitting on the device operation behavior trajectories to generate an operation behavior trajectory sequence.

[0125] In this embodiment, the optical flow method is selected for inter-frame smoothing optimization. The optical flow method can compensate for jitter or blur caused by inter-frame delay by analyzing the pixel motion in the image sequence. The window size of the optical flow method is set to 15x15 to adapt to the dynamic scene in the laboratory and ensure the smoothness of motion. The Lucas-Kanade method is used for optical flow calculation. The cv2.VideoCapture() function of OpenCV is used to extract the laboratory surveillance video frame by frame, and each frame image is stored as an array. The optical flow is calculated for adjacent frames, and the cv2.calcOpticalFlowFarneback() function is used to obtain the motion vectors. By analyzing the optical flow field, smooth intermediate frames are interpolated. The interpolated frames are combined with the original frames to form a new smoothed and optimized video. The cv2.VideoWriter() function is used to save the optimized video, with the output resolution set to 1920x1080 and the frame rate set to 30 FPS. Deep learning models such as YOLOv5 or RetinaNet are used for person detection, and these models can efficiently identify and label people in real-time videos. Public datasets with person annotations (such as COCO or custom datasets) are used to ensure that the model can identify people in the laboratory. The smoothed and optimized video is read frame by frame, and each frame image is extracted using cv2.VideoCapture(). Each frame image is input into the trained person detection model to obtain the bounding boxes and their confidence levels for each detected person. The cv2.rectangle() function is used to draw the bounding boxes and label the person IDs. The marked results are saved as a new video file and output using cv2.VideoWriter() to ensure that the marks are clearly visible. The Kalman filter is combined with the optical flow method for dynamic tracking. The Kalman filter can effectively handle noise and uncertainty in dynamic systems and achieve smooth trajectory estimation. The state transition matrix and observation matrix of the Kalman filter are set to adapt to the dynamic characteristics of person movement. The state transition matrix is set to 2x2 to track the person's position. For each detected person, the optical flow method is used to calculate their motion vectors in consecutive frames to obtain the dynamic trajectory. The motion vectors are input into the Kalman filter to update the state estimate and generate a smooth dynamic optical flow trajectory. The position information at each time point is recorded. The cv2.line() function is used to draw the dynamic optical flow trajectory of the person and saved as a new video file to show the movement path of the person. The time-stamp-based movement calculation is selected, and the displacement of the person in each time period is calculated by analyzing the time information in the optical flow trajectory. A time window (such as 5 seconds) is set to extract the displacement data in each time period, and the moving distance and direction are calculated. The dynamic optical flow trajectories of each time period are traversed, and the total displacement and direction change in each time period are calculated. The NumPy library of Python is used for vector operations to record the movement trajectory.Record the time-series movement trajectories of each person in a data table, including information such as person ID, timestamp, displacement, direction, etc. Adopt behavior recognition algorithms, such as HMM (Hidden Markov Model) or LSTM (Long Short-Term Memory Network), to mine device operation behaviors. These algorithms are suitable for processing time-series data and identifying behavior patterns. Set behavior categories (such as "approaching the device", "operating the device", "leaving the device"), and the time window used during model training is 10 seconds. Input the extracted dynamic optical flow trajectories into the behavior recognition model to identify the device operation behaviors of each person. Record the start and end times of each behavior. Save the recognized device operation behavior trajectories into a data table, including information such as person ID, behavior type, timestamp, etc., for subsequent analysis. Select a time-series logistic regression model or a Markov chain to fit the operation behavior trajectories. Establish a behavior transition probability model by analyzing the behavior sequence. Set the state space and observation space, record the transition probabilities of each behavior, and use maximum likelihood estimation for parameter optimization. According to the extracted device operation behavior trajectories, calculate the transition probabilities between each behavior and construct a time-series logic model. Generate the operation behavior trajectory sequences of each device, record the duration and transition probabilities of each behavior, for facilitating the analysis of device usage habits.

[0126] In this embodiment, the specific steps of step S31 are as follows:

[0127] Calculate the global average frame rate of the real-time panoramic monitoring video of the laboratory to obtain the video average frame rate;

[0128] Perform video frame rate fluctuation recognition based on the video average frame rate to generate video frame rate fluctuation features;

[0129] Use the video frame rate fluctuation features to perform frame-by-frame delay analysis on the real-time panoramic monitoring video of the laboratory, so as to obtain the inter-frame delay data;

[0130] Perform dynamic adaptive smoothing on the inter-frame delay data to construct an adaptive smoothing curve;

[0131] Perform delay interpolation reconstruction based on the adaptive smoothing curve to obtain an inter-frame smoothed and optimized video.

[0132] In this embodiment, the OpenCV library is used to read the video file, and the average frame rate of the video is obtained by calculating the ratio of the total number of video frames to the total duration of the video. The specific steps are as follows: Open the video through the cv2.VideoCapture() function, use cv2.get(cv2.CAP_PROP_FRAME_COUNT) to obtain the number of frames, and then use cv2.get(cv2.CAP_PROP_FPS) to obtain the frame rate. Use cv2.VideoCapture() of OpenCV to open the real-time panoramic monitoring video in the laboratory. Calculate the total number of frames: total_frames = video.get(cv2.CAP_PROP_FRAME_COUNT), to obtain the total number of video frames. Calculate the video duration: duration = total_frames / video.get(cv2.CAP_PROP_FPS), to calculate the total duration of the video (in seconds). Calculate the average frame rate: average_fps = total_frames / duration, to obtain the average frame rate of the video. Calculate the timestamp of each frame and compare it with the average frame rate to identify frame rate fluctuations. Use simple absolute difference or standard deviation methods to quantify the fluctuation characteristics. Traverse the video frames, record the timestamp of each frame, and calculate the actual time interval between each frame. Calculate the fluctuation value of each frame through abs(current_frame_time - average_fps) and record it. Use the NumPy library to calculate the difference of each frame relative to the average frame rate. Store the fluctuation characteristics of each frame in an array for subsequent analysis. Calculate the delay between adjacent frames through the difference in timestamps of adjacent frames, using the current_frame_time - previous_frame_time method. Read each frame of the video and record the timestamp of each frame. Calculate the delay between adjacent frames: frame_delay[i] = timestamps[i] - timestamps[i - 1], and store it in an array for subsequent analysis. Store the calculated delay data between adjacent frames in a CSV file or database for further analysis. Select the Savitzky-Golay filter or moving average method for adaptive smoothing to reduce the impact of noise on the delay data. Use the scipy.signal.savgol_filter() function of the SciPy library to perform Savitzky-Golay smoothing on the delay data. Set the window size to 5 and the polynomial order to 2. The generated smoothed array serves as an adaptive smoothing curve for subsequent interpolation reconstruction. Select linear interpolation or spline interpolation for reconstruction to smooth and optimize the inter-frame delay of the video.Use the np.interp() function of NumPy or scipy.interpolate.interp1d() of SciPy for interpolation calculation, and apply the adaptive smoothing curve to the original delay data to generate smooth new delay data. Use cv2.VideoWriter() of OpenCV to apply the interpolated and reconstructed delay data to the video frames, and output a smoothed and optimized video, ensuring that the video format is MP4, the resolution is 1920x1080, and the frame rate is 30FPS.

[0133] In this embodiment, the specific steps of step S4 are as follows:

[0134] Step S41: Extract the movement trajectory timestamps and behavior trajectory timestamps according to the time-series movement trajectories and operation behavior trajectory sequences of each person;

[0135] Step S42: Perform behavior space position registration on the time-series movement trajectories and operation behavior trajectory sequences of each person to mark the position configuration data of each segment of the trajectory;

[0136] Step S43: Perform time alignment processing based on the movement trajectory timestamps and behavior trajectory timestamps to obtain time-aligned behavior data;

[0137] Step S44: Perform full-cycle dynamic behavior trajectory evolution according to the position configuration data and time-aligned behavior data of each segment of the trajectory to construct a full-cycle dynamic behavior trajectory timeline;

[0138] Step S45: Perform real-time rendering of the behavior trajectory on the panoramic digital model of the laboratory based on the full-cycle dynamic behavior trajectory timeline to generate a dynamic trajectory rendering twin model.

[0139] In this embodiment, spatial coordinate information is extracted from each trajectory to ensure that the movement trajectory and the operation behavior trajectory at each time point can be mapped to specific spatial positions, usually including X, Y, and Z coordinates. Using the nearest neighbor algorithm or the dynamic time warping (DTW) method, the data of the two trajectories are registered. This means that according to time series analysis, each time point in the movement trajectory is mapped to the corresponding spatial position of the operation behavior trajectory. For example, a spatial index (such as a KD tree) is used to efficiently find the nearest paired points. The registered trajectory data is combined with timestamps to form a complete trajectory dataset containing position configuration data. These data will be used for subsequent analysis and visualization. Timestamp matching: Traverse the timestamps of the movement trajectory and the behavior trajectory, and process the unmatched timestamps through linear interpolation. Specifically, when a certain time point exists in one trajectory but is missing in the other trajectory, the missing value is filled by interpolation so that both time series have corresponding trajectory data at the same time point. The aligned timestamps and the corresponding trajectory data are integrated into a new data structure. Ensure that each time point has a corresponding movement trajectory and behavior trajectory to form the final time-aligned behavior dataset. The time-aligned behavior data is saved to a CSV or a database for subsequent analysis and visualization. Trajectory evolution analysis: Using the position configuration data and the time-aligned behavior data, a dynamic behavior trajectory timeline is established. Analyze the behavior and its transitions at each time point to understand the changing patterns of the behavior. Use a state transition model or a Markov chain to model the behavior trajectory. Record the relationship between the behavior at each time point and the behavior at the previous time point to generate the evolution process of the dynamic behavior trajectory. By calculating the transition probability between each behavior, a complete state transition matrix is constructed. The results of the trajectory evolution are recorded in a data table, including information such as the timeline, behavior type, and position coordinates, for subsequent analysis and visualization. Import the panoramic digital model of the laboratory into a 3D graphics engine, such as Unity or Unreal Engine. This step ensures that the coordinate system of the model is consistent with the behavior trajectory data for accurate rendering. Using the API of the 3D graphics engine, the data on the full-cycle dynamic behavior trajectory timeline is converted into a visual trajectory. Draw a dynamic trajectory based on the trajectory data and add visual effects (such as color changes, transparency changes) to make the trajectory more visually appealing. Implement real-time rendering of the dynamic trajectory in the 3D graphics engine, allowing users to interact and observe the trajectory evolution. This process will generate a twin model of the dynamic trajectory rendering, which can reflect the behavior patterns of the people in the laboratory in real time.

[0140] In this embodiment, the specific steps of step S5 are as follows:

[0141] Step S51: Perform in-depth behavior semantic parsing on the twin model of the dynamic trajectory rendering to generate semantic features for each segment of the behavior trajectory;

[0142] Step S52: Based on the preset behavior logic knowledge base, conduct normalized behavior difference recognition on the semantic features of each behavior trajectory, and mark the differential behavior trajectories;

[0143] Step S53: Conduct behavior safety risk analysis on the differential behavior trajectories to generate abnormal safety risk data;

[0144] Step S54: Generate behavior safety warning signals according to the abnormal safety risk data.

[0145] In this embodiment, multi-dimensional data related to each segment of the behavior trajectory is collected, including timestamps, spatial positions, behavior types, etc., to ensure consistent data formats. The data is organized into a format suitable for model input, for example, converting time-series behavior data into a feature matrix. A selected deep learning model is trained using a labeled behavior dataset (such as a behavior library). Training parameters are set, with a learning rate of 0.001, a batch size of 32, and 100 epochs for the training cycle. The cross-entropy loss function is used for optimization. The trained model is applied to dynamic trajectory data to parse the semantic features of each segment of the behavior trajectory. The output layer of the model is used to obtain the semantic labels of each behavior and their confidence levels, forming a behavior trajectory dataset with semantic features. A behavior logic knowledge base is constructed, containing various standards and preset behavior patterns. A rule engine (such as Drools) is used to define normal behaviors. The standard features of normal behaviors are defined in the knowledge base, for example, "the time for a person in the laboratory to approach the equipment is 2 minutes", and the behavior features include position, time, and behavior type. The semantic features of each segment of the behavior trajectory are traversed and compared with the preset behavior logic knowledge base to identify trajectories that do not conform to the normal behavior pattern. An algorithm (such as Euclidean distance or cosine similarity) is used to calculate the differences in behavior features. The identified differential behavior trajectories are marked, and their difference types (such as "abnormal approach to the equipment" or "excessive stay time") are recorded, and the results are stored in a data table for subsequent analysis. A risk scoring model is adopted, combining decision tree or random forest algorithms to conduct safety risk analysis on differential behaviors to generate abnormal safety risk data. The differential behavior trajectory data is sorted out to construct a feature matrix, including position, time, behavior type, etc., as the input of the risk assessment model. The risk assessment model is trained using historical safety event data, and the proportion of paired samples is set to ensure that normal behaviors and abnormal behaviors each account for 50%. The F1 score is used as the evaluation index to ensure the classification performance of the model. The differential behavior trajectories are input into the trained risk assessment model to calculate the risk score of each behavior trajectory. According to the set threshold (such as 0.7), it is marked as "high risk" or "low risk". The generated abnormal safety risk data is saved to the database, recording the risk score and classification result of each behavior trajectory for subsequent early warning processing. A threshold-based early warning mechanism is set. When the abnormal risk score is higher than the set threshold, an early warning signal is triggered. A message queue (such as RabbitMQ) can be used for signal broadcasting and processing. An early warning threshold is set, for example, when the risk score exceeds 0.7, an early warning is triggered. It is adjusted according to historical data and expert opinions to ensure the rationality of the threshold. The abnormal safety risk data is traversed to check whether the risk score exceeds the set threshold. If it exceeds, an early warning signal is generated, and event information (such as time, personnel ID, risk score, etc.) is recorded. The early warning signal is sent to the monitoring system or the safety management platform using a message queue to ensure that relevant personnel can receive the alarm information in a timely manner.

[0146] In this embodiment, the specific steps of step S6 are as follows:

[0147] Step S61: Locate the abnormal operation device based on the abnormal safety risk data to identify the device with abnormal behavior operations;

[0148] Step S62: Monitor the status parameters of the devices with normal behavior operations and extract the real-time status parameters of the devices;

[0149] Step S63: Detect the abnormal status parameters of the real-time status parameters of the devices. When abnormal parameter changes are detected, generate an abnormal warning signal for the device;

[0150] Step S64: Make a risk warning decision based on the behavior safety warning signal and the device abnormal warning signal to obtain the laboratory risk decision strategy;

[0151] Step S65: Perform real-time visualization processing on the laboratory risk decision strategy and the dynamic trajectory rendering twin model, so as to execute the full-cycle digital management operation of the laboratory.

[0152] In this embodiment, a rule-based system and data mining techniques are used to combine abnormal security risk data and equipment operation records to identify the equipment performing abnormal operations. The abnormal security risk data is integrated with the equipment operation history data to ensure that the operation record of each equipment is combined with its corresponding abnormal risk data to form a complete data set. The integrated data set is traversed, and a set rule (such as "if the equipment operation time exceeds the preset limit and the risk score is higher than the set threshold") is used for anomaly detection. The conditional filtering function of the pandas library is used to efficiently process the data. According to the detected abnormal situations, all the equipment involved in abnormal operations is marked, and information such as equipment ID, operation time, and abnormal type is recorded, and the results are stored in the database for subsequent analysis and processing. The Internet of Things (IoT) technology and a real-time data acquisition system are adopted to regularly monitor the state parameters of normal operation equipment. Necessary sensors (such as temperature, pressure, current, etc.) are installed on each equipment in normal operation to ensure that the state parameters of the equipment can be obtained in real time. The sensor data acquisition frequency is set to once a minute. A data acquisition system (such as Raspberry Pi or Arduino) is used to regularly collect data from the sensors and send the data to the central database through Wi-Fi or a wired network. The extracted real-time state parameters are stored in the database to ensure consistent data formats for subsequent analysis. Statistical analysis methods (such as Z-score) or machine learning methods (such as Support Vector Machine (SVM)) are used for anomaly detection of real-time state parameters. The normal state parameter range (such as the upper and lower limits of temperature and pressure) is defined, and the Z-score of the real-time state parameters is calculated through statistical analysis methods to determine whether it exceeds the normal range. If the Z-score exceeds the set threshold (such as 3), it is marked as an abnormal state, and an equipment anomaly warning signal is generated. The detected abnormal state and its related information (such as equipment ID, abnormal parameter value, timestamp) are sent to the monitoring system to ensure that relevant personnel can receive the alarm information in time. The Analytic Hierarchy Process (AHP) or decision tree model is used for risk assessment and decision-making, and a comprehensive analysis is carried out by combining the behavioral safety warning signal and the equipment anomaly warning signal. The behavioral safety warning signal and the equipment anomaly warning signal are integrated to form a comprehensive risk assessment data set. Each record should include information such as signal type, risk score, timestamp, etc. A decision model is applied to analyze the risk data, and corresponding decision-making strategies are set according to different risk levels, such as "immediately deactivate the equipment for high-risk behaviors". According to the analysis results, laboratory risk decision-making strategies are generated, and information such as the execution conditions, execution personnel, and time of each strategy is recorded for subsequent management and execution. A 3D visualization engine (such as Unity or WebGL) is used for real-time visualization of risk decision-making strategies and dynamic trajectories to ensure that operators can intuitively understand the risk situation. A data interface is designed to push the risk decision-making strategies and dynamic trajectory data to the visualization platform in real time for real-time update and display.In a 3D visualization platform, real-time rendering of dynamic trajectories is performed, and the visualization effects (such as color changes, tooltips, etc.) are adjusted according to the risk decision-making strategy to facilitate the rapid identification of risk areas by operators. Through the real-time monitoring function provided by the visualization platform, operators are allowed to view the dynamic trajectories, risk signals, and equipment status of the laboratory to ensure timely response and decision-making.

[0153] In this embodiment, a laboratory digital management system is provided for implementing the laboratory digital management method described above, including:

[0154] An object vision recognition module for obtaining real-time panoramic monitoring videos of the laboratory; performing multi-object vision recognition and regional layout analysis on the real-time panoramic monitoring videos of the laboratory to construct a laboratory panoramic structure diagram;

[0155] A trend mapping module for identifying laboratory equipment status parameters based on multiple sensors; performing equipment position registration on the laboratory panoramic structure diagram according to the laboratory equipment status parameters and performing trend mapping to construct a laboratory panoramic digital model;

[0156] A dynamic optical flow tracking module for optimizing the time-series frame-to-frame delay of the real-time panoramic monitoring videos of the laboratory and performing dynamic optical flow tracking to generate a time-series movement trajectory and an operation behavior trajectory sequence for each person;

[0157] An instant rendering module for performing full-cycle dynamic behavior trajectory evolution based on the time-series movement trajectory and operation behavior trajectory sequence of each person and performing instant rendering of the behavior trajectory on the laboratory panoramic digital model to generate a dynamic trajectory rendering twin model;

[0158] A behavior semantic analysis module for performing in-depth behavior semantic analysis and behavior safety risk analysis on the dynamic trajectory rendering twin model to generate behavior safety warning signals;

[0159] A real-time visualization module for making risk warning decisions based on the behavior safety warning signals and performing real-time visualization processing in combination with the dynamic trajectory rendering twin model to execute the full-cycle digital management operation of the laboratory.

[0160] By obtaining the panoramic surveillance video of the laboratory in real time, the present invention can comprehensively monitor the dynamics of the laboratory and enhance the monitoring ability of the laboratory. Through multi-object visual recognition and regional layout analysis, it can accurately identify different areas, equipment, personnel, etc. in the laboratory and construct a panoramic structure diagram of the laboratory. Using a variety of sensor data to monitor the status of laboratory equipment in real time (such as temperature, pressure, working status, etc.), it can accurately grasp the health status of the equipment. Registering the equipment status with the panoramic structure diagram of the laboratory improves the visualization of equipment management and the accuracy of spatial layout. The use of trend mapping can obtain the trend of equipment status changes in real time, providing data support for predictive maintenance and laboratory operation to prevent potential failures. The optimization of the time-series frame delay effectively reduces the delay in the video processing process and improves the response speed and smoothness of the real-time surveillance video. The dynamic optical flow tracking technology can track the real-time movement trajectory and operation behavior trajectory of each person in the laboratory. Through this technology, not only the position of personnel can be dynamically tracked, but also their behaviors and operation processes can be recorded, providing data support for subsequent behavior analysis and safety detection. The extraction of the personnel behavior trajectory sequence helps to better understand the activity patterns of personnel in the laboratory, thus making reasonable judgments on the safety and operation behaviors of the laboratory. By performing full-cycle dynamic behavior trajectory evolution on the time-series movement trajectory and operation behavior trajectory sequence of personnel, the long-term behavior patterns of personnel can be simulated and predicted, enhancing the management predictability of the laboratory. Rendering the behavior trajectory instantaneously into the panoramic digital model of the laboratory forms a dynamic trajectory rendering twin model. This twin model can display the real-time changes in personnel activities in a visual way, helping managers to instantly understand the dynamics in the laboratory. It provides an immersive monitoring method for managers, enabling them to quickly identify potential problems in a visual environment. Deep behavior semantic parsing extracts meaningful safety hazards from complex behavior trajectories to help identify potential safety problems. Through the safety risk analysis of behavior trajectories, dangerous behavior patterns and abnormal behaviors are accurately identified, generating safety warning signals to enhance the safety of the laboratory. The probability of laboratory accidents is effectively reduced, and intervention and correction are carried out in a timely manner to ensure the safety of personnel and equipment. According to the behavior safety warning signal, risk warning decisions are made, and corresponding measures can be taken before safety hazards occur to avoid potential hazards. The combination of the dynamic trajectory rendering twin model and real-time visualization processing can display the safety status in the laboratory in real time when a safety risk occurs, helping laboratory managers to quickly respond and make decisions. It realizes full-cycle digital management operations, making the management of the laboratory more intelligent and automated, thereby reducing manual intervention and improving work efficiency and management quality.

[0161] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be encompassed within the present invention.

[0162] As described above, these are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather will conform to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A laboratory digital management method, characterized in that: The following steps are involved: Step S1: Acquire a real-time panoramic surveillance video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic surveillance video of the laboratory to construct a panoramic structure diagram of the laboratory; Step S2: Identify laboratory equipment status parameters based on multiple sensors; perform equipment position registration on the laboratory panoramic structure diagram according to the laboratory equipment status parameters, and perform trend mapping to construct a laboratory panoramic digital model; Step S3: Optimize the time-series frame delay of the real-time panoramic monitoring video of the laboratory, and perform dynamic optical flow tracking to generate the time-series movement trajectory and operation behavior trajectory sequence of each person; Step S4: Perform full-cycle dynamic behavior trajectory evolution according to the temporal movement trajectory and operation behavior trajectory sequence of each person, and perform real-time behavior trajectory rendering on the laboratory panoramic digital model to generate a dynamic trajectory rendering twin model; Step S5: Perform deep behavior semantic analysis and behavior safety risk analysis on the dynamic trajectory rendering twin model to generate a behavior safety warning signal; Step S6: Make risk warning decisions based on behavioral safety warning signals, and combine them with the dynamic trajectory rendering twin model for real-time visualization, so as to execute the full-cycle digital management of the laboratory.

2. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire real-time panoramic monitoring video of the laboratory; Step S12: Perform multi-object visual recognition on the real-time panoramic monitoring video of the laboratory to extract all laboratory object nodes; Step S13: positioning all laboratory object nodes in three-dimensional space one by one to obtain the three-dimensional position coordinates of each object; Step S14: mining the inter-object topological associations of all laboratory object nodes to generate inter-object topological structure data; Step S15: performing regional layout analysis on the real-time panoramic monitoring video of the laboratory according to the topological structure data between objects to generate regional layout structure features; Step S16: Perform panoramic structural reconstruction of the regional layout structural features according to the three-dimensional position coordinates of each object to construct a panoramic structural diagram of the laboratory.

3. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: identifying laboratory equipment status parameters based on multiple sensors; Step S22: performing a time series operation analysis of the laboratory equipment status parameters at multiple time points to extract the equipment operation characteristics at multiple time points; Step S23: performing adjacent time differential detection on the equipment operation characteristics at multiple time points to generate operation difference change data for each equipment; Step S24: Perform real-time trend fitting on the operation difference change data of each device to construct a real-time status trend graph of each device; Step S25: According to the real-time status trend diagram of each device, the device position is aligned with the laboratory panoramic structure diagram, and trend mapping is performed to construct a laboratory panoramic digital model.

4. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: optimizing the time-series inter-frame delay of the real-time panoramic monitoring video of the laboratory to obtain an inter-frame smooth optimized video; Step S32: Performing frame-by-frame visual recognition of personnel on the inter-frame smoothing optimized video, and marking the personnel in the laboratory; Step S33: performing dynamic optical flow tracking on the personnel in the laboratory to obtain the dynamic optical flow trajectory of the personnel; Step S34: performing multi-period position movement calculation on the dynamic optical flow trajectory of the personnel to extract the temporal movement trajectory of each person; Step S35: mining the equipment operation behavior of the personnel's dynamic optical flow trajectory to extract the equipment operation behavior trajectory; Step S36: performing time-series logic fitting on the device operation behavior trajectory to generate an operation behavior trajectory sequence.

5. The laboratory digital management method according to claim 4, characterized in that: The specific steps of step S31 are: Calculate the global average frame rate of the real-time panoramic monitoring video of the laboratory to obtain the average frame rate of the video; Performing video frame rate fluctuation identification according to the average frame rate of the video to generate a video frame rate fluctuation feature; The video frame rate fluctuation characteristics are used to perform frame-by-frame delay analysis on the laboratory's real-time panoramic monitoring video to obtain the delay data between adjacent frames; Dynamically and adaptively smoothing the delay data between adjacent frames to construct an adaptive smoothing curve; Delayed interpolation reconstruction is performed based on an adaptive smooth curve to obtain smooth and optimized video between frames.

6. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: extracting movement trajectory timestamps and behavior trajectory timestamps according to the temporal movement trajectory and operation behavior trajectory sequence of each person; Step S42: performing behavior space position registration on the temporal movement trajectory and operation behavior trajectory sequence of each person, so as to mark the position configuration data of each segment of the trajectory; Step S43: performing time alignment processing based on the movement trajectory timestamp and the behavior trajectory timestamp, thereby obtaining time-aligned behavior data; Step S44: performing full-cycle dynamic behavior trajectory evolution according to the position configuration data and time-aligned behavior data of each trajectory segment to construct a full-cycle dynamic behavior trajectory time axis; Step S45: Based on the full-cycle dynamic behavior trajectory timeline, the behavior trajectory of the laboratory panoramic digital model is rendered in real time to generate a dynamic trajectory rendering twin model.

7. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: performing deep behavior semantic analysis on the dynamic trajectory rendering twin model to generate semantic features of each behavior trajectory; Step S52: performing normalized behavioral difference recognition on the semantic features of each behavioral trajectory based on a preset behavioral logic knowledge base, and marking the differentiated behavioral trajectory; Step S53: Performing behavioral safety risk analysis on the differentiated behavioral trajectories to generate abnormal safety risk data; Step S54: Generate a behavioral safety warning signal based on the abnormal safety risk data.

8. The laboratory digital management method according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: locating abnormally operated devices based on abnormal security risk data, and identifying devices with abnormal behavior operations; Step S62: Monitor the status parameters of the equipment in normal operation and extract the real-time status parameters of the equipment; Step S63: abnormal state parameter detection is performed on the real-time state parameters of the equipment, and when an abnormal change in the parameters is detected, an abnormal warning signal of the equipment is generated; Step S64: making risk warning decisions based on the behavior safety warning signals and the equipment abnormality warning signals to obtain a laboratory risk decision strategy; Step S65: Perform real-time visualization processing on the laboratory risk decision-making strategy and dynamic trajectory rendering twin model, so as to perform the digital management operation of the laboratory throughout the entire cycle.

9. A laboratory digital management system, characterized in that: Used to execute the laboratory digital management method as claimed in claim 1, comprising: The object visual recognition module is used to obtain real-time panoramic monitoring video of the laboratory; perform multi-object visual recognition and regional layout analysis on the real-time panoramic monitoring video of the laboratory to construct a panoramic structure diagram of the laboratory; The trend mapping module is used to identify the status parameters of laboratory equipment based on multiple sensors; the equipment position is registered on the laboratory panoramic structure diagram according to the laboratory equipment status parameters, and trend mapping is performed to build a panoramic digital model of the laboratory; Dynamic optical flow tracking module, which is used to optimize the time-series frame delay of the real-time panoramic monitoring video of the laboratory, and perform dynamic optical flow tracking to generate the time-series movement trajectory and operation behavior trajectory sequence of each person; The real-time rendering module is used to perform full-cycle dynamic behavior trajectory evolution according to the temporal movement trajectory and operation behavior trajectory sequence of each person, and to perform real-time behavior trajectory rendering on the panoramic digital model of the laboratory, thereby generating a dynamic trajectory rendering twin model; The behavior semantics analysis module is used to perform in-depth behavior semantics analysis and behavior safety risk analysis on the dynamic trajectory rendering twin model, and generate behavior safety warning signals; The real-time visualization module is used to make risk warning decisions based on behavioral safety warning signals, and combines the dynamic trajectory rendering twin model for real-time visualization processing, thereby executing the full-cycle digital management of the laboratory.

Images