Method and system for managing hazardous chemical substances in laboratory

Abstract

The invention provides a management method and system for hazardous chemicals in a laboratory, and relates to the technical field of hazardous articles, and the method comprises the steps: obtaining hazardous article operation video data of an operator through an image collection device, extracting key node coordinates and skeleton movement track information, and mapping the information into standardized operation behavior sequence data; and a time sequence behavior evaluation model constructed based on a long and short term memory network is used to identify the non-standard operation fragments, a risk level coefficient is calculated in combination with a dangerous goods attribute information base, and a multi-dimensional evaluation report is generated. And according to the evaluation report, calling a correction strategy library to generate an improvement suggestion containing characters, videos and three-dimensional models, and pushing the improvement suggestion to an operator. By monitoring the behavior of the operator after executing the improvement suggestion and dynamically updating the risk score and the time sequence behavior evaluation model, the dynamic optimization of the evaluation standard is realized, and the safety of the laboratory hazardous chemical substance operation is effectively improved.

Description

Technical Field

[0001] The present invention relates to hazardous chemicals technology, and in particular to a management method and system for hazardous chemicals in laboratories. Background Art

[0002] Laboratory safety, especially operational safety involving hazardous chemicals, has always been an important concern in scientific research and production activities. Traditional laboratory safety management mainly relies on manual supervision and paper records, which has problems such as low efficiency, strong subjectivity, and difficulty in comprehensive coverage. With the rapid development of technologies such as computer vision and artificial intelligence, more and more research is devoted to using these technologies to achieve automation and intelligence in laboratory safety management. For example, some studies use image recognition technology to detect the clothing of laboratory personnel and remind them to wear necessary protective equipment; other studies try to use sensors to monitor environmental parameters such as temperature, humidity, gas concentration, etc. to prevent potential dangers.

[0003] However, the existing laboratory safety management technology still has some defects and shortcomings: First, the analysis of operating behaviors is not detailed enough, and it is difficult to identify subtle violations. Most existing technologies focus on macro scene recognition, such as whether personnel are operating in designated areas, whether they are wearing protective equipment, etc., but lack effective analysis methods for the specific action norms during the operation. Secondly, there is a lack of targeted correction guidance. Even if illegal operations are found, the existing system can often only give simple warnings, but cannot provide specific improvement suggestions, making it difficult to help operators effectively correct errors. Finally, the evaluation standards are difficult to adjust dynamically. Laboratory operating specifications may be updated with new research progress or changes in safety standards. Existing systems are usually difficult to flexibly adapt to these changes, resulting in lagging evaluation standards and affecting the effectiveness of safety management. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for managing hazardous chemicals in a laboratory, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention, Provide management methods for hazardous chemicals in laboratories, including: The dangerous goods operation video data of the operator is obtained by image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include an infrared thermal imaging camera and a depth camera; the operation video data is input into a pre-trained human posture recognition model to extract the key node coordinate information and the skeletal motion trajectory information of the operator; based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data; The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, the risk level coefficient corresponding to each irregular operation is calculated in combination with a preset dangerous goods attribute information library; based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation segments, a multidimensional evaluation report including operation time, location, action type and risk score is generated; According to the multi-dimensional evaluation report, the preset correction strategy library is called to generate targeted operational behavior improvement suggestions, which include text descriptions, standard action videos and three-dimensional action demonstration models; the operational behavior improvement suggestions are pushed to operators through laboratory terminal devices; the operational behavior of operators after executing the improvement suggestions is monitored, and the risk score is updated. When the risk score is lower than the preset threshold for three consecutive times, the corresponding operational behavior sequence data is added to the standard operation sample library; based on the standard operation sample library, the parameters of the timing behavior evaluation model are regularly updated to achieve dynamic optimization of the evaluation criteria.

[0006] Inputting the operation video data into a pre-trained human posture recognition model, extracting the key node coordinate information and skeletal motion trajectory information of the operator includes: The operation video data is input into a pre-trained human posture recognition model, the human posture recognition model uses a two-stream feature extraction network to process RGB image sequences and depth image sequences respectively, the RGB image sequence is subjected to a ResNet-50 network to extract the operator's appearance features, and the depth image sequence is subjected to a PointNet++ network to process three-dimensional point cloud data to obtain the operator's spatial position information; based on the attention mechanism, the feature channel importance scores of the appearance features and the spatial position information are calculated, and the performance features and the spatial position information are weightedly fused according to the importance scores to obtain fused features; Input the fused features into the spatiotemporal convolution module, perform key node detection on the features output by the spatiotemporal convolution module through a multi-scale feature pyramid, and extract key node coordinate information in combination with a position sensitive loss function; A skeleton connection graph is constructed based on the key node coordinate information, the skeleton node sequence is processed by a graph convolutional network to learn the motion correlation between nodes, and the predicted trajectory of the skeleton node sequence is smoothed by Kalman filtering to obtain the skeleton motion trajectory information.

[0007] Based on the preset dangerous goods operation specification database, mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data includes: Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data, and the standard actions in the dangerous goods operation specification database are decomposed into basic action units, wherein the basic action units include the relative position relationship of key nodes and motion trajectory characteristics; A dynamic time warping algorithm is used to match and align the skeletal motion trajectory information with the basic action unit; an adaptive scale transformation module is used to normalize the key node coordinate information; Based on the probabilistic graphical model, a state transfer matrix containing timing constraints and spatial constraints is established, and the optimal state sequence is solved by the Viterbi algorithm in combination with the type of dangerous goods and the operating environment information; the optimal state sequence is converted into the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit timing relationship, normalized coordinates and motion characteristics.

[0008] The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, combined with a preset dangerous goods attribute information library, the risk level coefficient corresponding to each irregular operation is calculated, including: The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network, the input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to time windows with a duration of two seconds and an adjacent window overlap rate of fifty percent, and extracts the relative position relationship of key nodes, movement speed, acceleration, and joint angle change rate of the operation behavior sequence data in each time window to construct a multidimensional feature vector; three types of convolution kernels, namely one by one, three by three, and five by five, are used to process the multidimensional feature vector in parallel, and temporal features are obtained by adaptively fusing features of different scales through a channel attention mechanism; Inputting the temporal features into a bidirectional long short-term memory network, wherein the bidirectional long short-term memory network includes a forward long short-term memory network and a reverse long short-term memory network, wherein each network unit of the forward long short-term memory network and the reverse long short-term memory network includes an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism; calculating the importance weights of different time steps based on a temporal attention mechanism, and identifying irregular operation fragments in the operation behavior sequence data according to the importance weights; For the irregular operation fragments, combined with the preset hazardous goods attribute information library, the risk level coefficient corresponding to each irregular operation fragment is calculated, the hazardous characteristic parameters are extracted from the hazardous goods attribute information library, and the position deviation, speed deviation, posture deviation characteristics of the irregular operation fragments are multi-dimensionally mapped with the hazardous characteristic parameters to construct a risk assessment matrix; the hierarchical analysis method is used to determine the weight coefficients of different deviation dimensions, and the risk level coefficient is calculated using an exponentially growing nonlinear mapping function for deviations exceeding the safety threshold.

[0009] According to the multi-dimensional evaluation report, the preset correction strategy library is called to generate targeted operation behavior improvement suggestions, which include text descriptions, standard action videos and three-dimensional action demonstration models; the operation behavior improvement suggestions are pushed to the operator through the laboratory terminal device, including: Based on the risk features in the multidimensional assessment report, a feature vector is constructed; based on the knowledge graph retrieval algorithm, the similarity between the feature vector and the strategy nodes in the preset deviation correction strategy library is calculated, and multiple strategies with the highest similarity are selected to form a candidate strategy set; fuzzy reasoning is used to perform environmental constraint assessment, risk assessment, and operator capability assessment on the candidate strategy set, and the adaptability score of each strategy is obtained through weighted fusion; The strategy with the highest adaptability score in the preset correction strategy library is called to generate operational behavior improvement suggestions, and the strategy is converted into a text description containing specific parameters and operational points; a standard action sequence matching the current operation type is retrieved from the action template library, and a standard action video is generated through posture alignment and trajectory smoothing; a motion trajectory is constructed based on the key frames of the standard action sequence, and the motion trajectory is mapped to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model; the operational behavior improvement suggestions include the text description, the standard action video and the three-dimensional action demonstration model.

[0010] Constructing a motion trajectory based on the key frames of the standard motion sequence, and mapping the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional motion demonstration model includes: Calculating the optical flow field between adjacent frames in the standard action sequence, and constructing a motion energy function according to the amplitude change and direction change of the optical flow field; performing peak detection on the motion energy function based on an adaptive threshold, and marking the frame corresponding to the peak as a key frame; performing spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a standardized key frame; The key point positions in the standardized key frame are interpolated using a cubic spline interpolation function to generate an initial motion trajectory; joint angle constraints and speed constraints are constructed according to human kinematic characteristics, and the joint angle constraints and speed constraints are applied to the initial motion trajectory as boundary conditions; the initial motion trajectory is smoothed with minimizing acceleration as the optimization goal to obtain a smooth motion trajectory; the interpolation point density is adjusted based on the time information of the key frame to adjust the speed of the smooth motion trajectory; The human skeleton structure is decomposed into multiple kinematic chains, and priorities are assigned to each kinematic chain based on the importance of the end effector; the joint angles are directly calculated for the kinematic chains with analytical solutions, and the joint angles are iteratively solved using the least squares method with a damping factor for the kinematic chains without analytical solutions; a joint comfort evaluation function is constructed to calculate the deviation between the joint angle and the natural posture, and the deviation is added as a soft constraint to the optimization target; the error change rate during the iteration process is monitored, and the damping factor is dynamically adjusted according to the error change rate; The variation range of the joint angle between adjacent frames is detected to identify the joint angle causing the jump; the trajectory of the identified jump joint angle is replanned using the minimum acceleration criterion to achieve a smooth transition; the optimized joint angle sequence is applied to the human skeleton model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0011] According to a second aspect of the embodiments of the present invention, Provide a management system for hazardous chemicals in the laboratory, including: The first unit is used to obtain the dangerous goods operation video data of the operator through the image acquisition devices deployed in multiple areas of the laboratory, and the image acquisition devices include infrared thermal imaging cameras and depth cameras; input the operation video data into a pre-trained human posture recognition model to extract the key node coordinate information and skeleton motion trajectory information of the operator; based on a preset dangerous goods operation specification database, the key node coordinate information and skeleton motion trajectory information are mapped into standardized operation behavior sequence data; The second unit is used to input the operation behavior sequence data into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, in combination with a preset dangerous goods attribute information library, a risk level coefficient corresponding to each irregular operation is calculated; based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation segments, a multidimensional evaluation report including operation time, location, action type and risk score is generated; The third unit is used to call a preset correction strategy library according to the multi-dimensional evaluation report, generate targeted operation behavior improvement suggestions, and the operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; push the operation behavior improvement suggestions to operators through laboratory terminal devices; monitor the operation behavior of operators after executing the improvement suggestions, and update the risk score. When the risk score is lower than the preset threshold for three consecutive times, the corresponding operation behavior sequence data is added to the standard operation sample library; based on the standard operation sample library, the parameters of the timing behavior evaluation model are regularly updated to achieve dynamic optimization of the evaluation criteria.

[0012] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0013] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0014] The beneficial effects of this application are as follows: 1. Improve the safety of hazardous chemical operations: Through real-time monitoring and evaluation of operator behavior, timely detection and correction of irregular operations, effectively reduce the risk of accidents caused by human errors, and ensure the safety of laboratory personnel and the environment.

[0015] 2. Realize the standardization and normalization of operating behaviors: Use artificial intelligence technology to convert operating specifications into quantifiable indicators, and provide targeted improvement suggestions to help operators learn and master correct operating methods, and promote the standardization and normalization of laboratory operating behaviors.

[0016] 3. Continuously improve the evaluation model and operating specifications: By continuously accumulating and analyzing operating data, dynamically updating the evaluation model and standard operation sample library, the evaluation standards will be more in line with the actual operating conditions, and the continuous improvement and optimization of the evaluation system will be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for managing hazardous chemicals in a laboratory according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a management system for hazardous chemicals in a laboratory according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Figure 1 FIG. 1 is a flow chart of a method for managing hazardous chemicals in a laboratory according to an embodiment of the present invention. Figure 1 As shown, the method includes: S101. Acquire the operator's dangerous goods operation video data through image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include an infrared thermal imaging camera and a depth camera; input the operation video data into a pre-trained human posture recognition model to extract the operator's key node coordinate information and skeletal motion trajectory information; based on a preset dangerous goods operation specification database, map the key node coordinate information and skeletal motion trajectory information into standardized operation behavior sequence data; S102. Input the operation behavior sequence data into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, in combination with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each irregular operation; based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation segments, generate a multidimensional evaluation report including operation time, location, action type and risk score; S103. Based on the multi-dimensional evaluation report, call the preset correction strategy library to generate targeted operational behavior improvement suggestions, which include text descriptions, standard action videos and three-dimensional action demonstration models; push the operational behavior improvement suggestions to the operator through the laboratory terminal device; monitor the operational behavior of the operator after executing the improvement suggestions, update the risk score, and when the risk score is lower than the preset threshold for three consecutive times, add the corresponding operational behavior sequence data to the standard operation sample library; based on the standard operation sample library, regularly update the parameters of the timing behavior evaluation model to achieve dynamic optimization of the evaluation criteria.

[0021] In an optional implementation, the operation video data is input into a pre-trained human posture recognition model, and the key node coordinate information and skeletal motion trajectory information of the operator are extracted, including: The operation video data is input into a pre-trained human posture recognition model, the human posture recognition model uses a two-stream feature extraction network to process RGB image sequences and depth image sequences respectively, the RGB image sequence is subjected to a ResNet-50 network to extract the operator's appearance features, and the depth image sequence is subjected to a PointNet++ network to process three-dimensional point cloud data to obtain the operator's spatial position information; based on the attention mechanism, the feature channel importance scores of the appearance features and the spatial position information are calculated, and the performance features and the spatial position information are weightedly fused according to the importance scores to obtain fused features; Input the fused features into the spatiotemporal convolution module, perform key node detection on the features output by the spatiotemporal convolution module through a multi-scale feature pyramid, and extract key node coordinate information in combination with a position sensitive loss function; A skeleton connection graph is constructed based on the key node coordinate information, the skeleton node sequence is processed by a graph convolutional network to learn the motion correlation between nodes, and the predicted trajectory of the skeleton node sequence is smoothed by Kalman filtering to obtain the skeleton motion trajectory information.

[0022] In order to accurately recognize human postures and extract skeletal motion trajectories in operation videos, this embodiment proposes a human posture recognition method based on a dual-stream feature extraction network and spatiotemporal convolution. This method integrates the information of RGB images and depth images, and uses the attention mechanism and graph convolution network to effectively improve the posture recognition accuracy and trajectory smoothness.

[0023] First, the operation video is preprocessed and segmented into RGB image sequence and depth image sequence. Taking a video containing 100 frames as an example, it is segmented into 100 RGB images and corresponding 100 frames of depth image data.

[0024] Then, the RGB image sequence is input into the pre-trained ResNet-50 network to extract the operator's appearance features. For example, each frame of RGB image is input into the ResNet-50 network to obtain a 1024-dimensional feature vector, and finally a 100x1024 feature matrix is ​​obtained, which represents the appearance features of the video.

[0025] At the same time, the depth image sequence is converted into 3D point cloud data and input into the pre-trained PointNet++ network to obtain the spatial position information of the operator. Assuming that each frame of the depth image contains 1000 point cloud data, each point cloud data contains 3D coordinate information, then the output of the PointNet++ network can be a 100x1000x3 tensor, which is processed to obtain a 100x512 feature matrix, which represents the spatial position information of the video.

[0026] Next, the attention mechanism is used to calculate the importance scores of the feature channels of the appearance features and spatial position information. For example, for the appearance features, the mean or variance of each channel can be calculated as its importance score. Assume that 1024 score values ​​are calculated and normalized to between 0 and 1. Similar processing is performed on the spatial position information to obtain 512 normalized score values.

[0027] According to the calculated feature channel importance score, the apparent features and spatial position information are weightedly fused to obtain fused features. For example, each channel of the apparent features is multiplied by the corresponding score value, and each channel of the spatial position information is multiplied by the corresponding score value, and then the two are added together to obtain a 100x1536 fused feature matrix.

[0028] The fused features are input into the spatiotemporal convolution module, for example, using a 3D convolutional network to extract spatiotemporal features. Assume that the output of the spatiotemporal convolution module is a 100x256 feature matrix.

[0029] The features output by the spatiotemporal convolution module are used to detect key nodes through a multi-scale feature pyramid. For example, convolution kernels of different sizes are used to convolve the feature map to obtain feature maps of different scales, and then these feature maps are fused to obtain the final key node detection results. Assuming that 17 key nodes of the human body are detected, the output is a 100x17x2 tensor, which represents the two-dimensional coordinates of each key node at each moment.

[0030] Combined with the position-sensitive loss function, the key node coordinate information is further optimized to improve the detection accuracy. For example, the loss is calculated based on the predicted key node coordinates and the actual key node coordinates, and the network parameters are updated by back propagation.

[0031] The skeleton connection diagram is constructed based on the extracted key node coordinate information. For example, 17 key nodes are connected to form the skeleton structure of the human body.

[0032] The graph convolutional network is used to process the sequence of skeleton nodes and learn the motion correlation between nodes. For example, the skeleton connection graph at each moment is input into the graph convolutional network to obtain the feature vector of each node, and then these feature vectors are combined into a sequence to learn the motion relationship between nodes.

[0033] Kalman filtering is used to smooth the predicted trajectory of the skeleton node sequence to obtain the final skeleton motion trajectory information. For example, Kalman filtering is used to predict the position of each node at the next moment, and the prediction results are smoothed to remove noise and jitter.

[0034] The solution of this application can: Improved posture recognition accuracy: By fusing the information of RGB images and depth images and using the attention mechanism, the human posture features can be captured more comprehensively, thereby improving posture recognition accuracy. Enhanced robustness: Using the spatiotemporal convolution module and multi-scale feature pyramid, it can effectively cope with complex scenes such as occlusion and lighting changes, and enhance the robustness of the algorithm. Improved trajectory smoothness: Using a graph convolutional network to learn the motion correlation between nodes and combining it with Kalman filtering for smoothing can effectively remove noise and jitter in the skeleton motion trajectory and improve trajectory smoothness.

[0035] In an optional implementation, based on a preset dangerous goods operation specification database, mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data includes: Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data, and the standard actions in the dangerous goods operation specification database are decomposed into basic action units, wherein the basic action units include the relative position relationship of key nodes and motion trajectory characteristics; A dynamic time warping algorithm is used to match and align the skeletal motion trajectory information with the basic action unit; an adaptive scale transformation module is used to normalize the key node coordinate information; Based on the probabilistic graphical model, a state transfer matrix containing timing constraints and spatial constraints is established, and the optimal state sequence is solved by the Viterbi algorithm in combination with the type of dangerous goods and the operating environment information; the optimal state sequence is converted into the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit timing relationship, normalized coordinates and motion characteristics.

[0036] The core of the normalization method of dangerous goods operation behavior is to transform the key node coordinate information and skeletal motion trajectory information into standardized operation behavior sequence data. This method is based on a preset dangerous goods operation specification database and is implemented using technologies such as dynamic time warping, adaptive scaling, probabilistic graphical model and Viterbi algorithm.

[0037] First, prepare a preset database of dangerous goods operation specifications. This database contains the operation specifications of various dangerous goods, such as operation steps, key node positions, motion trajectories and other information. Taking the handling of hazardous chemicals as an example, the database will include standard actions such as "wearing protective clothing", "checking the sealing of containers", "handling with both hands", "handling with care", etc., and record in detail the relative position relationship and motion trajectory characteristics of the key nodes (such as hands, elbows, and containers) of each standard action. For example, the standard action of "hand-held handling" will record the relative position relationship between the hands and the container (such as hugging the container with both hands), as well as the motion trajectory of the hands during the handling process (such as smooth movement from the starting position to the target position).

[0038] Next, the standard actions in the database are decomposed into basic action units. Each basic action unit contains the relative position relationship of key nodes and motion trajectory features. For example, "two-handed transport" can be decomposed into basic action units such as "grabbing the container", "lifting the container", "moving the container" and "putting down the container". Each unit contains the relative position relationship of key nodes (two hands and container) and motion trajectory features. For example, the relative position relationship of the key nodes of the "grabbing container" unit is that both hands embrace the container, and the motion trajectory feature is the trajectory of both hands from far away from the container to close to the container and finally embracing the container.

[0039] Then, the key node coordinate information and skeletal motion trajectory information are extracted from the dangerous goods operation video to be analyzed. For example, the coordinate information of the operator's hands, elbows, torso and other key nodes in the process of carrying dangerous chemicals, as well as the motion trajectory information of these key nodes, are obtained through motion capture technology.

[0040] The extracted skeletal motion trajectory information is matched and aligned with the motion trajectory features of each basic action unit in the database. The dynamic time warping algorithm is used to calculate the similarity and find the basic action unit that best matches the current skeletal motion trajectory information. For example, the motion trajectory of the operator's hands during the handling process is compared with the motion trajectory features of basic action units such as "grabbing the container" and "lifting the container" to find the unit with the highest similarity.

[0041] At the same time, the extracted key node coordinate information is normalized. The adaptive scale transformation module is used to map the key node coordinate information of different operators and different operating environments to a unified scale range. For example, the coordinate information of the hands of operators of different heights is mapped to a standard human body model through scale transformation.

[0042] Next, a state transition matrix is ​​established based on the probabilistic graphical model. This matrix contains timing constraints and spatial constraints, which are used to describe the transition probabilities between different basic action units. The timing constraints reflect the temporal order of the basic action units, for example, "grabbing the container" must be before "lifting the container". The spatial constraints reflect the spatial position relationship of the basic action units, for example, after "lifting the container", the position of the container should be higher than the initial position. Combined with the type of dangerous goods and the operating environment information, the state transition matrix is ​​further refined. For example, when carrying flammable liquids, the probability of the "handle with care" action will be higher.

[0043] The Viterbi algorithm is used to solve the optimal state sequence, that is, to find the basic action unit sequence that best matches the observed key node coordinate information and skeletal motion trajectory information.

[0044] Finally, the optimal state sequence is converted into standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit temporal relationship, normalized coordinates and motion features. For example, the identified basic action units such as "grab container", "lift container", "move container" and "put down container" are arranged in time sequence and combined with the normalized coordinate information and motion features to generate standardized operation behavior sequence data, for example: {"carry": [{"grab container": [coordinate information 1, motion feature 1]}, {"lift container": [coordinate information 2, motion feature 2]}, {"move container": [coordinate information 3, motion feature 3]}, {"put down container": [coordinate information 4, motion feature 4]}]}.

[0045] The solution of this application can: Improve operational standardization: Convert the operator's behavior into standardized data to facilitate comparison with the preset operating specifications, so as to determine whether the operation complies with the specifications and improve the safety of the operation. Improve training efficiency: By analyzing the standardized operating behavior sequence data, it is possible to identify irregular behaviors during the operation process, provide targeted guidance for operator training, and thus improve training efficiency. Facilitate automated analysis: Standardized operating behavior sequence data facilitates computer automated analysis, such as identifying dangerous operations, predicting accident risks, etc., thereby providing support for the intelligent management of dangerous goods operations.

[0046] In an optional implementation, the operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, combined with a preset dangerous goods attribute information library, the risk level coefficient corresponding to each irregular operation is calculated, including: The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network, the input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to time windows with a duration of two seconds and an adjacent window overlap rate of fifty percent, and extracts the relative position relationship of key nodes, movement speed, acceleration, and joint angle change rate of the operation behavior sequence data in each time window to construct a multidimensional feature vector; three types of convolution kernels, namely one by one, three by three, and five by five, are used to process the multidimensional feature vector in parallel, and temporal features are obtained by adaptively fusing features of different scales through a channel attention mechanism; Inputting the temporal features into a bidirectional long short-term memory network, wherein the bidirectional long short-term memory network includes a forward long short-term memory network and a reverse long short-term memory network, wherein each network unit of the forward long short-term memory network and the reverse long short-term memory network includes an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism; calculating the importance weights of different time steps based on a temporal attention mechanism, and identifying irregular operation fragments in the operation behavior sequence data according to the importance weights; For the irregular operation fragments, combined with the preset hazardous goods attribute information library, the risk level coefficient corresponding to each irregular operation fragment is calculated, the hazardous characteristic parameters are extracted from the hazardous goods attribute information library, and the position deviation, speed deviation, posture deviation characteristics of the irregular operation fragments are multi-dimensionally mapped with the hazardous characteristic parameters to construct a risk assessment matrix; the hierarchical analysis method is used to determine the weight coefficients of different deviation dimensions, and the risk level coefficient is calculated using an exponentially growing nonlinear mapping function for deviations exceeding the safety threshold.

[0047] The operation behavior identification and risk assessment method is used to identify irregular operations in the operation behavior sequence data and assess their risk levels.

[0048] First, a temporal behavior evaluation model based on the long short-term memory network (LSTM) is constructed. The input layer of the model segments the operation behavior sequence data into time windows with a duration of two seconds and an overlap rate of 50% between adjacent windows. For the operation behavior sequence data in each time window, the relative position relationship, movement speed, acceleration and joint angle change rate of the key nodes are extracted to construct a multi-dimensional feature vector. For example, assuming that a time window contains 10 key nodes, the distance, direction, speed difference, acceleration difference and joint angle change rate difference of each key node relative to the other 9 key nodes can be extracted to form a high-dimensional feature vector.

[0049] Then, three different sizes of convolution kernels (1x1, 3x3, 5x5) are used to process these multi-dimensional feature vectors in parallel. The convolution operation can extract local features of different scales, for example, 1x1 convolution can extract point features, 3x3 convolution can extract local texture features, and 5x5 convolution can extract structural features of a larger range. After that, the features of different scales are adaptively fused through the channel attention mechanism. The channel attention mechanism can assign different weights according to the importance of different channel features, so as to better fuse information of different scales and obtain more representative temporal features. For example, if the features of a certain channel are more important in identifying irregular operations, the channel will be given a larger weight.

[0050] Next, the fused time series features are input into the bidirectional LSTM network. The bidirectional LSTM network contains a forward LSTM network and a reverse LSTM network, which can capture the forward and backward dependencies in the sequence data at the same time. Each LSTM network unit contains an input gate, a forget gate, and an output gate, and introduces residual connections and layer normalization mechanisms to improve the training efficiency and stability of the model. Residual connections can help gradients propagate better and avoid gradient disappearance or explosion; layer normalization can normalize the input of each time step and accelerate model convergence. Through the processing of the bidirectional LSTM network, the hidden state representation of each time step can be obtained.

[0051] Next, the importance weights of different time steps are calculated based on the temporal attention mechanism. The temporal attention mechanism can learn the importance of different time steps based on the hidden state representations of different time steps. For example, if the hidden state representation of a time step contains more information about irregular operations, the time step will be given a larger weight. Based on these importance weights, irregular operation fragments in the operation behavior sequence data are identified. For example, if the weights of multiple time steps in a certain time period are relatively high, then the time period is likely to correspond to an irregular operation fragment.

[0052] Finally, for the identified irregular operation segments, the risk level coefficient corresponding to each irregular operation segment is calculated in combination with the preset dangerous goods attribute information database. The dangerous goods attribute information database contains the characteristic parameters of various dangerous goods, such as flammability, explosiveness, toxicity, etc. The position deviation, speed deviation, and posture deviation characteristics of the irregular operation segments are multi-dimensionally mapped with the dangerous characteristic parameters to construct a risk assessment matrix. For example, if an irregular operation segment occurs in an area close to flammable goods, its risk level will be higher. The hierarchical analysis method is used to determine the weight coefficients of different deviation dimensions. For example, if the posture deviation has a greater impact on safety, the posture deviation will be given a greater weight. For deviations exceeding the safety threshold, an exponentially growing nonlinear mapping function is used to calculate the risk level coefficient. For example, if a deviation value exceeds twice the safety threshold, its risk level coefficient will grow exponentially. Assume that when an operator is carrying flammable liquid, his operating speed exceeds the safety threshold and the posture deviation also exceeds the safety threshold. According to the dangerous goods attribute information database, the dangerous characteristic parameter of flammable liquid is highly flammable. Through multi-dimensional mapping and hierarchical analysis, the weight coefficients of speed deviation and posture deviation can be determined, and the risk level coefficient of the irregular operation segment can be calculated.

[0053] The solution of this application can: Improve recognition accuracy: By combining multi-scale convolution, channel attention mechanism, bidirectional LSTM network and temporal attention mechanism, temporal features can be extracted and fused more effectively, thereby improving the accuracy of irregular operation recognition. Accurately assess risk level: By combining the dangerous goods attribute information library and multi-dimensional mapping, the risk level of irregular operations can be more accurately assessed, providing a more reliable basis for safety management. Enhance model robustness: By introducing residual connection and layer normalization mechanism, the training efficiency and stability of the model can be improved, and the robustness of the model can be enhanced.

[0054] In an optional implementation, based on the multi-dimensional evaluation report, a preset correction strategy library is called to generate targeted operation behavior improvement suggestions, wherein the operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; and the operation behavior improvement suggestions are pushed to the operator through the laboratory terminal device, including: Based on the risk features in the multidimensional assessment report, a feature vector is constructed; based on the knowledge graph retrieval algorithm, the similarity between the feature vector and the strategy nodes in the preset deviation correction strategy library is calculated, and multiple strategies with the highest similarity are selected to form a candidate strategy set; fuzzy reasoning is used to perform environmental constraint assessment, risk assessment, and operator capability assessment on the candidate strategy set, and the adaptability score of each strategy is obtained through weighted fusion; The strategy with the highest adaptability score in the preset correction strategy library is called to generate operational behavior improvement suggestions, and the strategy is converted into a text description containing specific parameters and operational points; a standard action sequence matching the current operation type is retrieved from the action template library, and a standard action video is generated through posture alignment and trajectory smoothing; a motion trajectory is constructed based on the key frames of the standard action sequence, and the motion trajectory is mapped to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model; the operational behavior improvement suggestions include the text description, the standard action video and the three-dimensional action demonstration model.

[0055] In order to improve the safety and efficiency of operators in complex environments, this embodiment provides a method and system for generating operation behavior improvement suggestions based on a multi-dimensional evaluation report. The method can provide personalized guidance based on the actual operation of the operator, help the operator correct wrong operations, and improve operating skills.

[0056] First, obtain the operator's operation data, conduct a multi-dimensional assessment, and generate a multi-dimensional assessment report. For example, through the sensor to collect the operator's position, posture, force / torque and other data, combined with the preset operation specifications, evaluate the operator's operation safety, efficiency, fluency and other indicators, and generate a multi-dimensional assessment report containing risk characteristics. Assume that the multi-dimensional assessment report contains risk characteristics such as "excessive posture deviation" and "excessive operation speed".

[0057] Then, based on the risk features in the multi-dimensional assessment report, a feature vector is constructed. For example, "excessive posture deviation" and "excessive operation speed" are quantified as 0.8 and 0.6 respectively, forming a feature vector [0.8, 0.6].

[0058] Next, the knowledge graph retrieval algorithm is used to calculate the similarity between the feature vector and the strategy node in the preset correction strategy library. The preset correction strategy library stores correction strategies for various operation scenarios, and each strategy node contains a corresponding feature vector. For example, the feature vector of strategy A "reduce the operation speed and maintain a stable posture" is [0.9, 0.5], and the feature vector of strategy B "adjust the posture and control the operation rhythm" is [0.7, 0.7]. By calculating the similarity between the feature vector [0.8, 0.6] and strategy A and strategy B, for example, the similarities are 0.92 and 0.85 respectively, and multiple strategies with the highest similarity are selected to form a candidate strategy set, such as selecting strategies A and B with the highest similarity.

[0059] Next, fuzzy reasoning is used to evaluate the candidate strategy set. Considering factors such as environmental constraints, danger level, and operator capabilities, each strategy is evaluated separately. For example, assuming that the current environment is small, the environmental constraint evaluation score of strategy A, "reducing the operating speed and maintaining a stable posture", is higher, while strategy B, "adjusting the posture and controlling the operating rhythm", requires a larger operating space and has a lower score. In terms of danger level assessment, assuming that strategy A has a lower danger level, the score is 0.9, and strategy B has a higher danger level, the score is 0.7. In terms of operator capability assessment, assuming that the operator is better at controlling speed, the operator capability assessment score of strategy A is higher. By weighted fusion of environmental constraint evaluation, danger level evaluation, and operator capability evaluation scores, the adaptability score of each strategy is obtained. For example, the adaptability score of strategy A is 0.88, and the adaptability score of strategy B is 0.75.

[0060] After that, the strategy with the highest adaptability score is selected to generate suggestions for improving the operation behavior. For example, strategy A, "Reduce the operation speed and maintain a stable posture", is selected. The strategy is converted into a text description containing specific parameters and operation points, such as "Reduce the operation speed by 20% and keep the wrist joint angle within 15 degrees".

[0061] At the same time, the standard action sequence matching the current operation type is retrieved from the action template library. For example, the standard action sequence of "grasping an object" is retrieved. The standard action video is generated through posture alignment and trajectory smoothing.

[0062] In addition, the motion trajectory is constructed based on the key frames of the standard action sequence, and the motion trajectory is mapped to the human skeleton model through the inverse kinematics algorithm to generate a three-dimensional action demonstration model.

[0063] Finally, the generated text description, standard action video and three-dimensional action demonstration model are pushed to the operator through the laboratory terminal device.

[0064] The solution of this application can: Improve operational safety: Through the assessment of operational risks and personalized correction suggestions, the operational error rate is effectively reduced, potential dangers are avoided, and the safety of operators is guaranteed. Improve operational efficiency: By optimizing the operating process and providing standardized guidance, operators are helped to master the best operating practices, shorten the operating time, and improve work efficiency. Accelerate skill improvement: Through 3D action demonstrations and standard action videos, the correct operating methods are intuitively displayed to help operators quickly understand and master operating skills and accelerate the learning process.

[0065] In an optional implementation, constructing a motion trajectory based on the key frames of the standard motion sequence, and mapping the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional motion demonstration model includes: Calculating the optical flow field between adjacent frames in the standard action sequence, and constructing a motion energy function according to the amplitude change and direction change of the optical flow field; performing peak detection on the motion energy function based on an adaptive threshold, and marking the frame corresponding to the peak as a key frame; performing spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a standardized key frame; The key point positions in the standardized key frame are interpolated using a cubic spline interpolation function to generate an initial motion trajectory; joint angle constraints and speed constraints are constructed according to human kinematic characteristics, and the joint angle constraints and speed constraints are applied to the initial motion trajectory as boundary conditions; the initial motion trajectory is smoothed with minimizing acceleration as the optimization goal to obtain a smooth motion trajectory; the interpolation point density is adjusted based on the time information of the key frame to adjust the speed of the smooth motion trajectory; The human skeleton structure is decomposed into multiple kinematic chains, and priorities are assigned to each kinematic chain based on the importance of the end effector; the joint angles are directly calculated for the kinematic chains with analytical solutions, and the joint angles are iteratively solved using the least squares method with a damping factor for the kinematic chains without analytical solutions; a joint comfort evaluation function is constructed to calculate the deviation between the joint angle and the natural posture, and the deviation is added as a soft constraint to the optimization target; the error change rate during the iteration process is monitored, and the damping factor is dynamically adjusted according to the error change rate; The variation range of the joint angle between adjacent frames is detected to identify the joint angle causing the jump; the trajectory of the identified jump joint angle is replanned using the minimum acceleration criterion to achieve a smooth transition; the optimized joint angle sequence is applied to the human skeleton model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0066] A method for constructing motion trajectories based on key frames and generating three-dimensional action demonstration models. The core of the method is to generate smooth and natural motion trajectories by analyzing the key frames of standard action sequences and mapping them to the human skeleton model.

[0067] First, start with a video or image sequence of a standard action sequence. In order to capture the dynamic characteristics of the action, the optical flow field between adjacent frames is calculated. The optical flow field describes the direction and speed of movement of image pixels in the time dimension. By analyzing the amplitude and direction changes of the optical flow field, a motion energy function can be constructed. The higher the value of this function, the greater the motion intensity of the frame. For example, if the pixel displacement in a certain area is large in two adjacent frames, the amplitude of the optical flow field in this area is large, and the corresponding motion energy function value is also high.

[0068] Next, the motion energy function is peak detected. An adaptive threshold is set, and the frames corresponding to the peaks above the threshold are marked as key frames. This threshold can be dynamically adjusted according to the motion energy distribution of the entire action sequence to ensure that the selected key frames can accurately reflect the key moments of the action. For example, if the entire action is relatively gentle, the threshold can be set lower; conversely, if the action is more intense, the threshold can be set higher. Suppose there is an action sequence containing 100 frames. After peak detection, the 10th, 30th, 60th and 90th frames are marked as key frames.

[0069] In order to eliminate the influence of shooting angle and distance on keyframes, keyframes need to be spatially normalized. This can be achieved by transforming the coordinates of key points (such as human joints) in the keyframes. Suppose the coordinates of a joint in the original keyframe are (10,20), and after spatial normalization, their coordinates become (0.5,0.8).

[0070] The key point positions in the standardized keyframes are interpolated using the cubic spline interpolation function to generate the initial motion trajectory. Cubic spline interpolation can ensure that the generated trajectory has good smoothness. For example, assuming that 20 interpolation frames need to be inserted between the 10th frame and the 30th frame, the coordinates of each key point in these 20 interpolation frames can be calculated by cubic spline interpolation.

[0071] In order to make the generated motion trajectory conform to the kinematic characteristics of the human body, joint angle constraints and velocity constraints need to be imposed. These constraints act as boundary conditions to limit the range of variation of joint angles and velocities. For example, the angle of the human elbow joint cannot exceed 180 degrees, and the velocity cannot be infinite. Applying these constraints to the initial motion trajectory can avoid generating unreasonable motion.

[0072] With the minimization of acceleration as the optimization goal, the initial motion trajectory is smoothed to obtain a smooth motion trajectory. This can be achieved through an optimization algorithm (such as gradient descent). Minimizing acceleration can make the generated motion trajectory more natural and smooth.

[0073] The interpolation point density is adjusted according to the time information of the keyframes to adjust the speed of the smooth motion trajectory. For example, if the time interval between two keyframes is short, the interpolation point density should be high to reflect the rapid change of the action; conversely, if the time interval is long, the interpolation point density should be low.

[0074] The human skeleton structure is decomposed into multiple kinematic chains, and each kinematic chain is prioritized according to the importance of the end effector (such as hands and feet). For example, in the action of grasping an object, the kinematic chain of the hand has a higher priority than the kinematic chain of the leg.

[0075] For kinematic chains with analytical solutions, the joint angles can be directly calculated. For kinematic chains without analytical solutions, the joint angles are iteratively solved using the least squares method with a damping factor. The damping factor can control the convergence speed and stability of the iterative process.

[0076] Construct a joint comfort evaluation function, calculate the deviation between the joint angle and the natural posture, and add the deviation as a soft constraint to the optimization objective. This can make the generated motion more ergonomic and avoid unnatural postures.

[0077] During the iterative solution process, the error change rate is monitored and the damping factor is dynamically adjusted according to the error change rate. This can improve the iteration efficiency and avoid falling into the local optimal solution.

[0078] Detect the change amplitude of joint angles between adjacent frames and identify the joint angles that cause the jump. Use the minimum acceleration criterion to replan the trajectory of the identified jump joint angles to achieve a smooth transition.

[0079] Finally, the optimized joint angle sequence is applied to the human skeleton model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0080] The solution of this application can: Improve the realism and naturalness of the action: By considering factors such as human kinematics and joint comfort, the generated motion trajectory is more in line with the laws of human motion, avoiding mechanical and stiff feelings. Simplify the motion capture process: Only a small number of keyframes are needed to generate a complete motion trajectory, reducing the workload and cost of motion capture. Enhance the editability of the action: The generated motion trajectory can be easily modified by adjusting the position, time and other parameters of the keyframes, improving the flexibility of action design.

[0081] The present application also provides a specific embodiment: The system of the present application may also include an environmental monitoring unit, which is composed of a temperature sensor, a humidity sensor, a gas concentration sensor and a visual recognition camera. The temperature sensor detects the laboratory temperature in real time, with a measurement range of -30°C to 70°C and an accuracy of ±0.5°C; the humidity sensor detects the relative humidity, with a measurement range of 0-100%RH and an accuracy of ±3%RH; the gas concentration sensor detects the content of harmful gases such as nitrogen and carbon monoxide, with a nitrogen detection range of 0-50% and a carbon monoxide detection range of 0-1000ppm; the visual recognition camera uses a high-definition camera with a deep learning algorithm to identify the operator's wearing of protective equipment and the standardization of operations.

[0082] The intelligent storage unit includes ordinary reagent cabinets and explosion-proof reagent cabinets. Ordinary reagent cabinets are used to store general reagents with good stability, and explosion-proof reagent cabinets are used to store flammable and explosive reagents. Each reagent cabinet is equipped with an electronic scale and a radio frequency identification card reader. The electronic scale has an accuracy of 0.1g and is used to accurately record the amount of reagents taken. The reagent cabinet door is equipped with an electromagnetic lock and a mechanical lock, and only authorized personnel can open it.

[0083] Warehouse management: Enter the basic information of hazardous chemicals through the system, including name, CAS number, purity, specification, production date, warranty period, etc. The system automatically assigns storage locations according to the properties of hazardous chemicals. For example, flammables are assigned to explosion-proof cabinets, and strong acids and alkalis are assigned to corrosion-resistant cabinets.

[0084] First, import the test standard method library used in the laboratory. When the sample is registered in the library, the system automatically associates the required reagent list and dosage standards according to the test items. When the inspector takes the reagent, the system displays the types of reagents that can be taken and records the actual amount taken through the electronic scale. After use, weigh and return it, and the system automatically calculates whether the actual amount used is within a reasonable range.

[0085] R&D personnel can choose the reagents they need. The system records the reagent usage, establishes a usage database, and issues an early warning when the single usage amount is abnormal. It also tracks the reagent inventory and automatically reminds you to restock when the inventory is below the threshold.

[0086] The solution of this application can: Safety effect: achieve full traceability of hazardous chemicals and reduce safety hazards; intelligent environmental monitoring and timely warning to ensure storage safety; permission management to prevent unauthorized use. Management effect: digitalization replaces traditional paper records to improve management efficiency; intelligent warning to avoid excessive use of reagents and inventory backlogs; data analysis assists decision-making and optimizes procurement plans. Economic effect: reduce manual record workload and reduce labor costs; optimize inventory management and reduce stagnant materials; improve reagent use efficiency and reduce waste.

[0087] Figure 2 FIG. 1 is a schematic diagram of a management system for hazardous chemicals in a laboratory according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to obtain the dangerous goods operation video data of the operator through the image acquisition devices deployed in multiple areas of the laboratory, and the image acquisition devices include infrared thermal imaging cameras and depth cameras; input the operation video data into a pre-trained human posture recognition model to extract the key node coordinate information and skeleton motion trajectory information of the operator; based on a preset dangerous goods operation specification database, the key node coordinate information and skeleton motion trajectory information are mapped into standardized operation behavior sequence data; The second unit is used to input the operation behavior sequence data into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, in combination with a preset dangerous goods attribute information library, a risk level coefficient corresponding to each irregular operation is calculated; based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation segments, a multidimensional evaluation report including operation time, location, action type and risk score is generated; The third unit is used to call a preset correction strategy library according to the multi-dimensional evaluation report, generate targeted operation behavior improvement suggestions, and the operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; push the operation behavior improvement suggestions to operators through laboratory terminal devices; monitor the operation behavior of operators after executing the improvement suggestions, and update the risk score. When the risk score is lower than the preset threshold for three consecutive times, the corresponding operation behavior sequence data is added to the standard operation sample library; based on the standard operation sample library, the parameters of the timing behavior evaluation model are regularly updated to achieve dynamic optimization of the evaluation criteria.

[0088] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0089] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0090] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The management method of hazardous chemicals in the laboratory is characterized by: include: Acquire the operator's dangerous goods operation video data through image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include infrared thermal imaging cameras and depth cameras; Inputting the operation video data into a pre-trained human posture recognition model to extract key node coordinate information and skeletal motion trajectory information of the operator; Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data; The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, the risk level coefficient corresponding to each irregular operation is calculated in combination with a preset dangerous goods attribute information library; Based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation fragment, a multidimensional assessment report including operation time, location, action type and risk score is generated; Based on the multi-dimensional evaluation report, a preset correction strategy library is called to generate targeted operation behavior improvement suggestions, which include text descriptions, standard action videos, and three-dimensional action demonstration models; and the operation behavior improvement suggestions are pushed to operators through laboratory terminal devices; Monitor the operator's operation behavior after implementing the improvement suggestions, update the risk score, and add the corresponding operation behavior sequence data to the standard operation sample library when the risk score is lower than the preset threshold for three consecutive times; The parameters of the timing behavior evaluation model are regularly updated based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria.

2. The method according to claim 1, characterized in that Inputting the operation video data into a pre-trained human posture recognition model, extracting the key node coordinate information and skeletal motion trajectory information of the operator includes: The operation video data is input into a pre-trained human posture recognition model, the human posture recognition model uses a two-stream feature extraction network to process RGB image sequences and depth image sequences respectively, the RGB image sequence is subjected to a ResNet-50 network to extract the operator's appearance features, and the depth image sequence is subjected to a PointNet++ network to process three-dimensional point cloud data to obtain the operator's spatial position information; based on the attention mechanism, the feature channel importance scores of the appearance features and the spatial position information are calculated, and the performance features and the spatial position information are weightedly fused according to the importance scores to obtain fused features; Input the fused features into the spatiotemporal convolution module, perform key node detection on the features output by the spatiotemporal convolution module through a multi-scale feature pyramid, and extract key node coordinate information in combination with a position sensitive loss function; A skeleton connection graph is constructed based on the key node coordinate information, the skeleton node sequence is processed by a graph convolutional network to learn the motion correlation between nodes, and the predicted trajectory of the skeleton node sequence is smoothed by Kalman filtering to obtain the skeleton motion trajectory information.

3. The method according to claim 1, characterized in that Based on the preset dangerous goods operation specification database, mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data includes: Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data, and the standard actions in the dangerous goods operation specification database are decomposed into basic action units, wherein the basic action units include the relative position relationship of key nodes and motion trajectory characteristics; A dynamic time warping algorithm is used to match and align the skeletal motion trajectory information with the basic action unit; an adaptive scale transformation module is used to normalize the key node coordinate information; Based on the probabilistic graphical model, a state transfer matrix containing timing constraints and spatial constraints is established, and the optimal state sequence is solved by the Viterbi algorithm in combination with the type of dangerous goods and the operating environment information; the optimal state sequence is converted into the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit timing relationship, normalized coordinates and motion characteristics.

4. The method according to claim 1, characterized in that: Inputting the operation behavior sequence data into a pre-trained temporal behavior evaluation model, wherein the temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; For the irregular operation fragments, combined with the preset dangerous goods attribute information database, the risk level coefficient corresponding to each irregular operation is calculated, including: The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network, the input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to time windows with a duration of two seconds and an adjacent window overlap rate of fifty percent, and extracts the relative position relationship of key nodes, movement speed, acceleration, and joint angle change rate of the operation behavior sequence data in each time window to construct a multidimensional feature vector; three types of convolution kernels, namely one by one, three by three, and five by five, are used to process the multidimensional feature vector in parallel, and temporal features are obtained by adaptively fusing features of different scales through a channel attention mechanism; Inputting the temporal features into a bidirectional long short-term memory network, wherein the bidirectional long short-term memory network includes a forward long short-term memory network and a reverse long short-term memory network, wherein each network unit of the forward long short-term memory network and the reverse long short-term memory network includes an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism; calculating the importance weights of different time steps based on a temporal attention mechanism, and identifying irregular operation fragments in the operation behavior sequence data according to the importance weights; For the irregular operation fragments, combined with the preset hazardous goods attribute information library, the risk level coefficient corresponding to each irregular operation fragment is calculated, the hazardous characteristic parameters are extracted from the hazardous goods attribute information library, and the position deviation, speed deviation, posture deviation characteristics of the irregular operation fragments are multi-dimensionally mapped with the hazardous characteristic parameters to construct a risk assessment matrix; the hierarchical analysis method is used to determine the weight coefficients of different deviation dimensions, and the risk level coefficient is calculated using an exponentially growing nonlinear mapping function for deviations exceeding the safety threshold.

5. The method according to claim 1, characterized in that According to the multi-dimensional evaluation report, a preset correction strategy library is called to generate targeted operation behavior improvement suggestions, wherein the operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; The operation behavior improvement suggestions pushed to the operator through the laboratory terminal device include: Based on the risk features in the multidimensional assessment report, a feature vector is constructed; based on the knowledge graph retrieval algorithm, the similarity between the feature vector and the strategy nodes in the preset deviation correction strategy library is calculated, and multiple strategies with the highest similarity are selected to form a candidate strategy set; fuzzy reasoning is used to perform environmental constraint assessment, risk assessment, and operator capability assessment on the candidate strategy set, and the adaptability score of each strategy is obtained through weighted fusion; The strategy with the highest adaptability score in the preset correction strategy library is called to generate operational behavior improvement suggestions, and the strategy is converted into a text description containing specific parameters and operational points; a standard action sequence matching the current operation type is retrieved from the action template library, and a standard action video is generated through posture alignment and trajectory smoothing; a motion trajectory is constructed based on the key frames of the standard action sequence, and the motion trajectory is mapped to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model; the operational behavior improvement suggestions include the text description, the standard action video and the three-dimensional action demonstration model.

6. The method according to claim 5, characterized in that Constructing a motion trajectory based on the key frames of the standard motion sequence, and mapping the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional motion demonstration model includes: Calculating the optical flow field between adjacent frames in the standard action sequence, and constructing a motion energy function according to the amplitude change and direction change of the optical flow field; performing peak detection on the motion energy function based on an adaptive threshold, and marking the frame corresponding to the peak as a key frame; performing spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a standardized key frame; The key point positions in the standardized key frame are interpolated using a cubic spline interpolation function to generate an initial motion trajectory; joint angle constraints and speed constraints are constructed according to human kinematic characteristics, and the joint angle constraints and speed constraints are applied to the initial motion trajectory as boundary conditions; the initial motion trajectory is smoothed with minimizing acceleration as the optimization goal to obtain a smooth motion trajectory; the interpolation point density is adjusted based on the time information of the key frame to adjust the speed of the smooth motion trajectory; The human skeleton structure is decomposed into multiple kinematic chains, and priorities are assigned to each kinematic chain based on the importance of the end effector; the joint angles are directly calculated for the kinematic chains with analytical solutions, and the joint angles are iteratively solved using the least squares method with a damping factor for the kinematic chains without analytical solutions; a joint comfort evaluation function is constructed to calculate the deviation between the joint angle and the natural posture, and the deviation is added as a soft constraint to the optimization target; the error change rate during the iteration process is monitored, and the damping factor is dynamically adjusted according to the error change rate; The variation range of the joint angle between adjacent frames is detected to identify the joint angle causing the jump; the trajectory of the identified jump joint angle is replanned using the minimum acceleration criterion to achieve a smooth transition; the optimized joint angle sequence is applied to the human skeleton model to generate a three-dimensional action demonstration model with continuity and naturalness.

7. A management system for hazardous chemicals in a laboratory, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to obtain the operator's dangerous goods operation video data through image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include an infrared thermal imaging camera and a depth camera; input the operation video data into a pre-trained human posture recognition model to extract the operator's key node coordinate information and skeletal motion trajectory information; Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data; The second unit is used to input the operation behavior sequence data into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, combined with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each irregular operation; Based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation fragment, a multidimensional assessment report including operation time, location, action type and risk score is generated; The third unit is used to call a preset correction strategy library according to the multi-dimensional evaluation report, generate targeted operation behavior improvement suggestions, and the operation behavior improvement suggestions include text descriptions, standard action videos and three-dimensional action demonstration models; and push the operation behavior improvement suggestions to the operator through the laboratory terminal device; Monitor the operator's operation behavior after implementing the improvement suggestions, update the risk score, and add the corresponding operation behavior sequence data to the standard operation sample library when the risk score is lower than the preset threshold for three consecutive times; The parameters of the timing behavior evaluation model are regularly updated based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images