Management methods and systems for hazardous chemicals in laboratories

Abstract

The present invention provides a management method and system for hazardous chemicals in a laboratory, which relates to the field of hazardous materials technology, including obtaining the operator's hazardous materials operation video data through an image acquisition device, extracting key node coordinates and skeletal motion trajectory information, and mapping them into standardized operation behavior sequence data. A temporal behavior evaluation model constructed based on a long short-term memory network is used to identify irregular operation fragments, and the risk level coefficient is calculated in combination with a hazardous materials attribute information library to generate a multidimensional evaluation report. According to the evaluation report, the correction strategy library is called to generate improvement suggestions containing text, video and three-dimensional models, and pushed to the operator. By monitoring the operator's behavior after executing the improvement suggestions, the risk score and the temporal behavior evaluation model are dynamically updated, and the dynamic optimization of the evaluation standard is realized, which effectively improves the safety of laboratory hazardous chemicals operations.

Description

Technical Field

[0001] The present invention relates to the technology of dangerous goods, and particularly to a management method and system for hazardous chemicals in a laboratory. Background Art

[0002] Laboratory safety, especially the operation safety of hazardous chemicals, has always been an important concern in scientific research and production activities. Traditional laboratory safety management mainly relies on manual supervision and paper records, which has problems such as low efficiency, strong subjectivity, and difficulty in comprehensive coverage. With the rapid development of technologies such as computer vision and artificial intelligence, more and more research efforts are dedicated to using these technologies to achieve the automation and intelligence of laboratory safety management. For example, some research uses image recognition technology to detect the clothing of laboratory personnel and remind them to wear necessary protective equipment; others try to use sensors to monitor environmental parameters such as temperature, humidity, and gas concentration to prevent potential hazards.

[0003] However, the existing laboratory safety management technologies still have some defects and deficiencies: First, the analysis of operation behaviors is not fine enough, and it is difficult to identify subtle violations. Most of the existing technologies focus on macroscopic scene recognition, such as whether a person is operating in a designated area and whether they are wearing protective equipment, etc., but lack effective analysis means for the specific action norms during the operation process. Second, there is a lack of targeted corrective guidance. Even if a violation is detected, the existing systems often can only give simple warnings and cannot provide specific improvement suggestions, making it difficult to help operators effectively correct mistakes. Finally, the evaluation criteria are difficult to adjust dynamically. Laboratory operation specifications may be updated with new research progress or changes in safety standards, and the existing systems usually have difficulty adapting to these changes flexibly, resulting in lagging evaluation criteria and affecting the effect of safety management. Summary of the Invention

[0004] Embodiments of the present invention provide a management method and system for hazardous chemicals in a laboratory, which can solve the problems in the existing technology.

[0005] In the first aspect of the embodiments of the present invention,

[0006] A management method for hazardous chemicals in a laboratory is provided, including:

[0007] Obtaining video data of the operation of dangerous goods by an operator through image acquisition devices deployed in multiple areas of the laboratory, where the image acquisition devices include an infrared thermal imaging camera and a depth camera; inputting the operation video data into a pre-trained human pose recognition model to extract the key node coordinate information and skeletal motion trajectory information of the operator; based on a preset database of hazardous chemical operation specifications, mapping the key node coordinate information and skeletal motion trajectory information into standardized operation behavior sequence data;

[0008] Input the operation behavior sequence data into a pre-trained temporal behavior evaluation model. The temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data. For the non-standard operation segments, in combination with a preset dangerous goods attribute information database, calculate the risk level coefficient corresponding to each non-standard operation. Based on the risk level coefficient and the spatio-temporal characteristics of the non-standard operation segments, generate a multi-dimensional evaluation report including operation time, location, action type, and risk score.

[0009] According to the multi-dimensional evaluation report, call a preset correction strategy library to generate targeted operation behavior improvement suggestions. The operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models. Push the operation behavior improvement suggestions to the operator through the laboratory terminal device. Monitor the operation behavior of the operator after implementing the improvement suggestions, update the risk score, and when the risk score is lower than the preset threshold for three consecutive times, add the corresponding operation behavior sequence data to the standard operation sample library. Regularly update the parameters of the temporal behavior evaluation model based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria.

[0010] Input the operation video data into a pre-trained human pose recognition model to extract the key node coordinate information and skeletal motion trajectory information of the operator, including:

[0011] Input the operation video data into a pre-trained human pose recognition model. The human pose recognition model uses a two-stream feature extraction network to process the RGB image sequence and the depth image sequence respectively. The RGB image sequence is processed by a ResNet-50 network to extract the performance features of the operator, and the depth image sequence is processed by a PointNet++ network to process the three-dimensional point cloud data to obtain the spatial position information of the operator. Calculate the feature channel importance scores of the performance features and the spatial position information based on the attention mechanism, and perform weighted fusion on the performance features and the spatial position information according to the importance scores to obtain the fused features.

[0012] Input the fused features into a spatio-temporal convolution module, detect key nodes through a multi-scale feature pyramid for the features output by the spatio-temporal convolution module, and extract the key node coordinate information in combination with a position-sensitive loss function.

[0013] Construct a skeletal connection graph based on the key node coordinate information, process the skeletal node sequence through a graph convolutional network to learn the motion correlation between nodes, and use Kalman filtering to smooth the predicted trajectory of the skeletal node sequence to obtain the skeletal motion trajectory information.

[0014] Mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data based on a preset dangerous goods operation specification database includes:

[0015] Based on a preset dangerous goods operation specification database, mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data, decomposing the standard actions in the dangerous goods operation specification database into basic action units, and the basic action units include the relative position relationship of key nodes and motion trajectory characteristics;

[0016] Using the dynamic time warping algorithm to match and align the skeletal motion trajectory information with the basic action units; performing normalization processing on the key node coordinate information through an adaptive scale transformation module;

[0017] Based on a probabilistic graph model, establishing a state transition matrix including temporal constraints and spatial constraints, combining dangerous goods type and operation environment information, and solving the optimal state sequence through the Viterbi algorithm; generating the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit temporal relationship, normalized coordinates and motion characteristics for the optimal state sequence.

[0018] Inputting the operation behavior sequence data into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data; for the non-standard operation segments, combining a preset dangerous goods attribute information library, calculating the risk level coefficient corresponding to each non-standard operation includes:

[0019] Inputting the operation behavior sequence data into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network, the input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to a time window with a duration of two seconds and an adjacent window overlap rate of 50%, and extracts the relative position relationship of key nodes, motion speed, acceleration, and joint angle change rate of the operation behavior sequence data within each time window to construct a multi-dimensional feature vector; using 1×1, 3×3, and 5×5 three convolution kernels to process the multi-dimensional feature vector in parallel, and adaptively fusing different-scale features through a channel attention mechanism to obtain temporal features;

[0020] Input the temporal features into a bidirectional long short-term memory network, which includes a forward long short-term memory network and a backward long short-term memory network. Each network unit of the forward long short-term memory network and the backward long short-term memory network contains an input gate, a forget gate, and an output gate, and a residual connection and a layer normalization mechanism are introduced; calculate the importance weights of different time steps based on a temporal attention mechanism, and identify irregular operation segments in the operation behavior sequence data according to the importance weights;

[0021] For the irregular operation segments, combine with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each irregular operation segment, extract dangerous characteristic parameters from the dangerous goods attribute information library, and perform multi-dimensional mapping of the position deviation, speed deviation, and attitude deviation characteristics of the irregular operation segments with the dangerous characteristic parameters to construct a risk assessment matrix; use the analytic hierarchy process to determine the weight coefficients of different deviation dimensions, and use an exponentially growing non-linear mapping function to calculate the risk level coefficient for deviations exceeding the safety threshold.

[0022] According to the multi-dimensional evaluation report, call a preset corrective strategy library to generate targeted operation behavior improvement suggestions, which include text descriptions, standard action videos, and three-dimensional action demonstration models; push the operation behavior improvement suggestions to the operator through a laboratory terminal device, including:

[0023] Based on the risk characteristics in the multi-dimensional evaluation report, construct a feature vector; calculate the similarity between the feature vector and the strategy nodes in the preset corrective strategy library based on a knowledge graph retrieval algorithm, and select multiple strategies with the highest similarity to form a candidate strategy set; use fuzzy inference to evaluate the environmental constraints, danger level, and operator ability of the candidate strategy set, and obtain the adaptability score of each strategy through weighted fusion;

[0024] Call the strategy with the highest adaptability score in the preset corrective strategy library to generate operation behavior improvement suggestions, and convert the strategy into a text description including specific parameters and operation points; retrieve the standard action sequence matching the current operation type from the action template library, and generate a standard action video through pose alignment and trajectory smoothing processing; construct a motion trajectory based on the key frames of the standard action sequence, and map the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model; the operation behavior improvement suggestions include the text description, the standard action video, and the three-dimensional action demonstration model.

[0025] Constructing a motion trajectory based on the key frames of the standard action sequence and mapping the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model includes:

[0026] Calculate the optical flow field between adjacent frames in the standard action sequence, and construct a motion energy function based on the amplitude change and direction change of the optical flow field; perform peak detection on the motion energy function based on an adaptive threshold, and mark the frame corresponding to the peak as a key frame; perform spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a normalized key frame;

[0027] Use a cubic spline interpolation function to interpolate the key point positions in the normalized key frame to generate an initial motion trajectory; construct joint angle constraints and speed constraints according to human kinematic characteristics, and apply the joint angle constraints and the speed constraints as boundary conditions to the initial motion trajectory; perform smoothing processing on the initial motion trajectory with the goal of minimizing acceleration to obtain a smooth motion trajectory; adjust the interpolation point density based on the time information of the key frame to adjust the speed of the smooth motion trajectory;

[0028] Decompose the human bone structure into multiple motion chains, and assign priorities to each of the motion chains based on the importance of the end effector; directly calculate the joint angles for the motion chains with analytical solutions, and use the least squares method with a damping factor to iteratively solve the joint angles for the motion chains without analytical solutions; construct a joint comfort evaluation function to calculate the deviation between the joint angles and the natural posture, and add the deviation as a soft constraint to the optimization goal; monitor the error change rate during the iteration process, and dynamically adjust the size of the damping factor according to the error change rate;

[0029] Detect the change amplitude of the joint angles between adjacent frames, and identify the joint angles that cause jumps; perform trajectory replanning on the identified jumping joint angles using the minimum acceleration criterion to achieve smooth transition; apply the optimized joint angle sequence to the human bone model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0030] In the second aspect of the embodiments of the present invention,

[0031] Provide a management system for hazardous chemicals in a laboratory, including:

[0032] A first unit for obtaining the hazardous chemical operation video data of an operator through image acquisition devices deployed in multiple areas of the laboratory, where the image acquisition devices include an infrared thermal imaging camera and a depth camera; input the operation video data into a pre-trained human posture recognition model to extract the key node coordinate information and bone motion trajectory information of the operator; based on a preset hazardous chemical operation specification database, map the key node coordinate information and bone motion trajectory information into standardized operation behavior sequence data;

[0033] A second unit is configured to input the operation behavior sequence data into a pre-trained temporal behavior evaluation model. The temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data. For each non-standard operation segment, in combination with a preset dangerous goods attribute information database, a risk level coefficient corresponding to each non-standard operation is calculated. Based on the risk level coefficient and the spatio-temporal characteristics of the non-standard operation segment, a multi-dimensional evaluation report including operation time, location, action type, and risk score is generated.

[0034] A third unit is configured to, according to the multi-dimensional evaluation report, call a preset rectification strategy library to generate targeted operation behavior improvement suggestions. The operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models. The operation behavior improvement suggestions are pushed to the operator through a laboratory terminal device. The operation behavior of the operator after executing the improvement suggestions is monitored, and the risk score is updated. When the risk score is lower than a preset threshold for three consecutive times, the corresponding operation behavior sequence data is added to the standard operation sample library. Based on the standard operation sample library, the parameters of the temporal behavior evaluation model are updated regularly to realize dynamic optimization of the evaluation standard.

[0035] In a third aspect of the embodiments of the present invention,

[0036] An electronic device is provided, including:

[0037] A processor;

[0038] A memory for storing instructions executable by the processor;

[0039] Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0040] In a fourth aspect of the embodiments of the present invention,

[0041] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0042] The beneficial effects of this application are as follows:

[0043] 1. Improve the safety of hazardous chemical operations: By real-time monitoring and evaluating the behavior of operators, non-standard operations can be discovered and corrected in a timely manner, effectively reducing the accident risk caused by human errors and ensuring the safety of laboratory personnel and the environment.

[0044] 2. Realize the standardization and normalization of operation behaviors: Using artificial intelligence technology, operation specifications are transformed into quantifiable indicators, and targeted improvement suggestions are provided to help operators learn and master correct operation methods, promoting the standardization and normalization of laboratory operation behaviors.

[0045] 3. Continuous improvement of the evaluation model and operation specifications: By continuously accumulating and analyzing operation data, dynamically update the evaluation model and the standard operation sample library, making the evaluation criteria more in line with the actual operation situation, and achieving the continuous improvement and optimization of the evaluation system. Description of the Drawings

[0046] Figure 1 It is a schematic flowchart of the management method for hazardous chemicals in the laboratory according to the embodiment of the present invention;

[0047] Figure 2 It is a schematic structural diagram of the management system for hazardous chemicals in the laboratory according to the embodiment of the present invention. Detailed Embodiments

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0050] Figure 1 It is a schematic flowchart of the management method for hazardous chemicals in the laboratory according to the embodiment of the present invention, as Figure 1 shown, the method includes:

[0051] S11. Obtain the dangerous goods operation video data of the operator through the image acquisition devices deployed in multiple areas of the laboratory. The image acquisition devices include infrared thermal imaging cameras and depth cameras; input the operation video data into a pre-trained human pose recognition model to extract the key node coordinate information and skeletal motion trajectory information of the operator; based on the preset dangerous goods operation specification database, map the key node coordinate information and skeletal motion trajectory information into standardized operation behavior sequence data;

[0052] S12. Input the operation behavior sequence data into a pre-trained temporal behavior evaluation model. The temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data. For each non-standard operation segment, in combination with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each non-standard operation. Based on the risk level coefficient and the spatio-temporal characteristics of the non-standard operation segment, generate a multi-dimensional evaluation report including operation time, location, action type, and risk score.

[0053] S13. According to the multi-dimensional evaluation report, call a preset correction strategy library to generate targeted operation behavior improvement suggestions. The operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models. Push the operation behavior improvement suggestions to the operator through the laboratory terminal device. Monitor the operation behavior of the operator after implementing the improvement suggestions, update the risk score, and when the risk score is lower than the preset threshold for three consecutive times, add the corresponding operation behavior sequence data to the standard operation sample library. Regularly update the parameters of the temporal behavior evaluation model based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria.

[0054] In an alternative implementation, input the operation video data into a pre-trained human pose recognition model. Extracting the key node coordinate information and skeletal motion trajectory information of the operator includes:

[0055] Input the operation video data into a pre-trained human pose recognition model. The human pose recognition model uses a two-stream feature extraction network to process the RGB image sequence and the depth image sequence respectively. The RGB image sequence is processed by a ResNet-50 network to extract the performance features of the operator, and the depth image sequence is processed by a PointNet++ network to process the three-dimensional point cloud data to obtain the spatial position information of the operator. Calculate the feature channel importance scores of the performance features and the spatial position information based on the attention mechanism, and perform weighted fusion on the performance features and the spatial position information according to the importance scores to obtain the fused features.

[0056] Input the fused features into a spatio-temporal convolution module, detect key nodes through a multi-scale feature pyramid for the features output by the spatio-temporal convolution module, and extract the key node coordinate information in combination with a position-sensitive loss function.

[0057] Construct a skeletal connection graph based on the key node coordinate information, process the skeletal node sequence through a graph convolutional network to learn the motion correlation between nodes, and use Kalman filtering to smooth the predicted trajectory of the skeletal node sequence to obtain the skeletal motion trajectory information.

[0058] In order to achieve accurate recognition of human body postures and extraction of skeletal movement trajectories in operation videos, this embodiment proposes a human body posture recognition method based on a two-stream feature extraction network and spatio-temporal convolution. This method integrates the information of RGB images and depth images, and uses an attention mechanism and a graph convolutional network to effectively improve the posture recognition accuracy and trajectory smoothness.

[0059] First, preprocess the operation video and segment it into an RGB image sequence and a depth image sequence. Taking a video containing 100 frames as an example, it is segmented into 100 RGB images and the corresponding 100 frames of depth image data.

[0060] Then, input the RGB image sequence into a pre-trained ResNet-50 network to extract the performance features of the operator. For example, input each frame of the RGB image into the ResNet-50 network to obtain a 1024-dimensional feature vector, and finally obtain a 100x1024 feature matrix, which represents the performance features of the video.

[0061] At the same time, convert the depth image sequence into three-dimensional point cloud data and input it into a pre-trained PointNet++ network to obtain the spatial position information of the operator. Assuming that each frame of the depth image contains 1000 point cloud data, and each point cloud data contains three-dimensional coordinate information, the output of the PointNet++ network can be a 100x1000x3 tensor, which is processed to obtain a 100x512 feature matrix, representing the spatial position information of the video.

[0062] Next, use the attention mechanism to calculate the feature channel importance scores of the performance features and the spatial position information respectively. For example, for the performance features, the average value or variance of each channel can be calculated as its importance score. Suppose 1024 score values are calculated and these score values are normalized to between 0 and 1. Similar processing is also performed on the spatial position information to obtain 512 normalized score values.

[0063] According to the calculated feature channel importance scores, perform weighted fusion on the performance features and the spatial position information to obtain the fused features. For example, multiply each channel of the performance features by the corresponding score value, multiply each channel of the spatial position information by the corresponding score value, and then add the two to obtain a 100x1536 fused feature matrix.

[0064] Input the fused features into the spatio-temporal convolution module, such as using a 3D convolutional network, to extract spatio-temporal features. Suppose the output of the spatio-temporal convolution module is a 100x256 feature matrix.

[0065] Detect key nodes for the features output by the spatio-temporal convolution module through a multi-scale feature pyramid. For example, convolve the feature map with convolutional kernels of different sizes to obtain feature maps of different scales, and then fuse these feature maps to obtain the final key node detection result. Assuming that 17 key nodes of a human body are detected, the output is a tensor of 100x17x2, representing the two-dimensional coordinates of each key node at each moment.

[0066] Combine with a position-sensitive loss function to further optimize the key node coordinate information and improve the detection accuracy. For example, calculate the loss based on the predicted key node coordinates and the true key node coordinates, and backpropagate to update the network parameters.

[0067] Construct a bone connection graph based on the extracted key node coordinate information. For example, connect the 17 key nodes to form the bone structure of the human body.

[0068] Process the bone node sequence through a graph convolutional network to learn the motion correlation between nodes. For example, input the bone connection graph at each moment into the graph convolutional network to obtain the feature vector of each node, and then combine these feature vectors into a sequence to learn the motion relationship between nodes.

[0069] Use Kalman filtering to smooth the predicted trajectory of the bone node sequence to obtain the final bone motion trajectory information. For example, use Kalman filtering to predict the position of each node at the next moment and smooth the prediction result to remove noise and jitter.

[0070] The solution of this application can:

[0071] Improve the pose recognition accuracy: By fusing the information of RGB images and depth images and using the attention mechanism, it is possible to more comprehensively capture the human pose features, thereby improving the pose recognition accuracy. Enhance the robustness: Utilize the spatio-temporal convolution module and the multi-scale feature pyramid to effectively handle complex scenarios such as occlusion and illumination changes, enhancing the robustness of the algorithm. Improve the trajectory smoothness: Use a graph convolutional network to learn the motion correlation between nodes and combine Kalman filtering for smoothing processing, which can effectively remove the noise and jitter in the bone motion trajectory and improve the trajectory smoothness.

[0072] In an optional implementation manner, based on a preset dangerous goods operation specification database, mapping the key node coordinate information and the bone motion trajectory information into standardized operation behavior sequence data includes:

[0073] Based on a preset dangerous goods operation specification database, map the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data. Decompose the standard actions in the dangerous goods operation specification database into basic action units, and each basic action unit includes the relative position relationship of key nodes and the motion trajectory characteristics.

[0074] Use the dynamic time warping algorithm to match and align the skeletal motion trajectory information with the basic action units; perform normalization processing on the key node coordinate information through an adaptive scale transformation module.

[0075] Based on a probabilistic graph model, establish a state transition matrix containing temporal constraints and spatial constraints. Combine the dangerous goods type and operation environment information, and solve the optimal state sequence through the Viterbi algorithm. Generate the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit temporal relationship, normalized coordinates, and motion characteristics for the optimal state sequence.

[0076] The core of the dangerous goods operation behavior standardization method is to convert the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data. This method is based on a preset dangerous goods operation specification database and is implemented using technologies such as dynamic time warping, adaptive scale transformation, probabilistic graph models, and the Viterbi algorithm.

[0077] First, prepare a preset dangerous goods operation specification database. This database contains operation specifications for various dangerous goods, such as operation steps, key node positions, motion trajectories, and other information. Taking the handling of dangerous chemicals as an example, the database will include standard actions such as "wear protective clothing", "check the container seal", "carry with both hands", and "handle with care". It also details the relative position relationship and motion trajectory characteristics of the key nodes (such as hands, elbows, and containers) for each standard action. For example, for the standard action of "carry with both hands", it will record the relative position relationship between the two hands and the container (such as the two hands surrounding the container), and the motion trajectory of the two hands during the handling process (such as a smooth movement from the starting position to the target position).

[0078] Next, decompose the standard actions in the database into basic action units. Each basic action unit includes the relative position relationship of key nodes and the motion trajectory characteristics. For example, "carry with both hands" can be decomposed into basic action units such as "grab the container", "lift the container", "move the container", and "put down the container". Each unit includes the relative position relationship of the key nodes (both hands and the container) and the motion trajectory characteristics. For instance, for the "grab the container" unit, the relative position relationship of the key nodes is that the two hands surround the container, and the motion trajectory characteristics are the trajectory of the two hands moving from away from the container to approaching the container and finally surrounding the container.

[0079] Then, extract the key node coordinate information and skeletal motion trajectory information from the dangerous goods operation video to be analyzed. For example, obtain the coordinate information of key nodes such as the hands, elbows, and torso of the operator during the handling of dangerous chemicals through motion capture technology, as well as the motion trajectory information of these key nodes.

[0080] Match and align the extracted skeletal motion trajectory information with the motion trajectory features of each basic action unit in the database. Use the dynamic time warping algorithm to calculate the similarity and find the basic action unit that best matches the current skeletal motion trajectory information. For example, compare the motion trajectories of the operator's hands during the handling process with the motion trajectory features of basic action units such as "grab the container" and "lift the container" to find the unit with the highest similarity.

[0081] At the same time, perform normalization processing on the extracted key node coordinate information. Use an adaptive scale transformation module to map the key node coordinate information under different operators and different operating environments to a unified scale range. For example, map the hand coordinate information of operators with different heights to a standard human model through scale transformation.

[0082] Next, establish a state transition matrix based on the probabilistic graph model. This matrix contains temporal constraints and spatial constraints, which are used to describe the transition probabilities between different basic action units. Temporal constraints reflect the chronological order of basic action units. For example, "grab the container" must be before "lift the container". Spatial constraints reflect the positional relationship of basic action units in space. For example, after "lift the container", the position of the container should be higher than the initial position. Combine the dangerous goods type and operating environment information to further refine the state transition matrix. For example, when handling flammable liquids, the probability of the "handle with care" action is higher.

[0083] Use the Viterbi algorithm to solve the optimal state sequence, that is, find the sequence of basic action units that best matches the observed key node coordinate information and skeletal motion trajectory information.

[0084] Finally, generate standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit chronological relationship, normalized coordinates, and motion characteristics. For example, arrange the identified basic action units such as "grab the container", "lift the container", "move the container", and "put down the container" in chronological order, and combine the normalized coordinate information and motion characteristics to generate standardized operation behavior sequence data, such as: {"handling": [{"grab the container": [coordinate information 1, motion characteristic 1]}, {"lift the container": [coordinate information 2, motion characteristic 2]}, {"move the container": [coordinate information 3, motion characteristic 3]}, {"put down the container": [coordinate information 4, motion characteristic 4]}]}.

[0085] The solution of this application can:

[0086] Improve operation standardization: Convert the behaviors of operators into standardized data, which is convenient for comparison with the preset operation specifications, so as to judge whether the operation conforms to the specifications and improve the safety of operations. Improve training efficiency: By analyzing the standardized operation behavior sequence data, non-standard behaviors in the operation process can be identified, providing targeted guidance for the training of operators, thus improving training efficiency. Facilitate automated analysis: The standardized operation behavior sequence data is convenient for computers to perform automated analysis, such as identifying dangerous operations and predicting accident risks, so as to provide support for the intelligent management of dangerous goods operations.

[0087] In an alternative embodiment, the operation behavior sequence data is input into a pre-trained temporal behavior evaluation model. The temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data; for the non-standard operation segments, in combination with a preset dangerous goods attribute information library, calculating the risk level coefficient corresponding to each non-standard operation includes:

[0088] Input the operation behavior sequence data into a pre-trained temporal behavior evaluation model. The temporal behavior evaluation model is constructed based on a long short-term memory network. The input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to time windows with a duration of two seconds and an adjacent window overlap rate of 50%, and extracts the relative positions of key nodes, movement speeds, accelerations, and joint angle change rates of the operation behavior sequence data within each time window to construct a multi-dimensional feature vector; Use three types of convolutional kernels of 1×1, 3×3, and 5×5 to process the multi-dimensional feature vector in parallel, and adaptively fuse features of different scales through a channel attention mechanism to obtain temporal features;

[0089] Input the temporal features into a bidirectional long short-term memory network. The bidirectional long short-term memory network includes a forward long short-term memory network and a backward long short-term memory network. Each network unit of the forward long short-term memory network and the backward long short-term memory network contains an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism; Calculate the importance weights of different time steps based on a temporal attention mechanism, and identify non-standard operation segments in the operation behavior sequence data according to the importance weights;

[0090] For the non-standard operation segments, in combination with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each non-standard operation segment, extract the dangerous characteristic parameters from the dangerous goods attribute information library, and construct a risk assessment matrix by performing multi-dimensional mapping of the position deviation, speed deviation, and attitude deviation characteristics of the non-standard operation segments and the dangerous characteristic parameters; use the analytic hierarchy process to determine the weight coefficients of different deviation dimensions, and use a non-linear mapping function with exponential growth to calculate the risk level coefficient for deviations exceeding the safety threshold.

[0091] An operation behavior recognition and risk assessment method for identifying non-standard operations in operation behavior sequence data and assessing their risk levels.

[0092] First, construct a temporal behavior assessment model based on a long short-term memory network (LSTM). The input layer of this model segments the operation behavior sequence data into time windows with a duration of two seconds and an adjacent window overlap rate of 50%. For the operation behavior sequence data within each time window, extract the relative position relationships, motion speeds, accelerations, and joint angle change rates of the key nodes to construct a multi-dimensional feature vector. For example, assuming that a time window contains 10 key nodes, information such as the distance, direction, speed difference, acceleration difference, and joint angle change rate difference of each key node relative to the other 9 key nodes can be extracted to form a high-dimensional feature vector.

[0093] Then, use three different sizes of convolutional kernels (1x1, 3x3, 5x5) to process these multi-dimensional feature vectors in parallel. The convolution operation can extract local features of different scales. For example, the 1x1 convolution can extract point features, the 3x3 convolution can extract local texture features, and the 5x5 convolution can extract larger-scale structural features. Then, perform adaptive fusion on the features of different scales through a channel attention mechanism. The channel attention mechanism can assign different weights according to the importance of different channel features, so as to better fuse information of different scales and obtain more representative temporal features. For example, if the features of a certain channel are more important in identifying non-standard operations, a larger weight will be assigned to this channel.

[0094] Next, input the fused temporal features into a bidirectional LSTM network. The bidirectional LSTM network includes a forward LSTM network and a backward LSTM network, which can capture the forward and backward dependencies in the sequence data simultaneously. Each LSTM network unit includes an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism to improve the training efficiency and stability of the model. The residual connection can help the gradient propagate better and avoid gradient vanishing or explosion; the layer normalization can normalize the input at each time step to accelerate the convergence of the model. Through the processing of the bidirectional LSTM network, the hidden state representation at each time step can be obtained.

[0095] Next, based on the temporal attention mechanism, calculate the importance weights for different time steps. The temporal attention mechanism can learn the importance of different time steps according to the hidden state representations at different time steps. For example, if the hidden state representation at a certain time step contains more information about non-standard operations, then a greater weight will be assigned to that time step. Based on these importance weights, identify the non-standard operation segments in the operation behavior sequence data. For example, if the weights of multiple time steps within a certain time period are relatively high, then that time period is likely to correspond to a non-standard operation segment.

[0096] Finally, for the identified non-standard operation segments, in combination with the preset dangerous goods attribute information database, calculate the risk level coefficients corresponding to each non-standard operation segment. The dangerous goods attribute information database contains the characteristic parameters of various dangerous goods, such as flammability, explosiveness, toxicity, etc. Perform multi-dimensional mapping of the position deviation, speed deviation, and attitude deviation features of the non-standard operation segments with the dangerous characteristic parameters to construct a risk assessment matrix. For example, if a non-standard operation segment occurs in an area close to flammable goods, its risk level will be higher. Use the analytic hierarchy process to determine the weight coefficients for different deviation dimensions. For example, if the attitude deviation has a greater impact on safety, then a greater weight will be assigned to the attitude deviation. For deviations exceeding the safety threshold, use an exponential growth non-linear mapping function to calculate the risk level coefficients. For example, if a certain deviation value exceeds twice the safety threshold, then its risk level coefficient will increase exponentially. Suppose an operator is handling flammable liquid, and their operation speed exceeds the safety threshold, and the attitude deviation also exceeds the safety threshold. According to the dangerous goods attribute information database, the dangerous characteristic parameter of flammable liquid is highly flammable. Through multi-dimensional mapping and the analytic hierarchy process, the weight coefficients of the speed deviation and attitude deviation can be determined, and the risk level coefficient of this non-standard operation segment can be calculated.

[0097] The solution of this application can:

[0098] Improve the recognition accuracy: By combining multi-scale convolution, channel attention mechanism, bidirectional LSTM network, and temporal attention mechanism, temporal features can be more effectively extracted and fused, thereby improving the accuracy of non-standard operation recognition. Accurately evaluate the risk level: By combining the dangerous goods attribute information database and multi-dimensional mapping, the risk level of non-standard operations can be more accurately evaluated, providing a more reliable basis for safety management. Enhance the robustness of the model: By introducing residual connections and layer normalization mechanisms, the training efficiency and stability of the model can be improved, enhancing the robustness of the model.

[0099] In an alternative embodiment, according to the multi-dimensional evaluation report, a preset deviation correction strategy library is called to generate targeted improvement suggestions for operation behaviors. The improvement suggestions for operation behaviors include text descriptions, standard action videos, and three-dimensional action demonstration models. Pushing the improvement suggestions for operation behaviors to the operator through the laboratory terminal device includes:

[0100] Based on the risk characteristics in the multi-dimensional evaluation report, a feature vector is constructed; based on the knowledge graph retrieval algorithm, the similarity between the feature vector and the policy nodes in the preset deviation correction strategy library is calculated, and multiple strategies with the highest similarity are selected to form a candidate strategy set; fuzzy inference is used to evaluate the environmental constraints, risk levels, and operator capabilities of the candidate strategy set, and the adaptability scores of each strategy are obtained through weighted fusion.

[0101] Call the strategy with the highest adaptability score in the preset deviation correction strategy library to generate improvement suggestions for operation behaviors, and convert the strategy into a text description including specific parameters and operation key points; retrieve the standard action sequence matching the current operation type from the action template library, and generate a standard action video through pose alignment and trajectory smoothing processing; construct a motion trajectory based on the key frames of the standard action sequence, and map the motion trajectory to the human skeleton model through the inverse kinematics algorithm to generate a three-dimensional action demonstration model. The improvement suggestions for operation behaviors include the text description, the standard action video, and the three-dimensional action demonstration model.

[0102] To improve the operation safety and efficiency of the operator in a complex environment, this embodiment provides a method and system for generating improvement suggestions for operation behaviors based on a multi-dimensional evaluation report. This method can provide personalized guidance according to the actual operation situation of the operator, help the operator correct wrong operations, and improve operation skills.

[0103] First, the operation data of the operator is obtained and multi-dimensionally evaluated to generate a multi-dimensional evaluation report. For example, data such as the position, posture, force / moment of the operator is collected through sensors, and combined with the preset operation specifications, indicators such as the operation safety, efficiency, and fluency of the operator are evaluated to generate a multi-dimensional evaluation report including risk characteristics. Assume that the multi-dimensional evaluation report contains risk characteristics such as "excessive posture deviation" and "too fast operation speed".

[0104] Then, based on the risk characteristics in the multi-dimensional evaluation report, a feature vector is constructed. For example, "excessive posture deviation" and "too fast operation speed" are respectively quantified as 0.8 and 0.6 to form a feature vector [0.8, 0.6].

[0105] Next, use the knowledge graph retrieval algorithm to calculate the similarity between the feature vector and the policy nodes in the preset deviation correction policy library. The preset deviation correction policy library stores deviation correction policies for various operation scenarios, and each policy node contains a corresponding feature vector. For example, the feature vector of policy A "reduce the operation speed and maintain a stable posture" is [0.9, 0.5], and the feature vector of policy B "adjust the posture and control the operation rhythm" is [0.7, 0.7]. By calculating the similarity between the feature vector [0.8, 0.6] and policies A and B, for example, similarity scores of 0.92 and 0.85 are obtained respectively, and select multiple policies with the highest similarity to form a candidate policy set. For example, select policies A and B with the highest similarity.

[0106] Next, use fuzzy inference to evaluate the candidate policy set. Consider factors such as environmental constraints, danger level, and operator ability, and evaluate each policy separately. For example, assume that the current environment is narrow, then the environmental constraint evaluation score of policy A "reduce the operation speed and maintain a stable posture" is relatively high, while policy B "adjust the posture and control the operation rhythm" has a lower score because it requires a larger operation space. In terms of danger level evaluation, assume that the danger level of policy A is low, with a score of 0.9, and the danger level of policy B is high, with a score of 0.7. In terms of operator ability evaluation, assume that the operator is more proficient in controlling speed, then the operator ability evaluation score of policy A is relatively high. By weighted fusion of the environmental constraint evaluation, danger level evaluation, and operator ability evaluation scores, the adaptability score of each policy is obtained. For example, the adaptability score of policy A is 0.88, and the adaptability score of policy B is 0.75.

[0107] After that, select the policy with the highest adaptability score to generate suggestions for improving the operation behavior. For example, select policy A "reduce the operation speed and maintain a stable posture". Convert this policy into a text description containing specific parameters and operation points, such as "reduce the operation speed by 20% and keep the wrist joint angle within 15 degrees".

[0108] At the same time, retrieve the standard action sequence that matches the current operation type from the action template library. For example, retrieve the standard action sequence for "grasping an object". Generate a standard action video through pose alignment and trajectory smoothing processing.

[0109] In addition, construct a motion trajectory based on the key frames of the standard action sequence, and map the motion trajectory to the human body bone model through the inverse kinematics algorithm to generate a three-dimensional action demonstration model.

[0110] Finally, push the generated text description, standard action video, and three-dimensional action demonstration model to the operator through the laboratory terminal device.

[0111] The solution of this application can:

[0112] Enhance operation safety: By evaluating operation risks and providing personalized corrective suggestions, effectively reduce the operation error rate, avoid potential dangers, and ensure the safety of operators. Improve operation efficiency: By optimizing operation processes and providing standardized guidance, help operators master the best operation practices, shorten operation time, and improve work efficiency. Accelerate skill improvement: Through three-dimensional motion demonstrations and standard action videos, intuitively display the correct operation methods, help operators quickly understand and master operation skills, and accelerate the learning process.

[0113] In an alternative implementation, constructing a motion trajectory based on the key frames of the standard action sequence and mapping the motion trajectory to a human skeletal model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model includes:

[0114] Calculate the optical flow field between adjacent frames in the standard action sequence, and construct a motion energy function based on the amplitude change and direction change of the optical flow field; perform peak detection on the motion energy function based on an adaptive threshold, and mark the frame corresponding to the peak as a key frame; perform spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a normalized key frame;

[0115] Use a cubic spline interpolation function to interpolate the key point positions in the normalized key frame to generate an initial motion trajectory; construct joint angle constraints and speed constraints according to human kinematic characteristics, and apply the joint angle constraints and the speed constraints as boundary conditions to the initial motion trajectory; perform smoothing processing on the initial motion trajectory with the goal of minimizing acceleration to obtain a smooth motion trajectory; adjust the interpolation point density based on the time information of the key frame to adjust the speed of the smooth motion trajectory;

[0116] Decompose the human skeletal structure into multiple motion chains, and assign priorities to each motion chain based on the importance of the end effector; directly calculate the joint angles for the motion chains with analytical solutions, and use the least squares method with a damping factor to iteratively solve the joint angles for the motion chains without analytical solutions; construct a joint comfort evaluation function to calculate the deviation between the joint angles and the natural posture, and add the deviation as a soft constraint to the optimization goal; monitor the error change rate during the iteration process, and dynamically adjust the size of the damping factor according to the error change rate;

[0117] Detect the change amplitude of the joint angles between adjacent frames, and identify the joint angles that cause jumps; perform trajectory replanning on the identified jumping joint angles using the minimum acceleration criterion to achieve smooth transition; apply the optimized joint angle sequence to the human skeletal model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0118] A method for constructing a motion trajectory based on key frames and generating a three-dimensional action demonstration model, the core of which is to analyze the key frames of a standard action sequence, generate a smooth and natural motion trajectory, and map it onto a human skeletal model.

[0119] First, start with a video or image sequence of a standard action sequence. To capture the dynamic characteristics of the action, calculate the optical flow field between adjacent frames. The optical flow field describes the motion direction and speed of image pixels in the time dimension. By analyzing the amplitude change and direction change of the optical flow field, a motion energy function can be constructed. The higher the value of this function, the greater the motion intensity of the frame. For example, assume that in two adjacent frames, the pixel displacement in a certain area is large, then the amplitude of the optical flow field in this area is large, and the corresponding value of the motion energy function is also high.

[0120] Next, perform peak detection on the motion energy function. Set an adaptive threshold, and the frames corresponding to the peaks higher than the threshold are marked as key frames. This threshold can be dynamically adjusted according to the motion energy distribution of the entire action sequence to ensure that the selected key frames can accurately reflect the critical moments of the action. For example, if the entire action is relatively gentle, the threshold can be set lower; conversely, if the action is relatively intense, the threshold can be set higher. Assume an action sequence containing 100 frames. After peak detection, the 10th, 30th, 60th, and 90th frames are marked as key frames.

[0121] To eliminate the influence of shooting angle and distance on the key frames, spatial normalization processing needs to be performed on the key frames. This can be achieved by transforming the coordinates of the key points (such as human joints) in the key frames. Assume that the coordinate of a certain joint in the original key frame is (10, 20). After spatial normalization processing, its coordinate becomes (0.5, 0.8).

[0122] Use the cubic spline interpolation function to interpolate the positions of the key points in the normalized key frames to generate an initial motion trajectory. Cubic spline interpolation can ensure that the generated trajectory has good smoothness. For example, assume that 20 interpolation frames need to be inserted between the 10th and 30th frames, then the coordinates of each key point in these 20 interpolation frames can be calculated through cubic spline interpolation.

[0123] To make the generated motion trajectory conform to human kinematic characteristics, joint angle constraints and speed constraints need to be imposed. These constraints, as boundary conditions, limit the change range of joint angles and speeds. For example, the angle of the human elbow joint cannot exceed 180 degrees, and the speed cannot be infinite. Imposing these constraints on the initial motion trajectory can avoid generating unreasonable motions.

[0124] Taking the minimization of acceleration as the optimization objective, the initial motion trajectory is smoothed to obtain a smooth motion trajectory. This can be achieved through optimization algorithms (such as the gradient descent method). Minimizing acceleration can make the generated motion trajectory more natural and smooth.

[0125] Adjust the density of interpolation points according to the time information of key frames, and adjust the speed of the smooth motion trajectory. For example, if the time interval between two key frames is short, the density of interpolation points should be high to reflect the rapid changes in the action; conversely, if the time interval is long, the density of interpolation points should be low.

[0126] Decompose the human body bone structure into multiple motion chains, and assign priorities to each motion chain according to the importance of the end effectors (such as hands, feet). For example, in the action of grasping an object, the motion chain of the hand has a higher priority than the motion chain of the leg.

[0127] For motion chains with analytical solutions, the joint angles can be directly calculated. For motion chains without analytical solutions, the least squares method with a damping factor is used to iteratively solve the joint angles. The damping factor can control the convergence speed and stability of the iterative process.

[0128] Construct a joint comfort evaluation function, calculate the deviation between the joint angle and the natural posture, and add the deviation as a soft constraint to the optimization objective. This can make the generated motion more ergonomic and avoid unnatural postures.

[0129] During the iterative solution process, monitor the error change rate, and dynamically adjust the size of the damping factor according to the error change rate. This can improve the iterative efficiency and avoid falling into local optimal solutions.

[0130] Detect the change amplitude of the joint angle between adjacent frames, and identify the joint angles that cause jumps. For the identified jumping joint angles, the trajectory is replanned using the minimum acceleration criterion to achieve smooth transition.

[0131] Finally, apply the optimized joint angle sequence to the human body bone model to generate a three-dimensional action demonstration model with continuity and naturalness.

[0132] The solution of this application can:

[0133] Improve the realism and naturalness of actions: By considering factors such as human kinematic characteristics and joint comfort, the generated motion trajectory is more in line with the human motion law, avoiding mechanical and rigid feelings. Simplify the motion capture process: Only a small number of key frames are needed to generate a complete motion trajectory, reducing the workload and cost of motion capture. Enhance the editability of actions: The generated motion trajectory can be easily modified by adjusting parameters such as the position and time of key frames, improving the flexibility of action design.

[0134] The present application also provides a specific embodiment:

[0135] The system of the present application may further include an environmental monitoring unit, which is composed of a temperature sensor, a humidity sensor, a gas concentration sensor, and a vision recognition camera. The temperature sensor detects the laboratory temperature in real time, with a measurement range of -30°C to 70°C and an accuracy of ±0.5°C; the humidity sensor detects the relative humidity, with a measurement range of 0-100%RH and an accuracy of ±3%RH; the gas concentration sensor detects the content of harmful gases such as nitrogen and carbon monoxide, with a nitrogen detection range of 0-50% and a carbon monoxide detection range of 0-1000 ppm; the vision recognition camera uses a high-definition camera combined with a deep learning algorithm to identify the wearing situation of the operator's protective equipment and the operation standardization.

[0136] The intelligent storage unit includes a general reagent cabinet and an explosion-proof reagent cabinet. The general reagent cabinet is used to store general reagents with better stability, and the explosion-proof reagent cabinet is used to store flammable and explosive reagents. Each reagent cabinet is equipped with an electronic scale and a radio frequency identification card reader. The electronic scale has an accuracy of 0.1 g and is used to accurately record the amount of reagent taken; the reagent cabinet door is provided with an electromagnetic lock and a mechanical lock, and only authorized personnel can open it.

[0137] Warehouse entry management: Enter the basic information of hazardous chemicals through the system, including name, CAS number, purity, specification, production date, quality guarantee period, etc. The system automatically assigns storage locations according to the properties of hazardous chemicals. For example, flammable items are assigned to the explosion-proof cabinet, and strong acids and alkalis are assigned to the corrosion-resistant cabinet.

[0138] First, import the detection standard method library used in the laboratory. When the sample is registered and stored in the warehouse, the system automatically associates the required reagent list and dosage standard according to the detection items. When the inspection personnel receive the reagents, the system displays the types of reagents that can be received and records the actual received amount through the electronic scale. After use, weigh and return, and the system automatically calculates whether the actual usage amount is within a reasonable range.

[0139] R & D personnel can independently select the required reagents. The system records the reagent usage records and establishes a usage amount database, and issues a warning when the single-time receiving amount is abnormal. At the same time, track the reagent inventory and automatically remind to replenish the stock when the inventory is lower than the threshold.

[0140] The solution of the present application can:

[0141] Safety effect: Realize the full traceability of hazardous chemicals, reduce safety hazards; intelligent environmental monitoring gives early warnings in a timely manner to ensure storage safety; permission management prevents unauthorized use. Management effect: Digital replaces traditional paper records, improving management efficiency; intelligent warning avoids overuse of reagents and inventory backlogs; data analysis assists in decision-making and optimizes the procurement plan. Economic effect: Reduce the workload of manual records, reduce labor costs; optimize inventory management, reduce sluggish materials; improve the use efficiency of reagents and reduce waste.

[0142] Figure 2 This is a schematic structural diagram of the management system for hazardous chemicals in the laboratory of the embodiment of the present invention. As Figure 2 shown, the system includes:

[0143] The first unit is used to obtain the operation video data of the operator by the image acquisition devices deployed in multiple areas of the laboratory. The image acquisition devices include an infrared thermal imaging camera and a depth camera; input the operation video data into a pre-trained human body pose recognition model to extract the key node coordinate information and skeletal motion trajectory information of the operator; based on a preset database of hazardous chemical operation specifications, map the key node coordinate information and skeletal motion trajectory information into standardized operation behavior sequence data;

[0144] The second unit is used to input the operation behavior sequence data into a pre-trained time series behavior evaluation model. The time series behavior evaluation model is constructed based on a long short-term memory network and is used to identify non-standard operation segments in the operation behavior sequence data; for the non-standard operation segments, in combination with a preset information library of hazardous chemical attributes, calculate the risk level coefficient corresponding to each non-standard operation; based on the risk level coefficient and the spatio-temporal characteristics of the non-standard operation segments, generate a multi-dimensional evaluation report including operation time, position, action type, and risk score;

[0145] The third unit is used to call a preset corrective strategy library according to the multi-dimensional evaluation report to generate targeted operation behavior improvement suggestions. The operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; push the operation behavior improvement suggestions to the operator through the laboratory terminal device; monitor the operation behavior of the operator after implementing the improvement suggestions, update the risk score, and when the risk score is lower than the preset threshold for three consecutive times, add the corresponding operation behavior sequence data to the standard operation sample library; regularly update the parameters of the time series behavior evaluation model based on the standard operation sample library to achieve dynamic optimization of the evaluation standard.

[0146] In the third aspect of the embodiment of the present invention,

[0147] A kind of electronic device is provided, including:

[0148] A processor;

[0149] A memory for storing instructions executable by the processor;

[0150] Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0151] In the fourth aspect of the embodiment of the present invention,

[0152] Provided is a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the foregoing method.

[0153] The present invention may be a method, an apparatus, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present invention.

[0154] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. The management method of hazardous chemicals in the laboratory is characterized by: include: Acquire the operator's dangerous goods operation video data through image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include infrared thermal imaging cameras and depth cameras; Inputting the operation video data into a pre-trained human posture recognition model to extract key node coordinate information and skeletal motion trajectory information of the operator; Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data; The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, the risk level coefficient corresponding to each irregular operation is calculated in combination with a preset dangerous goods attribute information library; Based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation fragment, a multidimensional assessment report including operation time, location, action type and risk score is generated; Based on the multi-dimensional evaluation report, a preset correction strategy library is called to generate targeted operation behavior improvement suggestions, which include text descriptions, standard action videos, and three-dimensional action demonstration models; and the operation behavior improvement suggestions are pushed to operators through laboratory terminal devices; Monitor the operator's operation behavior after implementing the improvement suggestions, update the risk score, and add the corresponding operation behavior sequence data to the standard operation sample library when the risk score is lower than the preset threshold for three consecutive times; Regularly updating the parameters of the temporal behavior evaluation model based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria; According to the multi-dimensional evaluation report, a preset correction strategy library is called to generate targeted operation behavior improvement suggestions, wherein the operation behavior improvement suggestions include text descriptions, standard action videos, and three-dimensional action demonstration models; The operation behavior improvement suggestions pushed to the operator through the laboratory terminal device include: Constructing a feature vector based on the risk features in the multidimensional assessment report; Based on the knowledge graph retrieval algorithm, the similarity between the feature vector and the strategy node in the preset correction strategy library is calculated, and multiple strategies with the highest similarity are selected to form a candidate strategy set; fuzzy reasoning is used to perform environmental constraint assessment, risk assessment, and operator capability assessment on the candidate strategy set, and the adaptability score of each strategy is obtained through weighted fusion; The strategy with the highest adaptability score in the preset correction strategy library is called to generate operational behavior improvement suggestions, and the strategy is converted into a text description containing specific parameters and operational points; a standard action sequence matching the current operation type is retrieved from the action template library, and a standard action video is generated through posture alignment and trajectory smoothing; a motion trajectory is constructed based on the key frames of the standard action sequence, and the motion trajectory is mapped to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional action demonstration model; the operational behavior improvement suggestions include the text description, the standard action video and the three-dimensional action demonstration model.

2. The method according to claim 1, characterized in that Inputting the operation video data into a pre-trained human posture recognition model, extracting the key node coordinate information and skeletal motion trajectory information of the operator includes: The operation video data is input into a pre-trained human posture recognition model, the human posture recognition model uses a two-stream feature extraction network to process RGB image sequences and depth image sequences respectively, the RGB image sequence is subjected to a ResNet-50 network to extract the operator's performance features, and the depth image sequence is subjected to a PointNet++ network to process three-dimensional point cloud data to obtain the operator's spatial position information; based on the attention mechanism, the feature channel importance scores of the performance features and the spatial position information are calculated, and the performance features and the spatial position information are weightedly fused according to the importance scores to obtain fused features; Input the fused features into the spatiotemporal convolution module, perform key node detection on the features output by the spatiotemporal convolution module through a multi-scale feature pyramid, and extract key node coordinate information in combination with a position sensitive loss function; A skeleton connection graph is constructed based on the key node coordinate information, the skeleton node sequence is processed by a graph convolutional network to learn the motion correlation between nodes, and the predicted trajectory of the skeleton node sequence is smoothed by Kalman filtering to obtain the skeleton motion trajectory information.

3. The method according to claim 1, characterized in that Based on the preset dangerous goods operation specification database, mapping the key node coordinate information and the skeletal motion trajectory information into standardized operation behavior sequence data includes: Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data, and the standard actions in the dangerous goods operation specification database are decomposed into basic action units, wherein the basic action units include the relative position relationship of key nodes and motion trajectory characteristics; A dynamic time warping algorithm is used to match and align the skeletal motion trajectory information with the basic action unit; an adaptive scale transformation module is used to normalize the key node coordinate information; Based on the probabilistic graphical model, a state transfer matrix containing timing constraints and spatial constraints is established, and the optimal state sequence is solved by the Viterbi algorithm in combination with the type of dangerous goods and the operating environment information; the optimal state sequence is converted into the standardized operation behavior sequence data according to the hierarchical structure of operation type, action unit timing relationship, normalized coordinates and motion characteristics.

4. The method according to claim 1, characterized in that: Inputting the operation behavior sequence data into a pre-trained temporal behavior evaluation model, wherein the temporal behavior evaluation model is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; For the irregular operation fragments, combined with the preset dangerous goods attribute information database, the risk level coefficient corresponding to each irregular operation is calculated, including: The operation behavior sequence data is input into a pre-trained temporal behavior evaluation model, the temporal behavior evaluation model is constructed based on a long short-term memory network, the input layer of the temporal behavior evaluation model segments the operation behavior sequence data according to time windows with a duration of two seconds and an adjacent window overlap rate of fifty percent, and extracts the relative position relationship of key nodes, movement speed, acceleration, and joint angle change rate of the operation behavior sequence data in each time window to construct a multidimensional feature vector; three types of convolution kernels, namely one by one, three by three, and five by five, are used to process the multidimensional feature vector in parallel, and temporal features are obtained by adaptively fusing features of different scales through a channel attention mechanism; Inputting the temporal features into a bidirectional long short-term memory network, wherein the bidirectional long short-term memory network includes a forward long short-term memory network and a reverse long short-term memory network, wherein each network unit of the forward long short-term memory network and the reverse long short-term memory network includes an input gate, a forget gate, and an output gate, and introduces a residual connection and a layer normalization mechanism; calculating the importance weights of different time steps based on a temporal attention mechanism, and identifying irregular operation fragments in the operation behavior sequence data according to the importance weights; For the irregular operation fragments, combined with the preset hazardous goods attribute information library, the risk level coefficient corresponding to each irregular operation fragment is calculated, the hazardous characteristic parameters are extracted from the hazardous goods attribute information library, and the position deviation, speed deviation, posture deviation characteristics of the irregular operation fragments are multi-dimensionally mapped with the hazardous characteristic parameters to construct a risk assessment matrix; the hierarchical analysis method is used to determine the weight coefficients of different deviation dimensions, and the risk level coefficient is calculated using an exponentially growing nonlinear mapping function for deviations exceeding the safety threshold.

5. The method according to claim 1, characterized in that Constructing a motion trajectory based on the key frames of the standard motion sequence, and mapping the motion trajectory to a human skeleton model through an inverse kinematics algorithm to generate a three-dimensional motion demonstration model includes: Calculating the optical flow field between adjacent frames in the standard action sequence, and constructing a motion energy function according to the amplitude change and direction change of the optical flow field; performing peak detection on the motion energy function based on an adaptive threshold, and marking the frame corresponding to the peak as a key frame; performing spatial normalization processing on the key frame to eliminate the deformation caused by the shooting angle and distance to obtain a standardized key frame; The key point positions in the standardized key frame are interpolated using a cubic spline interpolation function to generate an initial motion trajectory; joint angle constraints and speed constraints are constructed according to human kinematic characteristics, and the joint angle constraints and speed constraints are applied to the initial motion trajectory as boundary conditions; the initial motion trajectory is smoothed with minimizing acceleration as the optimization goal to obtain a smooth motion trajectory; the interpolation point density is adjusted based on the time information of the key frame to adjust the speed of the smooth motion trajectory; The human skeleton structure is decomposed into multiple kinematic chains, and priorities are assigned to each kinematic chain based on the importance of the end effector; the joint angles are directly calculated for the kinematic chains with analytical solutions, and the joint angles are iteratively solved using the least squares method with a damping factor for the kinematic chains without analytical solutions; a joint comfort evaluation function is constructed to calculate the deviation between the joint angle and the natural posture, and the deviation is added as a soft constraint to the optimization target; the error change rate during the iteration process is monitored, and the damping factor is dynamically adjusted according to the error change rate; The variation range of the joint angle between adjacent frames is detected to identify the joint angle causing the jump; the trajectory of the identified jump joint angle is replanned using the minimum acceleration criterion to achieve a smooth transition; the optimized joint angle sequence is applied to the human skeleton model to generate a three-dimensional action demonstration model with continuity and naturalness.

6. A management system for hazardous chemicals in a laboratory, used to implement the method described in any one of claims 1 to 5, characterized in that: include: The first unit is used to obtain the operator's dangerous goods operation video data through image acquisition devices deployed in multiple areas of the laboratory, wherein the image acquisition devices include an infrared thermal imaging camera and a depth camera; input the operation video data into a pre-trained human posture recognition model to extract the operator's key node coordinate information and skeletal motion trajectory information; Based on a preset dangerous goods operation specification database, the key node coordinate information and the skeletal motion trajectory information are mapped into standardized operation behavior sequence data; The second unit is used to input the operation behavior sequence data into a pre-trained temporal behavior evaluation model, which is constructed based on a long short-term memory network and is used to identify irregular operation segments in the operation behavior sequence data; for the irregular operation segments, combined with a preset dangerous goods attribute information library, calculate the risk level coefficient corresponding to each irregular operation; Based on the risk level coefficient and the spatiotemporal characteristics of the irregular operation fragment, a multidimensional assessment report including operation time, location, action type and risk score is generated; The third unit is used to call a preset correction strategy library according to the multi-dimensional evaluation report, generate targeted operation behavior improvement suggestions, and the operation behavior improvement suggestions include text descriptions, standard action videos and three-dimensional action demonstration models; and push the operation behavior improvement suggestions to the operator through the laboratory terminal device; Monitor the operator's operation behavior after implementing the improvement suggestions, update the risk score, and add the corresponding operation behavior sequence data to the standard operation sample library when the risk score is lower than the preset threshold for three consecutive times; The parameters of the timing behavior evaluation model are regularly updated based on the standard operation sample library to achieve dynamic optimization of the evaluation criteria.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images