Fire operation monitoring method and system based on evidence alignment self-supervised model

Abstract

The application discloses a fire operation monitoring method and system based on an evidence alignment self-supervised model. The method first collects fire operation images, extracts human posture data, and merges the data into a skeleton dataset to construct labeled and unlabeled samples. Second, an evidence alignment self-supervised model including a multi-scale gate graph convolution backbone network, a hierarchical uncertainty perception module, and an evidence dictionary is constructed. The unlabeled samples are generated into heterogeneous view pairs, and the heterogeneous view pairs and the labeled samples are subjected to feature extraction by the evidence alignment self-supervised model to obtain two perception evidences. Then, the perception evidences obtained from the unlabeled samples are calculated to obtain residual evidence, and a loss function is constructed to train the evidence alignment self-supervised model. Finally, based on the trained evidence alignment self-supervised model, the collected fire operation videos are tested to obtain monitoring results. The application can identify and warn the illegal operation of the fire operation personnel, thereby ensuring industrial safety.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and relates to a hot work monitoring method and system based on an evidence-aligned self-supervised model. Background Technology

[0002] Hot work is one of the eight special types of operations. The process includes pre-operation preparation, hot work operation, on-site monitoring, and post-operation cleanup. During routine operations, workers are prone to negligence, inadequate training, or other reasons, leading to improper handling of fire sources, dangerous contact between limbs and equipment, or failure to wear personal protective equipment correctly. These violations can cause fires, explosions, and other accidents, resulting in personal injury and economic losses. To improve operational safety, it is necessary to monitor the work behavior of hot work operators in real time.

[0003] With the rapid development of artificial intelligence technology, deep learning-based action recognition methods are receiving increasing attention in industrial behavior monitoring. Currently, mainstream action recognition models, due to their superior performance, are often optimized using data-driven approaches. However, hot work operations involve a wide variety of tasks and diverse working environments. Dedicated datasets need to be created for different industrial scenarios to fine-tune the models and ensure recognition stability. However, creating these datasets requires manual annotation, and extensive annotation work typically incurs high costs.

[0004] To reduce the cost of dataset production, self-supervised learning methods can be used to train models. However, self-supervised learning, limited by its learning mode, introduces significant uncertainty during training, thus reducing model performance. To mitigate the impact of uncertainty, this invention proposes a hot work monitoring method and system based on an evidence-aligned self-supervised model. This model can achieve end-to-end self-supervised learning using a small number of labeled samples and a large number of unlabeled samples, effectively reducing dataset production costs. Simultaneously, the model can perceive and identify uncertainty during training and perform targeted optimizations, significantly reducing the impact of uncertainty during the learning process and maximizing the model's recognition performance. Summary of the Invention

[0005] To overcome the above problems, the purpose of this invention is to provide a hot work monitoring method and system based on an evidence-aligned self-supervised model, capable of identifying and issuing early warnings of unauthorized actions by hot work operators, thereby ensuring industrial safety. The specific technical solution of this invention is as follows:

[0006] A hot work monitoring method based on an evidence-aligned self-supervised model includes the following steps:

[0007] S1: Use a camera to capture images of hot work operations, build and train an HRNet model, extract human pose data to aggregate a skeleton dataset, and construct labeled and unlabeled samples.

[0008] S2: Construct an evidence-aligned self-supervised model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary.

[0009] S3: Randomly extract labeled and unlabeled samples from the skeleton dataset. Apply data augmentation and strong augmentation to the unlabeled samples to generate heterogeneous view pairs. Extract features from the heterogeneous view pairs and labeled samples using an evidence alignment self-supervised model to obtain two perceptual evidences.

[0010] S4: For the perceptual evidence obtained from unlabeled samples, subtract the heterogeneous evidence from adjacent layers in each perceptual evidence to calculate and obtain residual evidence.

[0011] S5: Based on residual evidence, construct a loss function to achieve evidence alignment and train a self-supervised model for evidence alignment.

[0012] S6: Based on the trained evidence-aligned self-supervised model, the collected hot work operation videos are tested to obtain the hot work operation monitoring results.

[0013] The specific implementation process of the evidence alignment self-supervised model is as follows:

[0014] S2.1: Feature extraction for joint, bone, and motion skeleton samples: The multi-scale gated graph convolutional backbone network is composed of several enhanced spatiotemporal graph convolutional modules connected in series. Each enhanced spatiotemporal graph convolutional module is composed of an adaptive edge importance weighting module, graph convolution, and temporal convolution layers cascaded together. At the end of the multi-scale gated graph convolutional backbone network, there is a multi-scale fusion module. This multi-scale fusion module fuses the features output by the enhanced spatiotemporal graph convolutional modules through residual connections to obtain fused features.

[0015] S2.2: Input the fused features into the hierarchical uncertainty perception module, which consists of a multi-layer perception mechanism. Each layer of the perception mechanism is connected to an external evidence recognition layer to obtain the perceived evidence corresponding to the features of each layer. .

[0016] S2.3: The evidence dictionary is used to store prior evidence and corresponding features generated during training.

[0017] Step 3 is specifically implemented as follows:

[0018] S3.1: Randomly select labeled and unlabeled samples from the dataset, apply weak and strong augmentation to each stream of data in the unlabeled samples respectively, and generate heterogeneous view pairs. and ,in, and They refer to weak enhancement and strong enhancement, respectively. and These refer to the sample index respectively.

[0019] S3.2: Transfer heterogeneous view pairs and A shared multi-scale gated graph convolutional backbone network is used to extract fused features separately. and fusion features , will integrate features and fusion features Heterogeneous perceptual evidence is obtained by passing through a hierarchical uncertainty perception module. and Simultaneously, labeled samples are input into the evidence alignment self-supervised model to identify supervised perceptual evidence. .

[0020] Step S4 is specifically implemented as follows: obtaining the features and characteristics The data are input separately into the hierarchical uncertainty perception module, and the output features of each layer of the perception module are input into the external evidence recognition layer to obtain multi-layered heterogeneous evidence. , Subtracting the heterogeneous evidence from adjacent layers yields the residual evidence. , .

[0021] In another aspect, the present invention also provides a hot work monitoring system based on an evidence-aligned self-supervised model, comprising the following modules:

[0022] The hot work operation image sample module uses a camera to capture hot work operation images, builds and trains an HRNet model, extracts human posture data to aggregate a skeleton dataset, and constructs labeled and unlabeled samples.

[0023] The evidence alignment self-supervised module is used to construct an evidence alignment self-supervised model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary.

[0024] The perceptual evidence module is used to randomly extract labeled and unlabeled samples from the skeleton dataset, apply data augmentation and strong augmentation to the unlabeled samples to generate heterogeneous view pairs, and extract features from the heterogeneous view pairs and labeled samples respectively through the evidence alignment self-supervised model to obtain two perceptual evidences.

[0025] The residual evidence module is used to calculate residual evidence by subtracting heterogeneous evidence from adjacent layers in each perceptual evidence obtained from unlabeled samples.

[0026] The training module constructs a loss function based on residual evidence to achieve evidence alignment and trains a self-supervised model for evidence alignment.

[0027] The hot work monitoring results module, based on a trained evidence-aligned self-supervised model, tests the collected hot work videos to obtain the hot work monitoring results.

[0028] The beneficial effects of this invention are as follows:

[0029] 1. The method and system proposed in this invention employ a self-supervised action recognition method to identify violations by personnel performing hot work. Compared to traditional supervised methods, this invention can significantly reduce the manpower and time costs required for data annotation, facilitating rapid and low-cost deployment in industrial settings.

[0030] 2. The method and system proposed in this invention are based on an evidence-aligned self-supervised model for action recognition. Unlike existing methods, this invention proposes an evidence alignment mechanism that enables the model to simultaneously optimize the backbone network and uncertainty perception module end-to-end in a self-supervised mode, thereby avoiding the high uncertainty caused by the two-stage training paradigm. Furthermore, by aligning heterogeneous residual evidence with heterogeneous global evidence, the model can effectively reduce uncertainty during the optimization process, making it more suitable for complex and ever-changing industrial environments.

[0031] 3. In order to more accurately identify hot work operation actions, this invention proposes a multi-scale gated graph convolutional backbone network. This model combines graph convolutional networks with gated attention mechanisms, which effectively improves the mapping ability between features and action categories during feature extraction. It can solve the problem of misidentification caused by the ambiguity of similar actions in industrial environments. Attached Figure Description

[0032] Figure 1 A flowchart of a hot work monitoring method and system based on an evidence-aligned self-supervised model;

[0033] Figure 2 Align the evidence with the structure diagram of the self-supervised model;

[0034] Figure 3 This is a diagram of the multi-scale gated graph convolutional backbone network structure.

[0035] Figure 4 The method proposed for the example is shown in the recognition effect diagram;

[0036] Figure 5 Training curves of the model loss function proposed for the example;

[0037] Figure 6 The training curve for the accuracy of the model proposed for the example. Detailed Implementation

[0038] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0039] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0040] like Figure 1 As shown, this invention discloses a method and system for monitoring hot work operations based on an evidence alignment self-supervised model. The main steps of the method are as follows: First, images of hot work personnel's operations are collected, and a self-supervised dataset is constructed. Then, an evidence alignment self-supervised model is constructed for self-supervised learning. The model can acquire heterogeneous data through different augmentation methods and generate heterogeneous residual evidence and heterogeneous global evidence. Alignment loss is obtained by aligning the heterogeneous evidence. Then, by simultaneously constraining the alignment loss and the supervised loss, the model can achieve end-to-end optimization. Finally, the optimized model is deployed to the hot work site to perform action recognition on the real-time extracted human posture information and issue warnings based on the identified violations.

[0041] To facilitate understanding of the above technical solutions of the present invention, the following detailed description of the above technical solutions of the present invention will be provided through specific embodiments.

[0042] A hot work monitoring method based on an evidence-aligned self-supervised model includes the following steps:

[0043] S1: Use a camera to capture images of hot work operations, build and train an HRNet model, extract human pose data to aggregate a skeleton dataset, and construct labeled and unlabeled samples.

[0044] S2: Construct a self-supervised evidence alignment model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary, such as... Figure 2 As shown.

[0045] S3: Randomly extract labeled and unlabeled samples from the skeleton dataset. Apply data augmentation and strong augmentation to the unlabeled samples to generate heterogeneous view pairs. Extract features from the heterogeneous view pairs and labeled samples using an evidence alignment self-supervised model to obtain two perceptual evidences.

[0046] S4: For the perceptual evidence obtained from unlabeled samples, subtract the heterogeneous evidence from adjacent layers in each perceptual evidence to calculate and obtain residual evidence.

[0047] S5: Based on residual evidence, construct a loss function to achieve evidence alignment and train a self-supervised model for evidence alignment.

[0048] S6: Based on the trained evidence-aligned self-supervised model, the collected hot work operation videos are tested to obtain the hot work operation monitoring results.

[0049] The specific implementation process of step 1 is as follows: using a camera to collect data such as... Figure 4 The hot work operation images shown are used to construct and train an HRNet model to extract human pose data from the hot work operation images, and then aggregate the human pose data into a skeleton dataset. And some data in the skeleton dataset were labeled, and the labeled samples are denoted as: ,in, , , Each refers to the labeled first element in the dataset. Samples of joints, bones, and skeletal structures. Refers to the first in the dataset The labels corresponding to each sample; unlabeled samples are denoted as: ,in, , , They refer to the unlabeled first and second elements in the dataset. Samples of joints, bones, and motion skeletons.

[0050] The specific implementation process of the evidence alignment self-supervised model is as follows:

[0051] S2.1: Perform fusion feature extraction separately for joint, bone, and motion skeleton samples: such as Figure 3 As shown, the multi-scale gated graph convolutional backbone network consists of several enhanced spatiotemporal graph convolutional modules connected in series. Each enhanced spatiotemporal graph convolutional module is composed of an adaptive edge importance weighting module, graph convolution, and temporal convolution layers cascaded together. At the end of the multi-scale gated graph convolutional backbone network, there is a multi-scale fusion module. This multi-scale fusion module fuses the features output by the enhanced spatiotemporal graph convolutional modules through residual connections to obtain fused features.

[0052] S2.2: Input the fused features into the hierarchical uncertainty perception module, which consists of a multi-layer perception mechanism. Each layer of the perception mechanism is connected to an external evidence recognition layer to obtain the perceived evidence corresponding to the features of each layer. .

[0053] S2.3: The evidence dictionary is used to store prior evidence and corresponding features generated during training.

[0054] Step 3 is specifically implemented as follows:

[0055] S3.1: From the dataset Randomly select labeled samples and unlabeled samples ,right Weak and strong enhancements are applied to each stream of data to generate heterogeneous views. and ,in, and They refer to weak enhancement and strong enhancement, respectively. and Each refers to a sample index;

[0056] S3.1.1: Set a random seed from the dataset. Randomly select labeled samples and unlabeled samples .

[0057] S3.1.2: For Weak and strong enhancements are applied to each stream of data to generate heterogeneous views. and Weak enhancements include data clipping and pruning, while strong enhancements include clipping, spatial flipping, rotation, axis masking, pruning, time flipping, and Gaussian noise.

[0058] S3.2: Transfer heterogeneous view pairs and A shared multi-scale gated graph convolutional backbone network is used to extract fused features separately. and fusion features , will integrate features and fusion features Heterogeneous perceptual evidence is obtained by passing through a hierarchical uncertainty perception module. and At the same time, there will be labeled samples. The input is fed into the evidence alignment self-supervised model to identify supervised perceptual evidence. .

[0059] In the self-supervised evidence alignment model, the evidence dictionary is implemented as follows:

[0060] Calculate fusion features separately Fusion characteristics The distance to prior features in the evidence dictionary is used to obtain feature similarity, and features are selected accordingly. and Highest similarity Each prior feature; the prior evidence corresponding to the prior features in the evidence dictionary. The evidence is fused with the last layer of evidence in the corresponding multilayer perceptual evidence using the Murphy average fusion method to obtain the global evidence. , ; Calculate the uncertainty of global evidence using the evidence value distribution of global evidence. When uncertainty Less than the threshold When necessary, the evidence dictionary is updated using the corresponding global evidence and heterogeneous features.

[0061] Step S4 is implemented as follows:

[0062] The obtained features and characteristics The data are input separately into the hierarchical uncertainty perception module, and the output features of each layer of the perception module are input into the external evidence recognition layer to obtain multi-layered heterogeneous evidence. ,

[0063] Subtracting the heterogeneous evidence from adjacent layers yields the residual evidence. ,

[0064] Step S5 is implemented as follows:

[0065] Using KL divergence to calculate evidence of heterogeneous residuals and The evidence distance between them is denoted as the residual alignment loss. By constraining this loss, the evidence alignment model possesses reliable uncertainty quantification capabilities during feature mapping; simultaneously, it utilizes Euclidean distance to compute heterogeneous global evidence. and The evidence distance between them is denoted as the global alignment loss. This loss is used to guide the model to capture key pose features more effectively; the weights of the residual alignment loss are set to... The weights of the global alignment loss are By introducing weights to aggregate the two alignment losses, the model's uncertainty quantification and feature extraction capabilities are balanced, resulting in a comprehensive alignment loss. like Figure 5 As shown; the difference between the predicted results and the true labels of supervised samples is calculated using Euclidean distance, resulting in the following... Figure 5 The supervised loss shown This loss is used to improve the action category recognition capability of the uncertainty perception module; the comprehensive alignment loss will be used. With supervised loss Combine these elements to construct a complete self-supervised loss function. Set hyperparameters such as the number of training epochs, batch size, and learning rate. Based on the self-supervised loss function, use an optimizer to iteratively optimize the action recognition evidence model, resulting in the following: Figure 6 The training process shown ultimately yields the model weight file.

[0066] Step S6 is implemented as follows:

[0067] The hot work operation is captured in real time using a camera to obtain the operation video. The HRNet model is then used to extract human pose information from each frame of the image, resulting in the following: Figure 4 The human posture information shown is used to input joint, bone, and skeletal skeleton samples from the human posture information into a trained evidence alignment self-supervised model to obtain global evidence for each data stream. Uncertainty is generated for each data stream's global evidence, and fusion weights for each piece of evidence are constructed based on this uncertainty. The three global pieces of evidence are then fused to obtain the final identification evidence. By fusing action recognition results from multiple representations, the model can accurately capture the operational behavior at the current moment; based on the maximum value judgment principle, it integrates the recognition evidence... The action category corresponding to the highest evidence value is used as the operator's action at that moment. If the judgment result belongs to the preset hot work violation action category, an alarm will be triggered.

[0068] In another aspect, the present invention also provides a hot work operation monitoring system based on an evidence alignment self-supervised model, for implementing the hot work operation monitoring method, including a hot work operation image sample module, an evidence alignment self-supervised module, a perception evidence module, a residual evidence module, a training module, and a hot work operation monitoring result module.

[0069] The hot work operation image sample module uses a camera to capture hot work operation images, constructs and trains an HRNet model, extracts human posture data to aggregate a skeleton dataset, and constructs labeled and unlabeled samples.

[0070] The evidence alignment self-supervised module is used to construct an evidence alignment self-supervised model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary.

[0071] The perceptual evidence module is used to randomly extract labeled and unlabeled samples from the skeleton dataset, apply data augmentation and strong augmentation to the unlabeled samples to generate heterogeneous view pairs, and extract features from the heterogeneous view pairs and labeled samples respectively through an evidence alignment self-supervised model to obtain two perceptual evidences.

[0072] The residual evidence module is used to calculate and obtain residual evidence by subtracting heterogeneous evidence from adjacent layers in each perceptual evidence obtained from unlabeled samples.

[0073] The training module constructs a loss function based on residual evidence to achieve evidence alignment and trains a self-supervised evidence alignment model.

[0074] The hot work monitoring result module, based on a trained evidence-aligned self-supervised model, tests the collected hot work video to obtain the hot work monitoring result.

[0075] Example:

[0076] S1: Monitoring cameras are installed at the hot work site to capture and record the operations of personnel online, obtaining multiple video clips containing continuous dynamic movements. A human pose estimation model based on HRNet-W48 is constructed to identify the video clips, obtaining corresponding human pose data. This data is then aggregated into a skeleton dataset. We then manually labeled some of the data to obtain a self-supervised action recognition dataset.

[0077] S2: Construct as follows Figure 2 The evidence-aligned self-supervised model shown has an internal topology consisting of the following three core modules:

[0078] (1) Multi-scale gated graph convolutional backbone network: This network consists of 9 enhanced spatiotemporal graph convolutional modules connected in series. The number of input channels is first increased from 3 to 64 through the first convolutional layer, the number of channels in layers 2-4 is 124, the number of channels in layers 5-7 is 256, and the number of channels in layers 8-9 is expanded to 512. The features output from layers 3, 5, and 8 are aggregated through a multi-scale feature fusion module, and finally merged with the features output from layer 9 into the gated attention layer to obtain the global feature vector.

[0079] (2) Hierarchical Uncertainty Perception Module: This module is connected after the backbone network and consists of three fully connected layers in series. The number of neurons in each layer is as follows: 256 in the first layer, 128 in the second layer, and 64 in the third layer. At the output of each fully connected layer, an uncertainty perception fully connected layer with a Softplus activation function (output dimension equal to the number of categories, 11) is externally connected, directly mapping the hidden layer features to multi-layer heterogeneous evidence. .

[0080] (3) Evidence dictionary: A first-in-first-out queue with a fixed capacity of 1024 is allocated in memory, which is used to store historical prior representation vectors with low uncertainty and high confidence and their corresponding evidence distributions.

[0081] S3: Set the random number seed to RandomSeed=42, and randomly draw labeled samples from the dataset D. and unlabeled samples In order for the model to extract robust representations from self-supervised learning, this unlabeled sample... Weak enhancement is applied to each stream of data. Specifically, the weak enhancement operation involves randomly cutting the skeleton sequence along the time axis, with the cropping ratio maintained between [0.9, 1.0] of the original sequence length. Then, bilinear interpolation is used to readjust and restore the original number of frames, thereby generating a weakly enhanced view. In the strong enhancement operation, the sequence is first randomly rotated at a 15-degree angle and scaled by 0.9 times in space, and then the time axis is randomly reversed by 30% of the frames to generate a highly differentiated strong enhancement view. ;

[0082] Generate heterogeneous view pairs and and labeled samples The input is fed into a shared model backbone network. Taking joint flow as an example, the weakly enhanced view is extracted by the backbone network to obtain a weakly enhanced feature vector. The enhanced feature vector is obtained by extracting the enhanced view through the backbone network. Labeled samples are processed by a backbone network to obtain labeled sample feature vectors. The weakly enhanced low-dimensional feature vector, the strongly enhanced feature vector, and the labeled sample feature vector are input into the corresponding uncertainty perception module to obtain heterogeneous perception evidence. , and supervised perception evidence .

[0083] S4: To capture the subtle drift of features during the layer-by-layer abstraction process, weak enhancement evidence between adjacent layers is used. Subtraction yields evidence of weakly enhanced residuals. ,right Performing the same operation yields strongly enhanced residual evidence. .

[0084] S5: Using the weak enhancement features of the current sample As the primary factor, it is calculated to compare with the 1024 prior features stored in the evidence dictionary. Cosine similarity between them:

[0085]

[0086] Based on the similarity scores from highest to lowest, the four most similar prior features were retrieved. The corresponding prior evidence vectors stored in the dictionary are extracted and averaged with the evidence from the last layer output of the uncertainty perception module to obtain the average evidence. The average evidence is normalized after being multiplied five times to obtain the global evidence. Calculate the uncertainty based on the global evidence value distribution. Taking weak enhancement as an example, the total amount of evidence Then its uncertainty is ,in, This represents the total number of operation categories during hot work; the prediction uncertainty for the current sample is calculated. (In this embodiment, the uncertainty threshold is set to 0.25) when the extracted features of the sample are determined to be extremely stable; at this time, the current feature vector is... and the global evidence after fusion As a new prior entry, it is pushed to the tail of the evidence dictionary, and the oldest entry at the head of the queue is popped, thus achieving adaptive dynamic updating of the dictionary. The same method is applied to strongly augmented samples. Next, a loss function for the samples during the learning process is constructed, and the total loss function for model training is composed of the comprehensive evidence alignment loss. With supervised loss It consists of two weighted combinations, specifically, the overall alignment loss. Includes residual alignment loss And global alignment loss In this context, the residual alignment loss uses KL divergence to measure the distribution distance between weak and strong enhancements across the residual evidence at each level, while the global alignment loss uses Euclidean distance to enforce consistency in the global prediction evidence of both. In the embodiment, and The weights are respectively and For labeled data, a multi-class evidence cross-entropy loss function is used as a constraint to integrate the alignment loss. and supervised losses Adding them together forms the complete self-supervised loss function. In this embodiment, the AdamW optimizer (initial learning rate) is used. Weight decay coefficient , Set as The model is trained iteratively for 150 epochs on a graphics card. After training, the optimal model weight file is exported and saved. .

[0087] S6: Deploy the trained weight file movable_fire_model.pth on a GPU computing server at the edge of the hot work site. Dynamically capture video of the ongoing hot work operation using on-site cameras. The time index in this example is... The system then traces back 30 consecutive frames, including the current frame, to extract the human pose information within those 30 frames. This information is then converted into the number of three-stream skeletons at the current moment, following the steps in S1. These three streams of data are simultaneously input into a self-supervised action recognition model running on the server for identification. The joint flow, through its corresponding backbone network and uncertainty perception module, outputs global evidence of the joint flow, with its corresponding uncertainty calculated as 0.18 according to a formula. The skeletal flow, through its corresponding backbone network and uncertainty perception module, outputs global evidence of the skeletal flow, with its corresponding uncertainty calculated as 0.38 according to a formula. The motion skeleton flow, through its corresponding backbone network and uncertainty perception module, outputs global evidence of the motion skeleton flow, with its corresponding uncertainty calculated as 0.32 according to a formula. Based on the dynamic inverse principle of "lower uncertainty, higher weight," real-time fusion weight coefficients are constructed for each of the three streams of evidence.

[0088]

[0089] Similarly, the calculation yields and The weights are used to perform a weighted linear summation of the three streams of evidence to obtain the final adaptive three-stream fusion identification evidence vector. Analysis revealed that the third action category had the highest evidence value of 0.54, therefore the current [action category] was determined to be [the most valuable]. At the specified frame, the operator's actual action falls under category 3, namely "improper ignition of the gas cutting torch." The system searches its pre-set knowledge base of violations. Since "improper ignition of the gas cutting torch" is a high-risk violation, the server immediately sends a pulse trigger signal to the on-site audible and visual alarm. The red light flashes and an audio message plays: "Warning! Ignition operation is non-compliant. Please stop immediately and repeat the operation according to the standard procedure!" Simultaneously, the system automatically extracts and packages the 30-frame video clip of the current violation and the human skeleton data, uploads it to the factory safety management cloud platform for record-keeping, and generates the final monitoring feedback.

[0090] This invention, based on self-supervised action recognition, enables low-cost training of self-supervised action recognition models tailored to different factory areas, thereby facilitating real-time monitoring of hot work operations. Unlike other methods, this invention proposes an evidence-aligned self-supervised model based on self-supervised learning and evidence theory. On the one hand, self-supervised learning reduces dataset costs; on the other hand, the evidence alignment framework mitigates uncertainties during self-supervised training, improving model recognition performance and effectively ensuring industrial safety.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A hot work monitoring method based on an evidence-aligned self-supervised model, characterized in that, Includes the following steps: S1: Use a camera to capture images of hot work operations, build and train an HRNet model, extract human pose data to aggregate a skeleton dataset, and construct labeled and unlabeled samples. S2: Construct an evidence alignment self-supervised model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary; S3: Randomly extract labeled and unlabeled samples from the skeleton dataset, apply weak and strong enhancements to the unlabeled samples to generate heterogeneous view pairs, and extract features from the heterogeneous view pairs and labeled samples respectively through the evidence alignment self-supervised model to obtain two perceptual evidences. S4: For the perceptual evidence obtained from unlabeled samples, subtract the heterogeneous evidence from adjacent layers in each perceptual evidence to calculate and obtain the residual evidence; S5: Based on residual evidence, construct a loss function to achieve evidence alignment and train a self-supervised model for evidence alignment; S6: Based on the trained evidence-aligned self-supervised model, the collected hot work operation videos are tested to obtain the hot work operation monitoring results.

2. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 1, characterized in that, The specific implementation process of step S1 is as follows: using a camera to acquire images of hot work operations, constructing and training an HRNet model, extracting human posture data from the hot work operation images, and merging the human posture data into a skeleton dataset. And label a portion of the data in the skeleton dataset, denoted as . ,in, , , Each refers to the labeled first element in the dataset. Samples of joints, bones, and skeletal structures. Refers to the first in the dataset The labels corresponding to each sample; unlabeled samples are denoted as . ,in, , , They refer to the unlabeled first and second elements in the dataset. Samples of joints, bones, and motion skeletons.

3. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 2, characterized in that, The specific implementation process of the evidence alignment self-supervised model is as follows: S2.1: Feature extraction for joint, bone, and motion skeleton samples: The multi-scale gated graph convolutional backbone network is composed of several enhanced spatiotemporal graph convolutional modules connected in series. Each enhanced spatiotemporal graph convolutional module is composed of an adaptive edge importance weighting module, graph convolution, and temporal convolution layers cascaded together. At the end of the multi-scale gated graph convolutional backbone network, there is a multi-scale fusion module. This multi-scale fusion module fuses the features output by the enhanced spatiotemporal graph convolutional modules through residual connections to obtain fused features. S2.2: Input the fused features into the hierarchical uncertainty perception module, which consists of a multi-layer perception mechanism. Each layer of the perception mechanism is connected to an external evidence recognition layer to obtain the perceived evidence corresponding to the features of each layer. ; S2.3: The evidence dictionary is used to store prior evidence and corresponding features generated during training.

4. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 3, characterized in that, The specific implementation of step S3 is as follows: S3.1: From the dataset Randomly select labeled samples and unlabeled samples ,right Weak and strong enhancements are applied to each stream of data to generate heterogeneous views. and ,in, and They refer to weak enhancement and strong enhancement, respectively. and Each refers to a sample index; S3.1.1: Set a random seed from the dataset. Randomly select labeled samples and unlabeled samples ; S3.1.2: For Weak and strong enhancements are applied to each stream of data to generate heterogeneous views. and Weak enhancements include data clipping and pruning, while strong enhancements include clipping, spatial flipping, rotation, axis masking, pruning, time flipping, and Gaussian noise. S3.2: Transfer heterogeneous view pairs and A shared multi-scale gated graph convolutional backbone network is used to extract fused features separately. and fusion features , will integrate features and fusion features Heterogeneous perceptual evidence is obtained by passing through a hierarchical uncertainty perception module. and At the same time, there will be labeled samples. The input is fed into the evidence alignment self-supervised model to identify supervised perceptual evidence. .

5. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 4, characterized in that, In the self-supervised evidence alignment model, the evidence dictionary is implemented as follows: Calculate fusion features separately Fusion characteristics The distance to prior features in the evidence dictionary is used to obtain feature similarity, and features are selected accordingly. and Highest similarity A priori feature; The prior evidence corresponding to the prior features in the evidence dictionary is fused with the last layer of evidence in the corresponding multilayer perceptual evidence using the Murphy average fusion method to obtain the global evidence. , ; The uncertainty of global evidence is calculated using the evidence value distribution of global evidence. When the uncertainty is less than a threshold... When necessary, the evidence dictionary is updated using the corresponding global evidence and heterogeneous features.

6. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 5, characterized in that, Step S4 is implemented as follows: The obtained features and characteristics The data are input separately into the hierarchical uncertainty perception module, and the output features of each layer of the perception module are input into the external evidence recognition layer to obtain multi-layered heterogeneous evidence. , ; Subtracting the heterogeneous evidence from adjacent layers yields the residual evidence. , .

7. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 6, characterized in that, Step S5 is implemented as follows: Calculating evidence of heterogeneous residuals and The evidence distance between them is denoted as the residual alignment loss. Simultaneously calculate heterogeneous global evidence and The evidence distance between them is denoted as the global alignment loss. The two alignment losses are weighted and aggregated to obtain the comprehensive alignment loss. Combined with supervised loss of labeled samples Construct a complete self-supervised loss function Based on the self-supervised loss function, an optimizer is used to iteratively optimize the action recognition evidence model, and the model weight file is obtained through training.

8. The hot work monitoring method based on an evidence-aligned self-supervised model according to claim 7, characterized in that, Step S6 is implemented as follows: The hot work operation is captured in real time using a camera. The HRNet model is then used to extract human pose information from each frame. Joint, bone, and skeletal samples from this pose information are input into a trained evidence alignment self-supervised model to obtain global evidence for each data stream. Uncertainty is generated for each data stream's global evidence, and fusion weights are constructed based on this uncertainty. The three global evidence streams are then fused to obtain the final identification evidence. ; to identify evidence The action category corresponding to the highest evidence value is used as If the operator's actions are determined to fall under a preset category of hot work violation, an alarm will be triggered.

9. A hot work monitoring system based on an evidence-aligned self-supervised model, used to implement the hot work monitoring method according to any one of claims 1 to 8, characterized in that, Includes the following modules: The hot work operation image sample module uses a camera to capture hot work operation images, builds and trains an HRNet model, extracts human posture data to aggregate a skeleton dataset, and constructs labeled and unlabeled samples. The evidence alignment self-supervised module is used to construct an evidence alignment self-supervised model that includes a multi-scale gated graph convolutional backbone network, a hierarchical uncertainty perception module, and an evidence dictionary. The perceptual evidence module is used to randomly extract labeled and unlabeled samples from the skeleton dataset, apply data augmentation and strong augmentation to the unlabeled samples to generate heterogeneous view pairs, and extract features from the heterogeneous view pairs and labeled samples respectively through the evidence alignment self-supervised model to obtain two perceptual evidences. The residual evidence module is used to calculate residual evidence by subtracting heterogeneous evidence from adjacent layers in each piece of perceptual evidence obtained from unlabeled samples. The training module constructs a loss function based on residual evidence to achieve evidence alignment and trains a self-supervised model for evidence alignment. The hot work monitoring results module, based on a trained evidence-aligned self-supervised model, tests the collected hot work videos to obtain the hot work monitoring results.

Images