A method for continuous motion recognition of construction workers at smart construction sites

Abstract

This invention provides a method for continuous action recognition of construction site personnel in smart construction sites, relating to the field of digital technology for smart construction sites. The method includes the following steps: S1, allocating an independent decoding thread to each RTSP camera, decoding the video stream, and storing it in a cache to form a temporally continuous frame sequence; calculating the scene complexity value of each video stream, including a weighted calculation of occlusion rate and action density; allocating edge computing power according to the scene complexity value to obtain the frame sequence after power allocation. This method has the following beneficial effects: by focusing on key frames of behavior through a temporal attention mechanism and constructing an action state transition matrix to complete the transition frame features, it solves the problem of insufficient model generalization ability under rare violations and new construction scenarios, and compensates for the incomplete feature defects caused by the loss of transition frames in continuous actions. It can accurately distinguish similar actions such as bending over to pick up objects and illegal climbing, significantly reducing the false alarm rate.

Description

Technical Field

[0001] This invention relates to the field of digital technology for smart construction sites, and in particular to a method for recognizing continuous movements of personnel at construction sites in smart construction sites. Background Technology

[0002] As the infrastructure industry transforms towards digitalization and intelligence, the scale of construction sites, including power grid infrastructure, bridge high-altitude operations, and large-scale building construction, is constantly expanding, and the working environment is becoming increasingly complex. This leads to a significant increase in the risk of safety accidents caused by personnel violations such as not wearing safety helmets, unauthorized climbing, and not wearing safety belts while working at heights. Traditional manual inspection methods suffer from limited coverage, slow response times, and high labor costs, making it difficult to meet the needs of real-time control and accurate identification in safety management. Therefore, the industry urgently needs intelligent personnel behavior recognition technology. Through computer vision, sensor fusion, and other means, it is possible to achieve real-time monitoring and violation warnings of dynamic personnel behavior at construction sites, thereby reducing the accident rate and improving the efficiency of construction site safety management.

[0003] Currently, there are related technologies, such as patent number CN115830294A, which provides an identification method and device for personnel management at infrastructure construction sites. This method attempts to achieve personnel management at infrastructure construction sites through visual recognition and trajectory control. The technical solution is as follows: real-time images of the infrastructure construction site are collected by a visible light camera, the YOLO target detection algorithm is used to locate the facial position and behavioral actions of personnel, low-resolution facial images are reconstructed using the BasicVSR++ algorithm, feature vectors are extracted using the ShuffleNetV2 classification network, and negative sample feature library is used to filter false alarms. Finally, personnel identification, trajectory tracking, and boundary crossing warning are achieved, which solves the problems of "low identification resolution, high false alarm rate, and difficulty in warning of personnel trajectory crossing". However, the existing technology still has the following key defects, which cannot meet the needs of accurate identification of continuous actions of personnel at construction sites: The solution does not involve the temporal analysis of continuous actions of personnel, but only superficially identifies behavioral actions. For example, in determining whether there is a person's action, it does not analyze the continuous temporal logic of the action. For example, the complete process of "raising hand, grabbing pole, exerting force, and rising" in illegal climbing makes it impossible to accurately determine dynamic violations such as working at height without a safety belt. Furthermore, it does not have an enhancement mechanism for similar action features such as bending over to pick up objects and illegal climbing, and it does not handle the problem of lost action transition frames such as the force exertion frame in climbing. This results in a high false alarm rate for similar actions, making it difficult to meet the needs of accurate identification. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is to provide a method for recognizing continuous actions of construction site personnel in smart construction sites. This method can resist interference from extreme environments such as rain, fog, and nighttime through multimodal perception without reducing the accuracy of continuous action recognition. It can accurately analyze the temporal logic of continuous violations by personnel, dynamically adapt to changes in the construction scene, balance the high computing power requirements of continuous action temporal analysis with the real-time requirements of the edge, and compensate for the loss of features in the transition frames of continuous actions to reduce the false positive rate of similar actions. Ultimately, it can achieve real-time recognition and early warning of violations by construction site personnel with low false alarms and high robustness.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: The present invention provides a method for continuous action recognition of construction site personnel applied to smart construction sites, comprising the following steps: S1. Allocate an independent decoding thread to each RTSP camera, decode the video stream, and store it in a buffer to form a temporally continuous frame sequence; calculate the scene complexity value of each video stream, including a weighted calculation of occlusion rate and motion density; allocate edge computing power according to the scene complexity value to obtain the frame sequence after computing power allocation. ; S2. Construct the Pseudo-3D feature extraction module and extract frame sequences using decompositional 3D convolution. The spatiotemporal joint features are obtained; lightweight temporal features are obtained through teacher and student network distillation and path splitting quantization; a transition frame feature completion model is constructed to complete the missing transition frames in the lightweight temporal features, resulting in the completed temporal features. ; S3. Calculate the effect of hierarchical contrast loss on the completed temporal features. Enhance similar action features; construct a dynamic GNN model to analyze the temporal features after completion. Multi-target interactions in the data are used to obtain multi-target interaction features. Verify the causal relationship between actions and scenarios using a structural causal model; S4. Construct a dynamic scene parameter library for BIM and UWB, execute actions and compare virtual and real scenes, and analyze multi-objective interaction features. The system extracts the type of human movement and the three-dimensional position of the human, obtaining preliminary movement recognition results. Perform scenario parameter anomaly checks for mismatches. S5. Initial motion recognition results It performs encoding / decoding and packet loss recovery, outputs the final compressed result to the application, and triggers an alarm by linking the sound and light alarm device if it contains illegal actions, and stores the identification result.

[0006] In the preferred embodiment, step S1 consists of the following steps: S11. Allocate an independent GStreamer decoding thread to each RTSP camera, perform hardware-accelerated decoding of the RTSP stream based on the edge device, and store the decoded video frames in a ring buffer to obtain a temporally continuous frame sequence. ,in, Number of frames; S12. Call the image analysis algorithm to read the temporally continuous frame sequence. For each frame of pixel data, calculate the occlusion rate and motion density, and then calculate the scene complexity value for each video stream using the following formula. : ; in, This is the weighting coefficient for the occlusion rate. The weighting coefficient for motion density. , The scene complexity value is represented by [0,1]; the occlusion rate is represented by the percentage of pixels in the current frame where people are occluded by the device, and the value range is [0,1]; the motion density is represented by the ratio of the number of people with dynamic actions in the current frame to the total number of people in the frame, and the value range is [0,1]. S13. Based on the scene complexity value Allocate edge computing power: when When identified as a high-complexity scenario, allocation Proportional computing power; when The scenario was determined to be of medium complexity, and the allocation was... Proportional computing power; when When the scenario is determined to be low complexity, the allocation is... Proportional computing power; obtaining the frame sequence after computing power allocation. ; in, This is a high complexity threshold; This is the standard complexity threshold, and ; All values ​​represent computing power allocation ratios, ranging from [0,1]. .

[0007] In the preferred embodiment, step S2 consists of the following steps: S21. Construct the Pseudo-3D feature extraction module, using a convolutional kernel size of [size missing]. ,padding is 3D convolutional layers for frame sequences Perform convolution on the time dimension features to obtain the time features. ; S212, Spatial Dimension 2D Convolution: Using a convolution kernel size of... ,padding is 3D convolutional layers, for temporal features Spatial feature extraction is performed on each frame to obtain spatiotemporal joint features. ; S213, Spatiotemporal Joint Features Perform channel-level time-shift operation: One-quarter of the channels are shifted one frame to the left, one-quarter of the channels are shifted one frame to the right, and the remaining half of the channels remain unchanged, thus obtaining the time-shifted enhancement feature. ; S22. Construct a teacher-student distillation network to... Perform knowledge distillation and mixing precision training: S221. The teacher network adopts the SlowFast-8x8 model, outputting soft objectives. Through temperature The softmax function is used to process the soft target, resulting in a soft target: ; S222. The student network uses the ResNet-18 model with time-shifted input enhancement features. Output Calculate the soft target cross-entropy loss: ; in, This represents the knowledge distillation loss, with a value range of [0, +∞). This represents the raw predicted value output by the teacher network. This represents the raw predicted value output by the student network; This indicates a softened objective in the teacher's online output; S223. Path-specific quantization: Symmetric quantization is applied to the Slow path of the student network, with the quantization range set to... Asymmetric quantization is used for the Fast path, with the quantization range set to [0, 255]. S224. Mixed Precision Training: The student network is trained using FP16 / FP32 mixed precision, and a dynamic loss scaling factor is set. The initial value is Each training session Step-by-step check for gradient overflow: if continuous No gradient overflow. Increase If gradient overflow occurs, Decrease The formula is: ;in, Indicates the first The loss scaling factor for each step. Indicates the first Loss scaling factor for each step This indicates the gradient overflow check step size. This represents the scaling factor adjustment factor; lightweight temporal features are obtained through backpropagation and parameter update. ; S23. Construct a transition frame feature completion model for lightweight temporal features. Complete the missing transition frames: S231. Load a time-series sample library of typical construction site violations onto a computer: samples of illegal climbing. Group, samples of people not wearing safety helmets Group, among which For each set of samples, OpenCV is used to extract time-series frames, and ResNet-50 is used to extract frame features. These features are then input into an LSTM model to train action state classification, and the output is Q=5 consecutive action states, which are the initial state. Ready state Execution status Transitional state Completion status ; S233, For any two states and ,in, Calculate the transition probability: This yields a 5×5 action-state transition matrix. ; in, This represents the probability of transitioning from state i to state j; Representing state The number of transfers; This represents the total number of transitions in state i; S234. Action state transition matrix Perform normalization to obtain the normalized action-state transition matrix. ; When lightweight features are detected There are characteristic fractures in the process, based on the normalized action-state transition matrix. Predicting transition frame features: ,in, To predict transition frame features, the temporal features are obtained after completion. .

[0008] In the preferred embodiment, step S3 specifically involves the following steps: S31. Extract the semantic embedding vectors of the violation actions from the pre-trained BERT model. For the completed temporal features Each frame is split into Image blocks Extract each image patch eigenvectors Calculate image patches eigenvectors and cosine similarity : ; in, Represents image blocks The cosine similarity to the action semantics, with values ​​ranging from [-1, 1]; Represents image blocks eigenvectors; A semantic embedding vector representing a violation action; Represents the vector dot product. Describes the L2 norm of a vector. Indicates the number of image blocks in a single frame; Cosine similarity Normalization is performed to obtain semantic attention weights : ; Get image patch pairs for each frame According to image patch pairs Calculate the spatial cycle consistency loss : ; in, This represents the spatial cycle consistency loss, with a value range of [0, +∞). Indicates and The most similar soft nearest neighbor image patch; Represents the cosine similarity function; Integrating the contrast loss across global, temporal, and spatial dimensions, the total contrast loss is calculated: ; in, This represents the total contrast loss, with a value range of [0, +∞). Represents cross-entropy loss; Indicates global comparison loss; Indicates time-comparison loss; Indicates time cycle loss; Indicates spatial contrast loss; , , The weighting coefficients for each loss are all greater than 0; S32, Define the diagram structure :node Includes personnel nodes Equipment Nodes Scene nodes ;side Represents the interaction relationships between nodes; adjacency matrix elements ,in, For nodes and The Euclidean distance; The computer initializes the GNN network with the following number of layers: Define the propagation rules according to the GNN propagation rules: ; in, Represents a node In the The feature vector of the layer; Indicates the first Layer weight matrix; Represents a node The set of neighboring nodes; Indicates the activation function; The completed time series features Input a dynamic GNN model and execute... Layer propagation computation outputs multi-target interaction features. ; S33. Verify the causal relationship between actions and scenarios using a structural causal model, and define intervention variables. Calculate causal probability With statistical probability The difference: ;in, It represents the difference between the causal probability and the statistical probability; Indicates intervention action The probability of post-scene matching; Represents actions in their natural state The probability of matching the scene; Indicates the type of personnel action; Set preset threshold ,like If the correlation is positive, it is determined to be a causal relationship; otherwise, it is determined to be a false relationship, and the process returns to step S32 to re-extract the interaction features.

[0009] In the preferred embodiment, step S4 consists of the following steps: S41. Real-time positioning data of the temporary high-altitude platform is collected using UWB positioning sensors. The positioning data includes three-dimensional coordinates. and the boundary coordinates of the danger zone , will location data every Update the BIM model once per second; Extract scene parameters from BIM model Build a dynamic scene parameter library And set the parameter to update the timestamp; S42. Based on multi-target interaction characteristics Extracting personnel action types and 3D positions ; Set height difference threshold Query the dynamic scene parameter library Get the platform height of the current scene If the action type "High-altitude operation" and If the action is determined to be a mismatch between the action and the scene, scene parameter anomaly verification is triggered; otherwise, it is determined to be a match, and a preliminary action recognition result is obtained. ; Set error threshold When it is determined that the action does not match the scene, triggering scene parameter anomaly verification, the laser rangefinder is activated to remeasure the height of the temporary high-altitude platform. ,like Then update The platform height in the middle is Return to step S42 for re-comparison.

[0010] In the preferred embodiment, step S5 specifically involves the following steps: S51. Call the ASE codec to process the preliminary action recognition results. Intra-frame prediction and entropy coding are performed on the feature map blocks to obtain the compressed result. ; Compression results Add an FEC checksum and set the checksum length to [value]. Data length Proportion, and ,use Linear block codes, where , for Data length; S52, Set the signal-to-noise ratio threshold ,like Then the computer reduces the quantization step size to Return to step S51 to re-encode; until The final compression result is obtained. ; S53. Compress the final result. Output to the application: If the final compression result If the data contains a "violation" label, the computer sends a control command to the audible and visual alarm device, triggering an alarm; simultaneously, the final compressed result... Store in the local database.

[0011] In the preferred embodiment, in step S23, an action state transition matrix is ​​established based on the timing logic of the violation action. Specifically, it includes: S231. Load a time-series sample library of typical construction site violations onto a computer: samples of illegal climbing. Group, samples of people not wearing safety helmets Group( For each sample group, extract Q consecutive action states, which are the initial states. Ready state Execution status Transitional state Completion status ; S232, Statistics Group of illegal climbing samples and From the group of samples where no helmet was worn , The number of occurrences of all state transition combinations; the statistical results of the two types of samples are combined to obtain the total number of state transitions; S233, For any two states and ,in, Transition probability: This yields a 5×5 action-state transition matrix. ; S234. Action state transition matrix Perform normalization: ;in, Represents the normalized transition probability; when a lightweight feature is detected. There are characteristic fractures in the matrix, which are then normalized. Predicting transition frame features: ,in, To predict transition frame features, the temporal features are obtained after completion. .

[0012] In the preferred scheme, the adjacency matrix in step S32 elements It also meets the following filtering criteria: Preset node interaction distance threshold: Threshold for personnel-device interaction Interpersonal interaction threshold Personnel and scene interaction threshold ,in, ; S321, When node For personnel nodes ,node For device nodes Calculate the Euclidean distance between the two. ,like Then set It was determined that there was no interaction between personnel and equipment; S322, When node and nodes All are personnel nodes Calculate the Euclidean distance between the two. ,like Then set It was determined that there was no coordinated action between the personnel; S323, When node For personnel nodes ,node scene nodes Calculate the Euclidean distance between the two. ,like Then set It was confirmed that no personnel had entered the danger zone; S324. Update the adjacency matrix based on the above filtering results. The filtered adjacency matrix is ​​obtained. ,Will Return to step S32 for GNN propagation computation.

[0013] In the preferred embodiment, the specific implementation of "ASE encoding / decoding" in step S51 includes: S511, Set the image block size to The initial action recognition results The feature map is divided into multiple image blocks; for each target block, its upper and left adjacent blocks are selected as reference blocks, the average pixel value of the reference blocks is calculated, and the predicted pixel value of the target block is predicted based on the average pixel value of the reference blocks. The formula is: ;in, Represents the predicted pixel value of the target block; This indicates the pixel value of the reference block above; This represents the pixel value of the left reference block; Calculate the actual pixel value of the target block and the predicted pixel value of the target block. The difference is used to obtain the prediction residual; S512. Employ context-based adaptive arithmetic coding, inputting the prediction residuals and quantization parameters into the coding model, and setting the coding window size to [value missing]. Bytes are dynamically adjusted in terms of encoding probability distribution through a context model to output an encoded binary stream. S513. Calculate the mean square error (MSE) and peak signal-to-noise ratio (PSNR) of the data before and after compression: ; Where PSNR represents peak signal-to-noise ratio, For feature map height, The width of the feature map. These are the original pixel values. These are the compressed pixel values; S514, if Then update the quantization step size to ,and Return to step S51 to re-encode; until The final compression result is obtained. .

[0014] In the preferred scheme, the scene complexity value in step S12 A time decay factor was also introduced. Specifically: S121. The computer reads the timestamps of the current frame and the previous frame and calculates the time interval. : ; in, This represents the time interval between the current frame and the previous frame, obtained by the difference in timestamps from the camera. Indicates the timestamp of the current frame; Indicates the timestamp of the previous frame; S122, Calculate the time decay factor : ;in, This represents the time decay factor, with a value range of [0,1]. Indicates the attenuation coefficient. ; S123, Update the scene complexity value :; ;in, This represents the updated scene complexity value, with a range of [0,1]. , These are the weighting coefficients for occlusion rate and motion density, respectively. S124. Computer execution computing power allocation: Set frame interval threshold. In step S13, based on the updated scene complexity value Allocate edge computing power: when hour, Follow Increase and decrease, updated scene complexity value Synchronization is reduced, and the computing power allocated to this video stream by the computer is decreased. Proportion, and ;when At that time, the updated scene complexity value Maintain the original level and keep the computing power allocation unchanged.

[0015] This invention provides a method for recognizing continuous movements of construction workers at smart construction sites. Through the coordination of the aforementioned structures, it offers the following advantages compared to existing methods: First, it can effectively cope with complex environmental interferences such as rain, fog, dust, and sudden changes in lighting at construction sites, avoid target ambiguity caused by the failure of single-modal perception, and thus significantly improve the robustness of continuous human action recognition in complex environments, reducing missed judgments and misjudgments caused by environmental factors.

[0016] Secondly, by focusing on key frames of behavior through a temporal attention mechanism and constructing an action state transition matrix to complete the transition frame features, it not only solves the problem of insufficient generalization ability of the model in rare illegal actions and new construction scenarios, but also makes up for the feature incompleteness caused by the loss of continuous action transition frames. It can accurately distinguish similar actions such as bending over to pick up objects and illegal climbing, and significantly reduce the false alarm rate.

[0017] Third, an edge-cloud collaborative inference architecture is adopted. Lightweight models are deployed at the edge to filter most compliant scenarios and only upload suspected illegal fragments. The cloud processes complex inference through model sharding and load balancing. At the same time, it effectively balances the high computing power requirements of continuous action parsing with the real-time limitations of the edge, ensuring second-level response in high-risk scenarios and avoiding security risks caused by inference delays.

[0018] Fourth, a dynamic knowledge graph is constructed, which combines UWB positioning with real-time updates of scene parameters in the BIM model to achieve accurate association between continuous personnel actions and dynamic construction scenes. This breaks through the limitations of traditional static rules that cannot adapt to changes in scenes such as temporary high-altitude platforms and moving dangerous areas, and avoids misjudgment of action-scene association due to lag in scene parameters. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a main view structural diagram of the process of this invention; Figure 2 This is a deployment diagram of the present invention. Detailed Implementation

[0020] To better understand the purpose, system architecture, and functional implementation of this embodiment, the embodiments and features in the embodiments of this application can be combined with each other without conflict. The exemplary embodiments disclosed in this application will be described below with reference to the accompanying drawings, which include specific technical details disclosed in this embodiment to aid understanding; however, these details should be considered exemplary rather than restrictive. Therefore, those skilled in the art should understand that various improvements and adjustments can be made to the embodiments described herein without departing from the scope and core ideas of the invention. Similarly, for clarity, detailed descriptions of well-known technologies, functions, and structures (such as standard image processing algorithms and common communication protocols) are omitted in the following description.

[0021] With the development of intelligent construction technology, construction site personnel behavior recognition technology has gradually expanded from single visual perception to comprehensive solutions integrating multimodal fusion and edge-cloud collaboration to adapt to extreme construction environments and large-scale management needs. Multimodal perception technology can integrate multi-source data such as visible light, infrared thermal imaging, millimeter-wave radar, and UWB positioning, enhancing target perception capabilities in complex environments through complementarity; edge-cloud collaborative architecture can balance the computational power requirements and real-time requirements of model inference, achieving efficient processing of massive amounts of data. However, existing related technologies still have significant limitations in practical applications: some solutions rely on a single visual modality, have weak anti-interference capabilities, and are difficult to cope with extreme environments such as rain, fog, and nighttime; some solutions use heavy temporal models with large parameter counts and high computational consumption, failing to meet the real-time inference requirements of the edge; some solutions have fixed scene association rules, making it difficult to adapt to dynamic construction scenarios such as temporary high-altitude platforms and moving hazardous areas; at the same time, most solutions have not optimized for the loss of transition frame features in continuous actions, resulting in insufficient accuracy in distinguishing similar actions (such as bending over to pick up an object versus illegal climbing), leading to a high false alarm rate.

[0022] When addressing the problem of recognizing personnel behavior on construction sites, existing technologies either suffer from poor anti-interference capabilities due to limitations in single-modal perception, insufficient real-time performance due to a lack of lightweight optimization, weak scene adaptability due to reliance on static rules, or misjudgment of similar actions due to a failure to address transition frame feature completion. Therefore, these technologies struggle to simultaneously meet the comprehensive requirements of robustness in complex environments, real-time performance at the edge, dynamic scene adaptability, and the ability to distinguish similar actions, thus failing to fully support the refined management requirements for personnel safety in smart construction sites.

[0023] Example 1 like Figure 1 As shown, S11 allocates an independent GStreamer decoding thread to each RTSP camera, and performs hardware-accelerated decoding of the RTSP stream based on the VPU hardware decoding unit of the RK3588 platform, controlling the decoding thread to be greater than or equal to The frame rate is used to decode the RTSP stream, and the decoded video frames are stored in a ring buffer to obtain a temporally continuous frame sequence. ,in, Number of frames; S12. Calculate the scene complexity value for each video stream. The calculation formula is: ;in, The value represents the scene complexity, ranging from [0,1]; the occlusion rate represents the percentage of pixels in the current frame where people are occluded by the device, ranging from [0,1]; the action density represents the ratio of the number of people in the current frame who are performing dynamic actions (such as climbing illegally or raising their hands without wearing a helmet) to the total number of people in the frame, ranging from [0,1]. Specifically, in step S12, the "scene complexity value" "A time decay factor was also introduced." : S121, Calculate the time decay factor : ; in, This represents the time decay factor, with a value range of [0,1]. This represents the time interval between the current frame and the previous frame, obtained by the difference in timestamps from the camera. S122, Update the scene complexity value: ; in, This represents the updated scene complexity value, with a range of [0,1]. S123, Computing power allocation adaptation: In step S13, according to... Allocate edge computing power: when hour, , Follow The computational power allocated to this video stream is reduced by 10% as the load decreases. hour, , Maintaining the original level and keeping the computing power allocation unchanged; through this optimization, the edge computing power utilization rate in low frame rate scenarios is increased by ≥10%, avoiding computing power waste.

[0024] S13. Based on the scene complexity value Allocate edge computing power: when When identified as a high-complexity scenario, allocation Proportional computing power; when The scenario was determined to be of medium complexity, and the allocation was... Proportional computing power; when When the scenario is determined to be low complexity, the allocation is... Proportional computing power; obtaining the frame sequence after computing power allocation. ;in, This is a high complexity threshold, with a value range of [0.6, 0.8]. This is the standard complexity threshold, with a value ranging from [0.3, 0.5]. ; All values ​​represent computing power allocation ratios, ranging from [0,1]. .

[0025] In one feasible approach, when The value is 0.7. It was determined that 60% of edge computing power would be allocated to high-complexity scenarios; when The value is 0.3, when It was determined that 30% of the edge computing power would be allocated to medium-complexity scenarios.

[0026] The specific steps of step S2 are as follows: S21. Construct the Pseudo-3D feature extraction module for frame sequences. Perform the following operations: S211, Temporal Dimension 1D Convolution: Using a convolution kernel size of... ,padding is 3D convolutional layers, for Temporal features were extracted from five consecutive frames to obtain temporal features. ; S212, Spatial Dimension 2D Convolution: Using a convolution kernel size of... ,padding is 3D convolutional layers, for temporal features Spatial feature extraction is performed on each frame to obtain spatiotemporal joint features. ; S213, Spatiotemporal Joint Features Perform channel-level time-shift operation: One-quarter of the channels are shifted one frame to the left, one-quarter of the channels are shifted one frame to the right, and the remaining half of the channels remain unchanged, thus obtaining the time-shifted enhancement feature. ; Specifically, step S213, "TSM time-shift processing," also includes channel weight adjustment: S2131. Assign weights to the time-shifted channel features: set the weight of the time-shifted channel to the left to 0.3, the weight of the time-shifted channel to the right to 0.3, and the weight of the fixed channel to 0.4. S2132, Calculate time-shift enhancement features Weighted fusion results:

[0027] in, This represents the weighted time-shift enhancement feature; The channel characteristics indicate a leftward time shift; Channel characteristics indicating a rightward time shift; Indicates a fixed, stationary channel characteristic; S2133, will As input to the student network in step S22, it enhances the ability to distinguish time-dimensional features, thereby improving the accuracy of extracting time-series features of violations.

[0028] S22. Construct a teacher-student distillation network to... Perform knowledge distillation and mixing precision training: S221, Teacher Network: Employs the SlowFast-8x8 model to output soft targets. Through temperature The softmax function is used to process the soft target, resulting in a soft target: ; S222, Student Network: Employs ResNet-18 model with time-shifted input enhancement features. Output Calculate the soft target cross-entropy loss: ; in, This represents the knowledge distillation loss, with a value range of [0, +∞). This represents the raw predicted value output by the teacher network. This represents the raw predicted value output by the student network; This indicates a softened objective in the teacher's online output; S223. Path-specific quantization: Symmetric quantization is applied to the Slow path of the student network, with the quantization range set to... Asymmetric quantization is used for the Fast path, with the quantization range set to [0, 255]. S224. Mixed Precision Training: The student network is trained using FP16 / FP32 mixed precision, and a dynamic loss scaling factor is set. The initial value is When there is no gradient overflow for 1000 consecutive steps, Increase by a factor of 2; when gradient overflow occurs, Reduce by a factor of 2, the formula is: ; in, Indicates the first The loss scaling factor for each step. Indicates the first Loss scaling factor for each step; After the above processing, lightweight time-series features are obtained. ; Specifically, in step S224, "mixed precision training" also includes gradient clipping: S2241. Calculate the gradient norm of the student network parameters. : ; in, Represents the gradient value of a single parameter; Represents the global gradient norm; S2242, when At that time, perform gradient clipping: ; in, This represents the gradient after clipping. By clipping the gradient, the gradient explosion problem in mixed precision training is avoided, the training convergence speed of the student network is improved, and the Top-1 accuracy on the Kinetics-400 site violation action subset is maintained.

[0029] S23. Construct a transition frame feature completion model, for Complete the missing transition frames: S231. Based on the temporal logic of the violation actions (e.g., "violation of climbing = raising hand → grabbing the bar → exerting force → rising"), establish an action state transition matrix. ; In step S231, "an action state transition matrix is ​​established based on the timing logic of the violation action". Specifically, it includes: S2311. Collect time-series samples of typical violations at the construction site: 1000 sets of samples of illegal climbing and 800 sets of samples of not wearing safety helmets. Each set of samples contains 5 consecutive action states, namely the initial state. (e.g., a stationary state before a person takes any action), a ready state (e.g., the preparatory action of raising one's hand to grab the pole before climbing), execution state (such as the upward movement of the body during climbing), transitional state (such as a short pause to exert force during climbing), completed state (e.g., the stationary state after climbing to the target height); S2312. Count the number of state transitions for each sample group: For 1000 samples of illegal climbing, count the number of state transitions from... , Count the number of occurrences of all state transition combinations; similarly, count the number of state transitions in 800 groups of samples where no helmet was worn; combine the statistical results of the two types of samples to obtain the total number of state transitions. S2313. Calculate the transition probabilities and construct the matrix. For any two states and ( ), transition probability This yields a 5×5 action-state transition matrix. ; S2314, Regarding the matrix Perform normalization: ; in, This represents the normalized transition probability; Normalization ensures the matrix The sum of the elements in each row is 1. As the action state transition matrix used for transition frame prediction in step S231 This improves the accuracy of feature prediction for transition frames.

[0030] When lightweight features are detected Characteristic fractures exist in it, using Predicting transition frame features: in, To predict transition frame features, the temporal features are obtained after completion. .

[0031] S232, When lightweight features are detected There are characteristic fractures in it, which can be identified through the matrix. Predicting transition frame features: ; in, Indicates the predicted transition frame features, Indicates the first Temporal characteristics of frames, Indicates the first Temporal characteristics of frames, Represents the action state transition matrix; Obtain the completed temporal features ; The specific steps of step S3 are as follows: S31. Calculate the hierarchical contrast loss and perform a comparison of the completed temporal features. Enhanced similarity action features: S311. Constructing semantic attention weights First, extract the semantic embedding vectors of the violation actions from the pre-trained BERT model. Calculate image patches eigenvectors and cosine similarity : ;in, Represents image blocks The cosine similarity to the action semantics, with values ​​ranging from [-1, 1]; Represents image blocks The feature vector has a dimension of 512; The semantic embedding vector representing the violation action has a dimension of 512; Represents the vector dot product. The L2 norm of a vector; Then to Normalization yields : ;in, This represents the semantic attention weight, with a value range of [0,1]. This indicates the number of image blocks in a single frame, set to 32. S312, Calculate the spatial cycle consistency loss : ; in, This represents the spatial cycle consistency loss, with a value range of [0, +∞). Indicates and The most similar soft nearest neighbor image patch; Represents the cosine similarity function; S313. Calculate the total contrast loss: ; in, This represents the total contrast loss, with a value range of [0, +∞). Represents cross-entropy loss; Indicates global comparison loss; Indicates time-comparison loss; Indicates time cycle loss; Indicates spatial contrast loss; S32. Construct a dynamic GNN model to analyze the temporal features after completion. Multi-target interaction in: S321, Define the diagram structure :node Includes personnel nodes (characteristics) Personnel movement characteristics and equipment nodes (Features include device 3D coordinates) Scene nodes (Characteristics: boundary of dangerous area); edge Represents the interaction relationships between nodes; adjacency matrix elements ,in, For nodes and The Euclidean distance; S322. The computer initializes the GNN network with the following number of layers. Define the propagation rules according to the GNN propagation rules: ;in, Represents a node In the The feature vector of the layer; Indicates the first Layer weight matrix; Represents a node The set of neighboring nodes; Indicates the activation function; Specifically, the parameter optimization of the "GNN propagation rule" in step S322 includes: S3221, The GNN has 2 layers, and the weight matrix of the first layer is... The dimension is The second layer weight matrix The dimension is ; S3222. The activation function uses LeakyReLU instead of ReLU, and the formula is: ;in, This represents the input feature values ​​of the GNN layer; S3223, Output characteristics after propagation Perform L2 normalization: ;in, This represents the normalized node features; through the above optimization, multi-objective interaction features are obtained. The signal-to-noise ratio is improved, and the accuracy of multi-person collaborative violation identification is improved.

[0032] S323, Output multi-target interaction features ; S33. Causal Behavior Verification (Based on Reference 5): Verifying the causal relationship between actions and scenarios based on a structural causal model (SCM), defining intervention variables. Calculate causal probability With statistical probability The difference: ; in, It represents the difference between the causal probability and the statistical probability; Indicates intervention action The probability of post-scene matching; Indicates actions in a natural state The probability of matching the scene; if If the relationship is positive, it is determined to be a causal relationship; otherwise, it is determined to be a false relationship, and the process returns to step S32 to re-extract interaction features. The specific steps of step S4 are as follows: S41. Construct a BIM and UWB dynamic scene parameter library: S411. Real-time acquisition of the three-dimensional coordinates of the temporary high-altitude platform via UWB positioning sensors. Boundary coordinates of the danger zone It updates the BIM model once every second; Specifically, the UWB positioning sensor acquisition in step S411 also includes anti-interference processing: S4111. Kalman filtering is used to eliminate positioning noise. The state equation and observation equation are as follows: Equations of state: ; Observation equation: ; in, Indicates the first The platform position state vector at time (including x, y, z coordinates and velocity); Let represent the state transition matrix, denoted as the identity matrix; The process noise is represented by a mean of 0 and a variance of 1. Gaussian distribution; Indicates the first The observation location at that moment; Let the observation matrix be denoted as the identity matrix; The observation noise is represented by a mean of 0 and a variance of 1. Gaussian distribution; S4112. Filtered position data Perform a 5-frame moving average filter: ;in, This indicates the platform position after moving average; through anti-interference processing, UWB positioning accuracy is improved, and the scene parameter library is updated. The accuracy of the updates has been improved.

[0033] S412. Extracting scene parameters from the BIM model Build a dynamic scene parameter library ; S42. Comparison of executed actions with real-world scenes: S421, From Multi-Target Interaction Features Extracting personnel action types and 3D positions ; S422, If the action type "High-altitude operation" and This indicates a mismatch between the action and the scene, triggering an abnormal scene parameter check. Otherwise, it is determined to be a match, and a preliminary action recognition result is obtained. ; S43. Scene Parameter Anomaly Validation: When the action is determined to be mismatched with the scene, the laser rangefinder is activated to remeasure the height of the temporary high-altitude platform. ,like Then update The platform height in the middle is Return to step S42 and re-compare; The specific steps of step S5 are as follows: S51. Initial motion recognition results Perform ASE encoding / decoding and FEC packet loss recovery: S511, ASE encoding / decoding: (for...) Intra-frame prediction and entropy coding are performed on the feature map blocks to obtain the compressed result. ; Specifically, the implementation of "ASE encoding / decoding" in step S511 includes: S5111, Intra-frame prediction: Employing a 4×4 block prediction mode, the action recognition results are... The feature map is divided into multiple 4×4 pixel blocks; for each target block, the adjacent blocks above and to the left are selected as reference blocks, and the pixel value of the target block is predicted by the average pixel value of the reference blocks. The formula is: ;in, Represents the predicted pixel value of the target block; This indicates the pixel value of the reference block above; This represents the pixel value of the left reference block; Calculate the actual pixel value of the target block and The difference is used to obtain the prediction residual; S5112, Entropy Coding: Context-based adaptive arithmetic coding is used to encode the prediction residuals and quantization parameters; the coding window size is set to 16 bytes, that is, 16 bytes of data are read and encoded each time, and the coding probability is dynamically adjusted through the context model to improve coding efficiency; S5113, Compression Verification: Calculate the Peak Signal-to-Noise Ratio (PSNR) of the data before and after compression. Where PSNR represents peak signal-to-noise ratio; MSE represents the mean squared error of the data before and after compression, and the formula is: ,in, For feature map height, The width of the feature map. These are the original pixel values. These are the compressed pixel values; S5114, Quantization parameter adjustment: If If the quantization step size is reduced by 20%, return to step S5111 to re-execute intra-frame prediction and coding; repeat the adjustment until... The final compression result is obtained. This ensures that the visual quality of the compressed data meets the requirements for action recognition.

[0034] S5115, FEC packet loss recovery: for Add a 10% longer FEC checksum. Linear block code ( The receiving end recovers lost data using a checksum, ensuring a packet loss rate of ≤1%.

[0035] S512, FEC packet loss recovery: [Regarding] Add an FEC checksum; the checksum length is [length missing]. 10% of the data length, using Linear block codes, where ( for (data length) S513, Compression Verification: Calculate the peak signal-to-noise ratio (PSNR) of the data before and after compression. Reduce the quantization step size and re-encode until... The final compression result is obtained. ; S52, Output To the application end: If The system includes "violations of regulations" (such as not wearing a helmet or climbing illegally), triggering an alarm with sound and light; simultaneously... Store in a local database and retain for at least 30 days.

[0036] Example 2 This embodiment targets small and medium-sized construction sites, deploying only one edge gateway to achieve personnel behavior recognition from eight cameras. It verifies the technical effectiveness of lightweight temporal parsing and single-node dynamic scene management, covering functions such as concurrent processing of multiple cameras, transition frame completion, and differentiation of similar actions.

[0037] In this embodiment, the hardware selection is as follows: Edge gateway: Rockchip RK3588, equipped with 8GB LPDDR4 memory and 64GB eMMC storage; Surveillance cameras: Hikvision DS2CD3T46WDV3I5, deployed in 8 key areas such as construction site entrances and exits and around scaffolding, with a spacing of 25 meters and a height of 4 meters; Auxiliary equipment: UWB positioning sensor and laser rangefinder. The software environment is as follows: Operating System: Ubuntu 18.04LTS (ARM64 architecture); Deep Learning Framework: PyTorch 2.0.1; Dependencies: OpenCV 4.8.0 (for image processing), PyTorchGeometric 2.3.1 (for GNN computation), PyFEC 1.4.0 (for FEC packet loss recovery).

[0038] In this embodiment, the specific steps are as follows: Step 1: Preprocessing concurrent data from multiple cameras In this embodiment, the edge gateway accesses eight camera video streams via the RTSP protocol, and the computer performs the following operations: Specifically, S11, multi-threaded decoding and buffering: The computer calls the gstlaunch1.0 command to initialize the GStreamer decoding pipeline, allocating an independent thread to each camera and configuring the nvdec hardware decoding element parameters: the bitrate control mode is VBR (Variable Bitrate), the I-frame interval is 25 frames, and the pixel format is converted to YUV420p; at the same time, the ringbuffer buffer queue is initialized, the buffer depth is set to 10 frames, and read and write synchronization is achieved through pthread_mutex_lock and pthread_mutex_unlock. When the difference between the write pointer ptr_w and the read pointer ptr_r is ≥10, the write operation is paused and the read operation is triggered to ensure the continuity of the frame sequence, ultimately resulting in 8-channel frame sequences. .

[0039] S12, Scene Complexity Calculation: For each frame in the frame sequence The computer extracts the human region within the HSV color gamut using OpenCV's cv::inRange method. Skin color ranges are H: 0-180, S: 40-255, V: 40-255. The occlusion rate is calculated as (number of occluded pixels / total number of human pixels). The ByteTrack target tracking algorithm is called, with a confidence threshold of 0.5 and a maximum tracking loss frame count of 5 during tracker initialization. The number of human IDs for dynamic actions (such as raising a hand or climbing) within each frame is counted, and the action density is calculated as (number of dynamic human IDs / total number of human IDs). Substituting these values ​​into the formula... The complexity value of the 8-way scene was calculated. The value range is 0.2-0.8.

[0040] S13. Dynamic allocation of edge computing power: computer preset threshold , Call nvidiasmi to get the total available GPU computing power (approximately 15 TOPS); 60% of the computing power (9 TOPS) is allocated to the two cameras (around the scaffolding). The 4 cameras (entrance and exit cameras) are allocated 30% of the computing power (4.5 TOPS). Two cameras (open ground cameras) are allocated 10% of the computing power (1.5 TOPS). An inference container is created for each camera using Docker container scheduling. When the GPU utilization of a container exceeds 90%, 5% of the computing power is supplemented from low-complexity scenes. The final output is an 8-channel frame sequence after the computing power allocation. .

[0041] Step 2: Lightweight Timing Analysis of Continuous Actions In this embodiment, the computer uses Taking (the frame sequence of cameras around the scaffolding) as an example, perform the following operations: Specifically, S21 and P3D feature extraction and TSM time shift: The computer initializes the Pseudo3D module and loads the model parameters pre-trained on the Kinetics400 construction site violation action subset (including 10 types of actions such as not wearing a safety helmet and illegal climbing, totaling 50,000 frames): temporal dimension 1D convolution kernel size (5,1,1), padding (2,0,0), stride (1,1,1); spatial dimension 2D convolution kernel size (1,3,3), padding (0,1,1), stride (1,1,1); for Perform temporal convolution on 5 consecutive frames to output temporal features. (dimension) ); and then to Perform spatial convolution to obtain spatiotemporal joint features. ; In the preferred scheme, the computer calls torch.chunk to... The 64 channels are divided into 4 parts (16 channels each). The first part is shifted 1 frame to the left using torch.roll(shift=1,dims=3), the second part is shifted 1 frame to the right using torch.roll(shift=1,dims=3), and the 34th part remains unchanged. After concatenation using torch.cat, the data is normalized using nn.BatchNorm3d(64) to obtain the time-shifted enhanced features. (dimensions and) Consistent).

[0042] S22, Knowledge Distillation and Mixing Precision Training: In this embodiment, the teacher network uses SlowFast8x8, with a Slow path sampling rate of 8 and a Fast path sampling rate of 2. (Input...) Output Set temperature parameters Calculate soft targets The student network uses ResNet18, and the input... Output Calculate the cross-entropy loss of soft targets. ; In practice, the computer is configured with FP16 / FP32 mixed precision, and the loss scaling factor is initialized. Check for gradient overflow every 1000 steps: if there is no overflow for several consecutive steps, then... Double the value, halve it if it overflows; simultaneously perform gradient clipping, setting the gradient norm threshold to 10.0. hour, The Adam optimizer was used for 10,000 iterations to finally obtain the quantized lightweight temporal features. .

[0043] S23. Transition Frame Feature Completion: In this embodiment, the computer defines 5 action states based on the illegal climbing timing logic: (still), (Raises hand) (Grab bar) (Exerting force) (Ascending); Load 1000 climbing samples, count the number of state transitions, and obtain the transition matrix. ;pass Normalization; In one feasible approach, the computer... Perform frame continuity detection: Calculate the timestamp difference between consecutive frames. If the difference is greater than 0.5 seconds, it is determined to be a feature break, such as the absence of frame 101 between frame 100 and frame 102; substitute into the formula. Predict transition frame features; insert The completed temporal features are obtained. .

[0044] Step 3: Similar Action Feature Enhancement and Multi-Target Interaction Analysis In this embodiment, S31, hierarchical contrastive loss calculation: The computer loads the bertbaseuncased model and extracts the semantic embedding vector of "illegal climbing". Calling torch.nn.Unfold(kernel_size=16, stride=16) will... Each frame is divided into 196 16×16 image blocks, and the feature vector of each block is extracted using ResNet50. ;calculate Normalization yields ; Specifically, computer screening and Most similar Calculate the spatial cycle consistency loss ;set up , , Calculate the total loss The model parameters are updated using torch.autograd.backward to improve the distinguishability of similar action features.

[0045] In this embodiment, S32, Dynamic GNN Multi-Objective Interaction Analysis: Computer-Defined Graph Structure Personnel Node (characteristics) Action characteristics and device nodes (3D coordinates of the scaffolding, obtained via UWB), scene nodes (Danger zone boundary, read from BIM model); Calculate the Euclidean distance between nodes using scipy.spatial.distance.cdist. ,set up ; In the preferred scheme, the GNN network is configured with two layers: layer 1 is GATConv(512,256), layer 2 is GATConv(256,128), and the activation function is LeakyReLU (negative slope 0.01); the propagation rule is... ,in, for The neighboring nodes; input Perform two-layer propagation and output multi-target interaction features. .

[0046] S33. Causal Behavior Verification: The computer constructs an SCM model based on the DoWhy library, with exogenous variables... For illumination noise, an endogenous variable Personnel position (x, y, z) and action type; simulated intervention ,calculate ; in natural state , If a causal relationship is determined, output... .

[0047] In this embodiment, step 4: dynamic scene association and virtual-real comparison S41. Construction of BIM and UWB Dynamic Scene Parameter Library: UWB positioning sensors are deployed on the top of the scaffolding, and the computer reads the three-dimensional coordinates through the UWBSDK. ; Call the Revit API to load the site BIM model and obtain the BIM origin. With rotation matrix ; Specifically, perform coordinate transformation: Obtain BIM coordinates Update the BIM model every second and extract scene parameters. Build a parameter library .

[0048] S42. Real-world motion scene comparison: The computer... Extracting action types Personnel BIM coordinates ; query have to ,calculate The action scenario is determined to be mismatched, triggering an exception check. Start the laser rangefinder and read the actual height of the scaffolding. ,calculate If the parameters are found to be normal, compare them again: Still no match, output preliminary results. .

[0049] Step 5: Result Output and ASE Encoding / Decoding Transmission S51, ASE Encoding / Decoding and FEC Packet Loss Recovery: The computer calls the ASE encoding / decoding interface to... (dimension) The target pixel value is predicted by dividing the image into 4×4 pixel blocks and using the mean of neighboring blocks to obtain the prediction residual. Context-based adaptive arithmetic coding (16-byte coding window) is used to encode the residual with a compression ratio of 2:1. (dimension) ); In this embodiment, a 10% FEC checksum is added, and a (563,512) linear block code is used; transmission is achieved through a 5G module (100Mbps). The receiving end packet loss rate is 2%. Data is recovered using checksums. Output .

[0050] S52. Result Output and Alarm Linkage: Computer will Stored in a MySQL database, then pushed to the monitoring platform via WebSocket; because The device is labeled "Unauthorized Climbing". An HTTP POST command is sent to the audible and visual alarm, triggering an 85dB audible and visual alarm. At the same time, an SMS message is sent to the safety officer via SMSAPI: "2024100114:30, unauthorized climbing detected in the scaffolding area, personnel location (10.2,5.3,3.1)".

[0051] Example 3 This embodiment targets large-scale power grid infrastructure construction sites such as 500kV substations. These scenarios are characterized by wide construction areas, clearly defined functional zones (e.g., construction areas, material storage areas, office areas, equipment installation areas), high personnel / equipment mobility, and a high demand for multi-area collaborative operations. To achieve comprehensive, blind-spot-free control, a collaborative system consisting of three edge nodes and one cloud server was deployed. The system verifies the effectiveness of technologies such as dynamic scheduling of multi-edge computing power, distributed time-series analysis, cross-regional scene correlation, and FEC packet loss recovery. It covers functions such as concurrent processing of multiple cameras, edge-cloud collaborative inference, and multi-target interactive analysis, ensuring the real-time performance and accuracy of cross-regional personnel behavior recognition.

[0052] In this embodiment, the hardware selection needs to meet the requirements of cross-regional coverage and multi-edge collaboration in large construction sites. The specific configuration is as follows: Edge nodes: 3 NVIDIA Jetson AGX Orin edge gateways, each equipped with an 8-core ARM Cortex A78AE CPU, NVIDIA Ampere architecture GPU, 32GB LPDDR5 memory, and 256GB NVMe SSD storage; The deployment locations are as follows: Edge 1: Construction area (main transformer installation area of ​​substation), responsible for connecting 8 cameras (covering the main transformer foundation and high-voltage area); Edge 2: Material storage area (cable / tower component storage area), responsible for connecting 8 cameras (covering material loading and unloading and temporary storage area); Edge 3: Office area and equipment installation area (control room equipment installation), responsible for connecting 8 cameras (covering office entrance and control room). Cloud server: Huawei Cloud ECS Elastic Cloud Server, used for global model management and cross-regional data fusion; Surveillance cameras: 24 Hikvision DS2CD3T86WDV3I5 network cameras, with the following deployment parameters for each camera: Cameras in the construction area cover high-risk areas such as main transformer installation and busbar erection; cameras in the material area cover material loading and unloading and temporary storage boundaries; cameras in the office / equipment area cover personnel entrances and exits and the entrance to the control room. Auxiliary sensing equipment: UWB positioning base stations: 6 Decawave DW1000UWB base stations, deployed at the four corners and center of the substation to build a global positioning network; Laser rangefinders: 4 Leica DISTOD510, deployed at the boundaries of the material area and construction area for dynamic scene parameter calibration; Transmission equipment: 3 Huawei 5GCPEPro3 industrial routers, connected to 3 edge nodes respectively, to achieve low-latency communication between the edge and cloud and between edges.

[0053] The specific steps of this embodiment are as follows: Step 1: Concurrent Data Preprocessing for Multiple Edge Nodes In this embodiment, the three edge nodes achieve frame data collaboration through a Kubernetes Edge cluster. Taking edge 1 (construction area) as an example, the specific execution is as follows: Initialize decoding threads: The computer calls the gstlaunch1.0 command to create 8 independent GStreamer decoding pipelines; Ringbuffer configuration: Initialize a ringbuffer with a depth of 12 frames. When the difference between the write pointer ptr_w and the read pointer ptr_r is ≥10, the frame reading logic is triggered. If a camera loses 3 consecutive frames, edge 1 automatically sends a frame request to the adjacent edge node (such as edge 2). Edge 2 queries the corresponding frame data through the Redis cache and sends it back to ensure the temporal integrity of the frame sequence. Frame preprocessing: The decoded BGR image is scaled and normalized proportionally to obtain an 8-channel normalized frame sequence. .

[0054] This step requires aggregating the scene complexity of multiple edge nodes through the cloud to achieve optimal global computing power allocation. Specifically, it is executed as follows: Single-edge scene complexity calculation: Edge 1 pairs each frame sequence... The scene complexity is calculated in the following way. : 1. Occlusion rate calculation: OpenCV's cv::findContours is used to extract personnel and equipment contours. The equipment contours are obtained through a pre-trained YOLOv5 equipment detection model, and the categories include tower cranes, cable reels, and scaffolding. Occlusion rate = (intersection area of ​​personnel contours and equipment contours / total area of ​​personnel contours). 2. Motion density calculation: The ByteTrack target tracking algorithm is called to count the number of personnel IDs with dynamic actions in each frame. Motion density = (number of dynamic personnel IDs / total number of personnel IDs). 3. Complexity formula: The calculation shows that there are 8 roads in the construction area. The value ranges from 0.32 to 0.89; Cross-edge complexity summary: Edge 1 uses the NodeStatus interface of KubernetesEdge to... The computing power utilization (GPU utilization, CPU utilization) of the edge nodes is uploaded to the cloud once every 5 seconds; similarly, Edge 2 (materials area) and Edge 3 (office / equipment area) simultaneously upload data for their respective areas. Value and computing power status; Global computing power allocation in the cloud: A computing power scheduling module is deployed in the cloud and executes the following logic: 1. Total computing power calculation: Total GPU computing power of 3 edge nodes = 200 × 3 = 600 TOPS, with 10% redundant computing power reserved (60 TOPS), and allocable computing power = 540 TOPS; 2. Determining Regional Weights: Based on The mean value determines the region weight, and the formula is as follows: In this embodiment, , , Calculated , , ; 3. Computing power allocation results: Construction area (edge ​​1) allocated 540×0.5=270TOPS (1.35x GPU computing power scheduling enabled, GPU frequency adjusted via NVIDIA SMILGC1350), material area (edge ​​2) allocated 540×0.35=189TOPS, office area (edge ​​3) allocated 540×0.15=81TOPS; Computing power command issuance and execution: The cloud sends computing power allocation commands to each edge node through the NodeConfig interface of KubernetesEdge; after receiving the command, Edge 1 adjusts the resource quota of the 8-camera inference container by using dockerupdatecpus=6memory=24g (40% computing power is allocated to the cameras installed on the main transformer in the construction area, and 60% is allocated to the rest) to ensure sufficient computing power in high-complexity areas.

[0055] Step 2: Lightweight Timing Analysis for Multi-Edge Collaboration This embodiment adopts a mode of edge local feature extraction and cloud feature fusion to avoid full data transmission. The specific execution is as follows: Edge Local P3D Feature Extraction: Frame Sequence After Each Path of Computing Power Allocation for Edge 1 Initialize the Pseudo3D (P3D) module with the following parameter configuration: Temporal Dimension 1D Convolution: Kernel Size ,padding stride Output channels: 64; Spatial dimension: 2D convolution; Kernel size: ,padding stride The number of output channels is 64; the PyTorch nn.Conv3d interface is called to... Perform temporal convolution on 5 consecutive frames, then perform spatial convolution on the output to obtain the spatiotemporal joint features. That is, the dimension is , For batch size, ); TSM time-shift enhancement: Edge 1 pair Perform channel-level time shift: Call torch.chunk to split the 64 channels into 4 parts (16 channels each). The first part is shifted 1 frame to the left (torch.roll(shift=1,dims=4)), the second part is shifted 1 frame to the right (torch.roll(shift=1,dims=4)), and the 34th part remains unchanged. After concatenation by torch.cat, the time-shifted enhanced features are obtained by normalization with nn.BatchNorm3d(64) and activation with nn.ReLU(). ; Cross-edge feature compression transmission: edge 1 pair Perform channel pruning (retaining 60% of core channels, filtered using L1 regularization), then apply FP16 precision compression, and upload to the cloud via 5G network (transmission rate approximately 80Mbps, latency ≤20ms); edge devices 2 and 3 simultaneously upload to their respective regions. The cloud-based torch.cat is used to concatenate the data according to regional weights to obtain cross-region time-shift features. .

[0056] To improve the accuracy of the lightweight model for edge nodes, a distributed distillation architecture of cloud-based teacher network and edge student network is adopted, specifically as follows: Cloud-based teacher network deployment: A pre-trained SlowFast8x8 model is loaded into the cloud for fine-tuning, with the Slow path sampling rate set to 8 and the Fast path sampling rate to 2. Input... Output soft target ; through temperature parameters Calculate soft targets: ; Edge Student Network Training: Deploy a ResNet18 student network at Edge 1, inputting local... Output Calculate the soft target cross-entropy loss: ; Simultaneously enable FP16 mixed-precision training (torch.cuda.amp.autocast) and initialize the loss scaling factor. Check for gradient overflow every 1000 steps: if there is no overflow for several consecutive steps, then... Double the amount; if it overflows, halve the amount. Gradient Aggregation and Parameter Update: Edge 1 Calculation gradient The gradients are uploaded to the cloud via the torch.distributed.all_reduce interface. The cloud uses a federated averaging algorithm to aggregate the gradients of the three edge nodes (with weights equal to the proportion of data volume of each edge node) and update the teacher network parameters. Every 5,000 iterations, the cloud distributes the teacher network soft objective and the updated parameters to the edge nodes, and the edge nodes fine-tune the student network. After 100,000 iterations, the student network's Top-1 accuracy on the power grid construction site dataset is improved to 93.8%, the number of parameters is compressed to 12M, and the inference speed at the edge reaches 32fps.

[0057] Due to frame synchronization discrepancies between edge nodes in large construction sites, transition frames need to be completed via the cloud. The specific execution is as follows: Transition frame missing detection: cloud-based Perform frame continuity detection: Calculate the cosine similarity of features of consecutive frames. If the similarity is <0.6 (determined as feature breakage), record the position of the missing frame. Cross-edge transition frame prediction: Loading pre-trained action state transition matrices in the cloud Training was conducted based on 1500 sets of samples of violations at power grid construction sites, using a 5×5 dimension, such as... The transition probability of "raising hand → grabbing bar" is used to predict the features of missing frames in the formula: ; in, Features of missing frames Features of the frame preceding the missing frame (edge ​​1, frame 150). Features of the frame following the missing frame (edge ​​1, frame 152). Feature completion distribution: Cloud will The message is sent to the corresponding edge node (edge ​​1), and edge 1 inserts it. The completed temporal features are obtained. .

[0058] Step 3: Cross-regional multi-objective interaction analysis A dynamic GNN model covering the entire construction site is built in the cloud to enable interactive analysis of personnel, equipment, and scenarios across different regions. The specific execution is as follows: Node feature definition: 1. Personnel Nodes Feature dimension 512, including edge node uploads. (256-dimensional), UWB positioning coordinates (3-dimensional), personnel identification tag (1-dimensional, 0 = unauthorized / 1 = authorized), device interaction status (1-dimensional, 0 = no interaction / 1 = interaction). 2. Equipment Nodes Features have 512 dimensions, including device type (10 dimensions), device location (3 dimensions), and device working status (1 dimension, 0 = disabled / 1 = running). 3. Scene Nodes The feature dimensions are 512, including scene type (5 dimensions), scene danger level (1 dimension, 0=low / 1=medium / 2=high), and scene boundary coordinates (4 dimensions). Adjacency matrix construction: The cloud-based scipy.spatial.distance.cdist is used to calculate the Euclidean distance between any two nodes. Define the elements of the adjacency matrix: ;in, The interaction distance threshold is used for cross-regional nodes (such as personnel in the construction area and equipment in the material area). Nodes within the region ; The cloud-based GNN model employs a 2-layer GAT (GraphAttention Network) structure, which is implemented as follows: First layer GAT: Input node feature dimension 512, output dimension 256, 8 attention heads, activation function is LeakyReLU (negative slope 0.01), propagation rule is: ;in, Let be the weight of node ij of the k-th attention head. Let be the weight matrix for the k-th attention head; The second layer of GAT has an input dimension of 256, an output dimension of 128, 4 attention heads, and the same propagation rule as above. The final output is the global multi-objective interaction feature. (dimension) , (Total number of nodes across the entire construction site). To avoid spurious associations in cross-regional scenarios, perform causality checks: SCM Model Construction: Building a Structural Causal Model (SCM) based on the DoWhy library, defined as follows: Exogenous variables Environmental noise, with a value range of 0 to 0.3; Endogenous variables Personnel action type, personnel location, equipment status, and scene type; Causal function The logical relationship is: "personnel actions → device interaction → scene matching"; Intervention Calculation and Verification: Simulated Intervention (The type of forced personnel action is) ), calculate causal probability Simultaneously calculate the statistical probability under the state of nature. ,like If the interaction is positive, it is considered a causal relationship; otherwise, it is considered a false relationship and the interaction feature is removed. In this embodiment, the construction area personnel , , , If the interaction is deemed a genuine violation, the interaction feature will be retained.

[0059] Step 4: Cross-regional dynamic scene association and virtual-real comparison The cloud-based system, based on BIM, UWB, and laser ranging, constructs a cross-regional scene parameter library, as detailed below: BIM model to UWB coordinate mapping: 1. BIM Model Import: Using the Autodesk Revit 2023 API, the substation BIM model is imported into the cloud, and the origin of the BIM coordinate system is obtained. With rotation matrix ; 2. UWB coordinate transformation: The UWB base station acquires the three-dimensional coordinates of the cable reel in the material area in real time. Convert to BIM coordinates using the following formula: ; get ; Scene parameter update and storage: Every second, the coordinates after UWB conversion and the material area boundary measured by the laser rangefinder are updated to the MongoDB database to build a dynamic scene parameter library. The parameters include: Area ID: Material Area (ID=2); Scene Type: Material Stacking Area; Hazard Level: Medium (Level 1); Boundary Coordinates: Equipment parameters: Cable reel location Quantity = 5; Update timestamp: 2024101509:45:32.

[0060] Cross-regional motion scene virtual-real comparison: cloud-based Extract personnel movements and positions, and... The comparison is performed as follows: Feature extraction: Call torch.argmax from Extracting personnel action types Personnel BIM coordinates ; Parameter library query: based on personnel coordinates Query Obtain material region boundary ,determination ,and This triggers anomaly verification of scene parameters; Anomaly Verification: The cloud sends a laser ranging command to Edge 2 (material area), Edge 2 activates its laser rangefinder, and the left boundary of the material area is measured. ,calculate (Error threshold) Determine if BIM model boundary parameters are outdated and update them. The boundary of the material region is ; Re-comparison: Recalculation (In the materials area) Output preliminary recognition results .

[0061] Step 5: Result output and ASE encoding / decoding transmission. To ensure the integrity of cross-regional result transmission, the following steps are performed: ASE Intra-frame Prediction and Entropy Coding: 1. Intra-frame prediction: Divide the data into 8×8 pixel blocks and use a dual reference block (top + left) for prediction. The formula is as follows: The predicted residuals are obtained; 2. Entropy Coding: Context-based adaptive arithmetic coding is used with a 32-byte encoding window. The encoding probability is dynamically adjusted based on the statistical features of the first 20 bytes, achieving a compression ratio of 3:1, resulting in the compressed output. (dimension) ); FEC packet loss recovery: 1. Checksum Generation: Add a 15% FEC checksum, using a (786, 683) linear block code, the generator polynomial is as follows: ; 2. Transmission and Recovery: Transmitted via 5G network The receiving end (monitoring center server) uses the Viterbi decoding algorithm for decoding. When the packet loss rate is ≤8%, the packet loss recovery rate is ≥99%. In this embodiment, the transmission packet loss rate is 3%, and the recovery calculation is performed... Output .

[0062] Local storage and platform push: The cloud will The data is stored in MongoDB and simultaneously pushed to the construction site monitoring platform via the WebSocket protocol. The platform displays the location, action type, and time of violation of the personnel in real time. Multi-dimensional alerts: 1. Audible and visual alarm: Send an HTTP POST command to the audible and visual alarm at edge 2 (material area) to trigger a 90dB audible and visual alarm; 2. SMS Alert: Utilize Alibaba Cloud SMS API to send an SMS to the safety officer's mobile phone number (138XXXX1234): "[Power Grid Infrastructure Alert] 2024101509:46, Construction area personnel (ID:008) have entered the material area without authorization. Location: BIM coordinates (79.2, 61.5, 0.9). Please handle this promptly!" 3. Voice broadcast: Send a TTS voice command to the IP broadcast speaker in the materials area to play in a loop the message "Attention! Construction personnel have entered the materials area without authorization. Please intercept them!"

[0063] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0064] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for continuous motion recognition of construction site personnel applied to smart construction sites, characterized in that, Includes the following steps: S1. Acquire multiple RTSP cameras, allocate an independent decoding thread to each RTSP camera, decode the video stream and store it in a buffer to form a time-sequential frame sequence. Calculate the scene complexity value for each video stream, including a weighted calculation of occlusion rate and motion density; Edge computing power is allocated based on the scene complexity value, resulting in a frame sequence after computing power allocation. ; S2. Construct the Pseudo-3D feature extraction module and extract frame sequences using decompositional 3D convolution. The spatiotemporal joint features are obtained; lightweight temporal features are obtained through teacher and student network distillation and path splitting quantization; a transition frame feature completion model is constructed to complete the missing transition frames in the lightweight temporal features, resulting in the completed temporal features. ; Construct a transition frame feature completion model and establish an action state transition matrix based on the temporal logic of the violation action. When lightweight timing features are detected There are characteristic fractures in it, which can be identified through the matrix. Predicting transition frame features For lightweight time-series characteristics The missing transition frames are padded to obtain the padded timing features. ; S3. Calculate the effect of hierarchical contrast loss on the completed temporal features. Enhanced similarity action features; Construct a dynamic GNN model and analyze the completed temporal features. Multi-target interactions in the data are used to obtain multi-target interaction features. Verify the causal relationship between actions and scenarios using a structural causal model; To verify the causal relationship between actions and scenarios using a structural causal model, intervention variables are defined. Calculate causal probability With statistical probability The difference, Indicates the type of personnel action; S4. Construct a dynamic scene parameter library for BIM and UWB, execute actions and compare virtual and real scenes, and analyze multi-objective interaction features. The system extracts the type of human movement and the three-dimensional position of the human, obtaining preliminary movement recognition results. ; Perform scenario parameter anomaly checks for mismatches; S5. Initial motion recognition results Perform encoding / decoding and packet loss recovery, and output the final compressed result. On the application side, if any violations are detected, an audible and visual alarm will be triggered, and the identification results will be stored.

2. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Allocate an independent GStreamer decoding thread to each RTSP camera, perform hardware-accelerated decoding of the RTSP stream based on the edge device, and store the decoded video frames in a ring buffer to obtain a temporally continuous frame sequence. ,in, Number of frames; S12. Call the image analysis algorithm to read the temporally continuous frame sequence. From the pixel data of each frame, the occlusion rate and motion density are obtained, and the scene complexity value of each video stream is calculated based on the occlusion rate and motion density. ; S13. Based on the scene complexity value Allocate edge computing power: when When identified as a high-complexity scenario, allocation Proportional computing power; when The scenario was determined to be of medium complexity, and the allocation was... Proportional computing power; when When the scenario is determined to be low complexity, the allocation is... Proportional computing power; obtaining the frame sequence after computing power allocation. ; in, This is a high complexity threshold; This is the standard complexity threshold, and ; All values ​​represent computing power allocation ratios, ranging from [0,1]. .

3. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 1, characterized in that, The specific steps of step S2 are as follows: S21. Construct the Pseudo-3D feature extraction module, using a pre-trained 1D temporal convolutional kernel to extract features from frame sequences. Perform convolution on the time dimension features to obtain the time features. ; Employing spatial 2D convolution kernels to process temporal features Spatial feature extraction is performed on each frame to obtain spatiotemporal joint features. ; spatiotemporal joint features Perform channel-level time-shift operation: One-quarter of the channels are shifted one frame to the left, one-quarter of the channels are shifted one frame to the right, and the remaining half of the channels remain unchanged, thus obtaining the time-shifted enhancement feature. ; S22. Construct a teacher-student distillation network to... Perform knowledge distillation and mixing precision training: The teacher network uses a SlowFast-8x8 model, and the output is the raw predicted value from the teacher network. The softening target is calculated using the softmax function. ; The student network uses the ResNet-18 model with time-shifted input enhancement features. Output the raw predicted values ​​of the student network. The SoftTargetCrossEntropyLoss function is called to calculate the soft target cross-entropy loss. Symmetric quantization is used for slow paths in the student network, and asymmetric quantization is used for fast paths. The student network was trained using FP16 / FP32 mixed precision, and a dynamic loss scaling factor was set. The initial value is Each training session Step-by-step check for gradient overflow: if continuous No gradient overflow. Increase If gradient overflow occurs, Decrease times; Lightweight temporal features are obtained through backpropagation and parameter update. .

4. The method for continuous motion recognition of construction site personnel applied to smart construction sites according to any one of claims 1 to 3, characterized in that, The specific steps of step S3 are as follows: S31. Extract the semantic embedding vectors of the violation actions from the pre-trained BERT model. For the completed temporal features Each frame is split into Image blocks Extract each image patch eigenvectors Calculate image patches eigenvectors and cosine similarity ; Cosine similarity Normalization is performed to obtain semantic attention weights ; Get image patch pairs for each frame According to image patch pairs Calculate the spatial cycle consistency loss ; Integrate the contrast loss from global, temporal, and spatial dimensions to calculate the total contrast loss. ; S32, Define the diagram structure :node Includes personnel nodes Equipment Nodes Scene nodes ;side Represents the interaction relationships between nodes; adjacency matrix elements ,in, For nodes and The Euclidean distance; The computer initializes the GNN network with the following number of layers: Define the GNN propagation rules; and use the completed temporal features. Input a dynamic GNN model and execute... Layer propagation computation outputs multi-target interaction features. . S33, Set preset threshold ,like If the relationship is positive, it is determined to be a causal relationship; otherwise, it is determined to be a false relationship, and the process returns to step S32 to re-extract the interaction features.

5. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 1, characterized in that, The specific steps of step S4 are as follows: S41. Real-time positioning data of the temporary high-altitude platform is collected using UWB positioning sensors. The positioning data includes three-dimensional coordinates. and the boundary coordinates of the danger zone , will location data every Update the BIM model once per second; Extract scene parameters from BIM model Build a dynamic scene parameter library And set the parameter to update the timestamp; S42. Based on multi-target interaction characteristics Extracting personnel action types and 3D positions ; Set height difference threshold Query the dynamic scene parameter library Get the platform height of the current scene If the action type "Working at height" and If the action is determined to be a mismatch between the action and the scene, scene parameter anomaly verification is triggered; otherwise, it is determined to be a match, and a preliminary action recognition result is obtained. ; Set error threshold When it is determined that the action does not match the scene, triggering scene parameter anomaly verification, the laser rangefinder is activated to remeasure the height of the temporary high-altitude platform. ,like Then update The platform height in the middle is Return to step S42 for re-comparison.

6. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 1, characterized in that, The specific steps of step S5 are as follows: S51. Call the ASE codec to process the preliminary action recognition results. Intra-frame prediction and entropy coding are performed on the feature map blocks to obtain the compressed result. ; Compression results Add an FEC checksum and set the checksum length to [value]. Data length Proportion, and ,use Linear block codes, where , for Data length; S52, Set the signal-to-noise ratio threshold ,like Then the computer reduces the quantization step size to Return to step S51 to re-encode; until The final compression result is obtained. ; S53, Compress the final result Output to the application: If the final compression result If the data contains a "violation" tag, the computer sends a control command to the audible and visual alarm device, triggering an alarm; simultaneously, the final compressed result... Store in the local database.

7. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 3, characterized in that, In step S23, an action state transition matrix is ​​established based on the timing logic of the violation action. Specifically, it includes: S231. Computer loading of a time-series sample library of typical construction site violations: samples of illegal climbing. Group, samples of people not wearing safety helmets Group, among which For each set of samples, OpenCV is used to extract time-series frames, and ResNet-50 is used to extract frame features. These features are then input into an LSTM model to train action state classification, and the output is Q=5 consecutive action states, which are the initial state. Ready state Execution status Transitional state Completion status ; S233, For any two states and ,in, Calculate the transition probability This yields a 5×5 action-state transition matrix. ; S234. Action state transition matrix Perform normalization to obtain the normalized action-state transition matrix. ; When lightweight features are detected There are characteristic fractures in the process, based on the normalized action-state transition matrix. Predicting transition frame features: ,in, To predict transition frame features, the temporal features are obtained after completion. .

8. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 4, characterized in that, Adjacency matrix in step S32 elements It also meets the following filtering criteria: Preset node interaction distance threshold: Threshold for personnel-device interaction Interpersonal interaction threshold Personnel and scene interaction threshold ,in, ; S321, When node For personnel nodes ,node For device nodes Calculate the Euclidean distance between the two. ,like Then set It was determined that there was no interaction between personnel and equipment; S322, When node and nodes All are personnel nodes Calculate the Euclidean distance between the two. ,like Then set It was determined that there was no coordinated action between the personnel; S323, When node For personnel nodes ,node scene nodes Calculate the Euclidean distance between the two. ,like Then set It was confirmed that no personnel had entered the danger zone; S324. Update the adjacency matrix based on the above filtering results. The filtered adjacency matrix is ​​obtained. ,Will Return to step S32 for GNN propagation computation.

9. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 6, characterized in that, The specific steps of ASE encoding and decoding in step S51 include: S511, Set the image block size to The initial action recognition results The feature map is divided into multiple image blocks; for each target block, its upper and left adjacent blocks are selected as reference blocks, the average pixel value of the reference blocks is calculated, and the predicted pixel value of the target block is predicted based on the average pixel value of the reference blocks. ; Calculate the actual pixel value of the target block and the predicted pixel value of the target block. The difference is used to obtain the prediction residual; S512. Employ context-based adaptive arithmetic coding, inputting the prediction residuals and quantization parameters into the coding model, and setting the coding window size to [value missing]. Bytes are dynamically adjusted in terms of encoding probability distribution through a context model to output an encoded binary stream. S513. Calculate the mean square error (MSE) and peak signal-to-noise ratio (PSNR) of the data before and after compression. S514, if Then update the quantization step size to ,and Return to step S51 to re-encode; until The final compression result is obtained. .

10. The method for continuous action recognition of construction site personnel applied to smart construction sites according to claim 2, characterized in that, Scene complexity value in step S12 A time decay factor was also introduced. Specifically: S121. The computer reads the timestamps of the current frame and the previous frame and calculates the time interval. : ; in, This represents the time interval between the current frame and the previous frame, obtained by the difference in timestamps from the camera. Indicates the timestamp of the current frame; Indicates the timestamp of the previous frame; S122, Calculate the time decay factor : ;in, Indicates the time decay factor; Indicates the attenuation coefficient. ; S123, Update the scene complexity value :; ;in, This represents the updated scene complexity value; , These are the weighting coefficients for occlusion rate and motion density, respectively. S124. Computer execution computing power allocation: Set frame interval threshold. In step S13, based on the updated scene complexity value Allocate edge computing power: when hour, Follow Increase and decrease, updated scene complexity value Synchronization is reduced, and the computing power allocated to this video stream by the computer is decreased. Proportion, and ;when At that time, the updated scene complexity value Maintain the original level and keep the computing power allocation unchanged.

Images