Pose temporal evolution and anomaly detection method considering spatial semantic prior

Abstract

The application discloses a pose time sequence evolution and abnormality detection method considering spatial semantic prior, and relates to the technical field of computer vision and behavior recognition. The method comprises the following steps: performing matrix multiplication operation on a human body skeleton key point coordinate tensor and a preset polyhedral boundary parameter matrix, and subtracting a boundary bias vector to obtain an algebraic residual tensor; performing linear mapping on the algebraic residual tensor to obtain a spatial prior feature tensor, and keeping the key point dimension unchanged; taking the spatial prior feature tensor as a query tensor, and performing cross attention fusion on the key point time sequence feature to output a fusion feature, taking the key point dimension as an independent operation unit; outputting a behavior category confidence according to the fusion feature, and performing logical judgment on a fall arrest cable hanging detection state, a visual confidence of the human body skeleton key point coordinate tensor and a preset safety rule to output an abnormality detection decision result. The application is used to solve the problems of grid topology search leading to calculation graph blocking, global pooling leading to local feature loss and pose semantic confusion.

Description

Technical Field

[0001] This invention relates to the field of computer vision and behavior recognition technology, and more specifically, to a method for pose temporal evolution and anomaly detection that takes into account spatial semantic priors. Background Technology

[0002] Hazardous area boundary crossing detection and abnormal behavior recognition in unstructured industrial work sites are core technologies for safety production monitoring. Typically, edge-based visual computing devices are used to analyze monitoring video streams in real time, extract the coordinate sequences of key points on the human skeleton, and combine this with the spatial geometric boundaries of the work site to complete the temporal evolution analysis of human pose and provide early warnings of high-risk conditions.

[0003] Existing technologies typically combine on-site 3D data for spatial topology inclusion determination and behavioral sequence prediction. For example, Chinese patent CN120542438B discloses an intelligent geometric reasoning and semantic understanding method based on a 3D large language model, which aggregates point cloud and text features using a cross-modal cross-attention module and configures low-rank adaptive updates. However, it mainly focuses on attribute completion of static building components and cannot handle the temporal evolution of dynamic human poses. Chinese patent CN121938052B discloses a dynamic human fall detection method, which constructs a temporal motion representation of the human body by performing 3D pose estimation on image sequences and combines calibrated ground parameters for risk judgment. However, its temporal encoder uses a global average pooling layer to aggregate spatiotemporal features, and its graph structure and boundary determination mechanism are too simplistic, making it unable to handle complex cross-boundary topology changes.

[0004] However, the aforementioned existing technologies have the following limitations in application: First, traditional polyhedral projection or planar calibration algorithms use hard discrete boundary division, cutting off the spatial distance gradient, which leads to the need to perform high-frequency grid topology search in the forward computation, causing topology search blockage; Second, the global average pooling operation in the multimodal feature fusion stage flattens and erases the discrete key point channels, resulting in the loss of limb local boundary crossing action features, leading to high-risk boundary crossing false negatives; Third, the model lacks effective constraints of spatial scene context, which leads to semantic confusion of behavioral poses with similar spatial geometry but different safety semantics, reducing the certainty of anomaly detection decisions. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide a pose temporal evolution and anomaly detection method that takes into account spatial semantic priors. The method obtains algebraic residuals through multiplying and adding multicellular boundary matrices, performs cross-attention fusion while maintaining the dimension of key points, and makes logical judgments based on preset safety rules. This solves the technical problems of computation graph blocking caused by topology search, loss of local features caused by global pooling, and pose semantic confusion.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for pose temporal evolution and anomaly detection that considers spatial semantic priors includes the following steps: acquiring the coordinate tensor of key points in the human skeleton, the temporal features of key points, the detection status of fall arrestor slings, and the visual confidence of the coordinate tensor of key points in the human skeleton; performing matrix multiplication on the coordinate tensor of key points in the human skeleton and a preset multi-cell boundary parameter matrix, and subtracting the corresponding boundary bias vector to obtain the algebraic residual tensor of each key point relative to each boundary plane; performing linear mapping on the algebraic residual tensor to obtain the spatial prior feature tensor, while keeping the key point dimension unchanged; using the spatial prior feature tensor as the query tensor, performing cross-attention fusion with the temporal features of key points through a cross-attention fusion layer, with the key point dimension as the independent operation unit, and outputting the fused features; outputting the behavior category confidence based on the fused features, and logically judging the behavior category confidence, the fall arrestor sling detection status, and the visual confidence of the coordinate tensor of key points in the human skeleton with preset safety rules, and outputting the anomaly detection decision result.

[0007] Furthermore, this invention also provides a pose temporal evolution and anomaly detection system that considers spatial semantic priors, comprising: a data acquisition module for acquiring the coordinate tensor of key points of the human skeleton, the temporal features of the key points, the detection status of the fall arrestor rope attachment, and the visual confidence of the coordinate tensor of the key points of the human skeleton; an algebraic residual module for performing matrix multiplication operations between the coordinate tensor of the key points of the human skeleton and a preset polytopic boundary parameter matrix, and subtracting the corresponding boundary bias vector to obtain the algebraic residual tensor of each key point relative to each boundary plane; and a spatial encoding module for using linear... The mapping layer maps the algebraic residual tensor to obtain the spatial prior feature tensor while keeping the keypoint dimension unchanged. The feature fusion module uses the spatial prior feature tensor as the query tensor and performs cross-attention fusion with the keypoint temporal features through the cross-attention fusion layer, with the keypoint dimension as the independent operation unit, to output the fused features. The decision reasoning module outputs the behavior category confidence based on the fused features and performs logical judgment on the behavior category confidence, the fall arrestor rope attachment detection status, and the visual confidence together with the preset safety rules to output the anomaly detection decision result.

[0008] The present invention provides a computer device, including a processor and a memory: the memory is used to store a computer program and send the instructions of the computer program to the processor; the processor executes the pose temporal evolution and anomaly detection method taking into account spatial semantic priors according to the instructions of the computer program.

[0009] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the pose temporal evolution and anomaly detection method taking into account spatial semantic priors.

[0010] In summary, the pose temporal evolution and anomaly detection method of the present invention, which takes into account spatial semantic priors, has the following technical effects and advantages: This invention obtains an algebraic residual tensor by performing matrix multiplication on the human skeleton keypoint coordinate tensor and a preset polytopic boundary parameter matrix, and then subtracting the boundary bias vector. A spatial prior feature tensor is obtained through a linear mapping that preserves the keypoint dimensions. This spatial prior feature tensor is then used as a query tensor and cross-attention fusion with the keypoint temporal features, where the keypoint dimensions are independent computational units, to output a fused feature. Finally, the behavior category confidence, fall arrestor rope attachment detection status, and visual confidence of the human skeleton keypoint coordinate tensor, based on the fused feature output, are logically judged and output according to preset safety rules. The anomaly detection decision-making results address the technical problems of traditional geometrically inclusive boundary crossing detection, such as computational graph topology search blocking due to reliance on complex 3D mesh topology search, loss of gradients at discrete joint local geometric edges due to global average pooling aggregation of multimodal features, and semantic confusion of behavioral poses caused by lack of spatial scene context constraints. It achieves branchless solution of spatial directed residuals through matrix multiplication and addition operations, and cross-attention fusion with keypoint dimensions as independent computational units, eliminating latency in network forward propagation and improving the accuracy of discrete joint local boundary crossing action recognition and the determinism of anomaly detection decisions. Attached Figure Description

[0011] Figure 1 A schematic diagram of the pose temporal evolution and anomaly detection method considering spatial semantic priors provided in an embodiment of the present invention; Figure 2 This is a graph showing the spatial location trajectory and the corresponding algebraic residual tensor variation trend in an embodiment of the present invention; Figure 3 This is a two-dimensional heatmap of the discrete temporal attention weight matrix in an embodiment of the present invention; Figure 4 This is a graph showing the convergence of the loss function and the proportion of updatable parameters in an embodiment of the present invention. Figure 5 A block diagram of a pose temporal evolution and anomaly detection system that takes into account spatial semantic priors, provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0013] Figure 1 This invention presents a method for pose temporal evolution and anomaly detection that takes into account spatial semantic priors, comprising the following steps: S1, obtain the coordinate tensor of key points of human skeleton, the temporal features of key points, the detection status of fall arrestor rope attachment, and the visual confidence of the coordinate tensor of key points of human skeleton.

[0014] In this embodiment, step S1 specifically includes the following steps: S101, obtain the coordinate tensor of key points of human skeleton and the temporal features of key points; Specifically, a continuous sequence of video frames is input to the forward input, and a keypoint detection network is used to extract the two-dimensional pixel coordinates of each skeletal keypoint on the pixel plane. Simultaneously, the image frame is input into a preset depth estimation network for forward feature extraction and depth regression, outputting physical depth values ​​that correspond one-to-one with the two-dimensional pixel coordinates. Subsequently, inverse perspective projection transformation is performed on the two-dimensional pixel coordinates and the absolute physical depth values ​​using the pre-calibrated camera intrinsic parameter matrix. This process calculates the three-dimensional absolute physical coordinate components of each skeletal keypoint in a unified physical space coordinate system (i.e., the world coordinate system), and reconstructs the coordinate tensor of the human skeletal keypoints. .in, For batch size, This represents the total number of time frames within the behavior time sliding window. Given the preset total number of human skeletal joints, the number of channels in the feature channel dimension (3) corresponds to the three absolute coordinate axis components in a unified physical space coordinate system. The transformation equation for the inverse perspective projection transformation is expressed as follows: (1) In the formula: This is the calculated three-dimensional absolute physical coordinate vector of the corresponding skeletal keypoints in a unified physical space coordinate system; The physical depth values ​​output by the depth estimation network for the corresponding skeletal keypoints; For the pre-calibrated camera intrinsic parameter matrix, This is the inverse of the camera intrinsic parameter matrix; This refers to the homogeneous pixel coordinate vectors of the corresponding skeletal keypoints in the digital image pixel coordinate system.

[0015] Subsequently, the reconstructed human skeleton key point coordinate tensor is used... The input is fed into a graph-based temporal feature extraction network. In this embodiment, the graph-based temporal feature extraction network specifically adopts a cascaded multi-layer spatiotemporal graph convolutional network (ST-GCN). In the spatial dimension, the spatial graph convolution is initialized using a learnable edge importance allocation parameter matrix, and based on a spatial morphology partitioning strategy, a graph topology structure containing centripetal sub-adjacency matrices, centrifugal sub-adjacency matrices, and root node adjacency matrices is constructed to achieve the aggregation of local spatial features of skeletal nodes. In the temporal dimension, temporal features are captured through cascaded one-dimensional dilated temporal convolutional layers, wherein the temporal convolutional kernel size is preset. expansion coefficient This expands the temporal receptive field. Simultaneously, a residual connection mechanism is introduced between each convolutional layer, adding the input features to the convolutional output features element-wise to ensure stable gradient propagation. The final output is a high-dimensional representation of limb motion evolution features, i.e., keypoint temporal features. ,in This represents the number of feature channels.

[0016] As an equivalent implementation, the graph-based temporal feature extraction network can also be replaced by a cascaded temporal convolutional network (TCN) or a Transformer temporal network based on a self-attention mechanism to complete the mapping of temporal behavioral features.

[0017] S102, further, obtain the visual confidence of the human skeleton key point coordinate tensor, including the following steps: S1021, Input the image frame into the key point detection network to obtain the predicted probability distribution of each skeletal key point on the pixel plane; Specifically, a single frame image is input into a preset keypoint detection network. Through forward propagation via a feature extraction network and a regression detection head, a two-dimensional predicted probability map (Heatmap) is output for each skeletal keypoint. The spatial resolution of the two-dimensional predicted probability map maintains a fixed ratio between the downsampling step size of the keypoint detection network and the input image frame. The response values ​​of each pixel position in the two-dimensional predicted probability map are normalized and mapped to the interval [0, 1] using a Sigmoid non-linear activation function, generating a discrete probability distribution tensor reflecting the probability of the existence of each skeletal keypoint.

[0018] S1022, extract the highest probability response value corresponding to each skeletal key point as the single-node location confidence score; Specifically, a maximum value extraction function is used to perform a maximum value search on the spatial resolution dimension of the discrete probability distribution tensor to extract the pixel position with the highest response intensity. The highest probability value corresponding to this position is then used as the single-node localization confidence score of the corresponding skeletal keypoint under the current action time sliding window frame. Simultaneously, the pixel coordinate index of the highest probability response value is extracted as the original two-dimensional coordinate of the skeletal key point in the digital image pixel coordinate system, which is used to map the three-dimensional absolute physical coordinate components of the corresponding key point in the inverse perspective projection transformation described in step S101 in the forward data stream.

[0019] S1023, calculate the average value of the location confidence scores of each single node within the behavior time sliding window to obtain the global visual confidence score.

[0020] Specifically, the arithmetic mean method is used to calculate the double-cumulative mean of the confidence scores of all single nodes within the current entire behavior time sliding window. The calculation formula is as follows: (2) In the formula, Indicates global visual confidence; Indicates the total number of time frames within the behavior time sliding window; This indicates the total number of preset key points in the human skeleton; Indicates the first Frame 1 The highest probability response value extracted from the predicted probability distribution map for each skeletal keypoint. As a preferred implementation, the global visual confidence score can also be equivalently replaced by the classification confidence score of the target detection box, or by the inverse of the posterior covariance matrix output by a state estimator such as a Kalman filter.

[0021] S103, further, obtaining the fall arrestor cable attachment detection status includes the following steps: S1031, using an object detection network to identify the location information of safety hooks, fall arrestors and steel beam anchor points in the image; Specifically, the image frame is input into the object detection network for forward feature extraction and bounding box regression, and the forward output includes a two-dimensional pixel bounding box region containing the safety hook. A two-dimensional pixel bounding box region containing the steel beam anchor points. and the two-dimensional pixel bounding box region containing the fall arrestor cable. The two-dimensional pixel bounding box regions are all represented by a four-dimensional vector containing the pixel coordinates of the upper left and lower right corners.

[0022] S1032, Locate the joint key point coordinates of the target worker based on the human skeleton key point coordinate tensor, and determine the local region of interest containing the joint key point coordinates. Specifically, the human skeletal keypoint coordinate tensor obtained from the aforementioned regression is read, and the pixel coordinates of the joint keypoints corresponding to the target worker's left wrist keypoint, right wrist keypoint, or back center point are retrieved and extracted. Using the pixel coordinates of the joint key points and the two-dimensional pixel bounding box region of the safety hook detected in step S1031. Using the geometric envelope reference, determine the pixel coordinates of the joint key points. With the two-dimensional pixel bounding box region The minimum bounding rectangle is determined. Using this minimum bounding rectangle as the spatial center, the region is expanded outwards according to preset pixel width and height to determine the four corner coordinates of the cropping boundary matrix. Using the global maximum pixel width and height of the image frame, the four corner coordinates of the cropping boundary matrix are truncated to restrict them from exceeding the physically valid index range of the image matrix. Based on the truncated boundary coordinates, matrix slicing is performed on the image frame matrix to extract and output a local region of interest (ROI) image patch containing the coordinates of the joint keypoints and the boundary region of the safety hook.

[0023] S1033, Extract the connectivity path information of the fall arrestor cable within the local region of interest; Furthermore, the captured region of interest (ROI) image patch is input into a pre-defined semantic segmentation network, which outputs a binary pixel mask matrix for the fall arrestor category. Pixels belonging to the fall arrestor category are assigned a value of 1, while background pixels are assigned a value of 0. The binary pixel mask matrix is ​​clustered using a two-pass connected component labeling algorithm (Two-Pass CCL) to filter out the connected component pixels belonging to the target fall arrestor. A pre-defined Zhang-Suen thinning algorithm is then used to perform single-pixel thinning of the connected component pixels, extracting the skeleton connected path information representing the rope's centerline.

[0024] S1034. Based on the spatial proximity between the safety hook and the steel beam anchorage point, and the topological connectivity between the key joint point and the safety hook, the detection status of the fall arrestor cable is determined.

[0025] Specifically, the safety hook bounding box is calculated using the intersection-union ratio operator. Boundary frame of steel beam anchorage point The spatial overlap and intersection-union ratio are calculated using the following formulas: (3) In the formula, This represents the intersection-union ratio (IoU) of the two-dimensional pixel bounding box of the safety hook and the two-dimensional pixel bounding box of the steel beam anchor point. This represents the pixel area of ​​the intersection region of the two bounding boxes. This represents the pixel area of ​​the union region of the two bounding boxes.

[0026] Based on this, a decision branch is set: when the intersection-union ratio... When the distance exceeds a preset proximity threshold, and the skeleton connection path of the fall arrestor remains continuous and uninterrupted between the target worker's joint key points and the safety hook, the high-altitude operation safety protection status is determined to be established, and the fall arrestor connection detection status is set to [value missing]. The value is assigned to 1; if any of the above conditions are not met, it is determined that no effective protection has been implemented, and the status bit is changed. The value is assigned to 0. As an equivalent method, the detection status of the fall arrestor cable can also be equivalently replaced by the wireless near-field communication induction signal deployed at the work site or the physical contact sensor electrical signal of the safety belt buckle.

[0027] This step solves the technical problem of feature loss caused by physical occlusion, asynchronous multi-source data, and degradation of single visual modality environment in traditional behavior recognition by extracting and mapping multimodal perception information. It achieves the technical effect of providing spatiotemporally consistent multimodal data input for forward propagation of network layer.

[0028] In this embodiment, the key point detection network, the graph-based temporal feature extraction network, and the lightweight semantic segmentation network used in the aforementioned steps are all in a pre-trained and converged ready state.

[0029] Specifically, the key point detection network and the graph-based temporal feature extraction network are supervised learning through a pre-collected temporal dataset of high-altitude workers' postures. The mean squared error loss function and the classification cross-entropy loss function are used as objective functions, respectively, and the backpropagation algorithm is used to iteratively update the network weight matrix until the loss converges. The lightweight semantic segmentation network is trained in a supervised manner on a difficult negative sample dataset of fall arrest cables and various types of cables intertwined in a complex high-altitude background. A weighted combination of cross-entropy loss and Dice loss is used as the objective function for gradient backpropagation to guide the network to perform directional convergence optimization of the edge segmentation features of slender topological ropes, ensuring the accuracy of the output of fall arrest cable connectivity paths during the forward inference stage.

[0030] S2 performs matrix multiplication on the coordinate tensor of key points of the human skeleton with the preset multi-cell boundary parameter matrix, and subtracts the corresponding boundary bias vector to obtain the algebraic residual tensor of each key point relative to each boundary plane.

[0031] In this embodiment, step S2 is specifically divided into the following steps: S201, the boundary parameter matrix and boundary offset vector of the multi-cell structure are determined through the following steps: S2011, analyzes the geometric topology of the warning area in the building information model, and decomposes the warning area into a finite set of convex multicells jointly defined by linear half-space inequalities; Specifically, the topological coordinate matrix of the 3D mesh vertex of a predefined dynamic high-risk work area or warning zone in the Building Information Model (BIM) is read through a standard data interface for industrial basic types. When the warning zone exhibits a non-convex spatial topology, a 3D convex decomposition algorithm is invoked at the analytical geometry level to automatically divide the non-convex spatial topology into a finite number of discrete convex multicellular regions, which are then combined to form the convex multicellular set. The physical spatial extent of the warning zone is defined by the geometric union of all convex multicellular regions in the convex multicellular set. Thus, each decomposed convex multicellular sub-region is geometrically bounded by the intersection of a set of linear half-space inequalities.

[0032] As a specific application example, if the warning area is an L-shaped suspended platform, a three-dimensional convex decomposition algorithm is used to divide the L-shaped suspended platform into a first convex polytopic sub-region and a second convex polytopic sub-region, which are then included in the convex polytopic set. Taking the first convex polytopic sub-region as an example, its geometric boundary is constrained by the intersection of the linear half-space inequalities corresponding to the six enclosing surfaces. For any nth convex polytopic sub-region... The bounding surface is mathematically represented by the linear half-space inequality: In the formula, For the first convex multicellular sub-region Normal vectors of the enclosing surface; To unify the three-dimensional coordinate vectors in the physical space coordinate system; For the first convex multicellular sub-region The boundary offset values ​​corresponding to each bounding surface.

[0033] The physical spatial boundary of the first convex polytopic sub-region is determined by the intersection of the linear half-space inequalities corresponding to the six enclosing surfaces, and the physical spatial range of the warning area is determined by combining the physical spatial boundaries of each sub-region in the convex polytopic set.

[0034] S2012, extract the normal vector and boundary offset value of the bounding surface of each convex polycell, and construct the polycell boundary parameter matrix and boundary offset vector respectively.

[0035] Specifically, the standard linear equation coefficients of the bounding surface of each convex polytope are extracted at the analytical geometry level. The normal vectors of the bounding surfaces of each convex polytope sub-region obtained from step S2011 are then decomposed. The matrix boundary parameter matrix is ​​constructed by stacking and combining matrix dimensions. In this embodiment, it is a three-dimensional matrix within the computational graph. The formal reconstruction representation, where matrix elements Rigid correspondence The first convex multicellular shape The normal vector components of the enclosing surface; simultaneously, the boundary offset values ​​of each convex multicell sub-region. The boundary bias vector is constructed by stacking and combining tensor dimensions; in this embodiment, it is a matrix. The formal reconstruction representation, where matrix elements Corresponding to the The first convex multicellular shape The boundary offset values ​​of the enclosing surfaces.

[0036] in, The total number of convex multicells obtained from the decomposition. This represents the maximum number of enclosing faces contained in a single convex multicell. For cells with fewer than [a certain number of enclosing faces]... For the convex multicell, redundant matrix elements are zero-padded. The boundary parameter matrix of the completed multicell is then constructed. With the boundary bias vector Stored as a matrix constant, it serves as the input for matrix multiplication in the forward computation graph.

[0037] As an equivalent implementation, the polytopic boundary parameter matrix and boundary offset vector can also be equivalently replaced by the directed distance field continuous surface parameter matrix, the hierarchical bounding box analytic tree matrix, or the implicit surface function matrix to characterize the geometric spatial boundary distribution of high-risk areas.

[0038] S202, obtaining the algebraic residual tensor of each key point relative to each boundary plane includes: performing matrix multiplication on the coordinate tensor of the human skeleton key points of each frame within the behavior time sliding window and the multi-cell boundary parameter matrix, and subtracting the corresponding boundary bias vector to obtain the algebraic residual tensor.

[0039] Specifically, the human skeleton key point coordinate tensor obtained in step S1 (Where the number of channels in the feature dimension 3 corresponds to the three absolute coordinate axis components in the unified physical space coordinate system obtained by inverse perspective projection transformation as described in step S101, i.e.) The coordinates are then used to perform affine matrix multiplication with the multi-cell boundary parameter matrix constructed in step S2012. This is done using a large-scale parallel multiply-add operation to calculate the algebraic residual tensor from each discrete skeletal keypoint to all boundary planes of the convex multicell in a single operation. The specific calculation equation for the algebraic residual tensor is as follows: (4) In the formula: The algebraic residual tensor calculated The component elements in; For the batch size of the forward computation, For the corresponding batch size index; This represents the total number of time frames within the behavior time sliding window. Index of the sliding window time frame; This represents the total number of key points in the human skeleton. Index for key point nodes; The total number of convex multicells, Index for convex multicellular regions; The maximum number of enclosing faces contained in a single convex multicell. Indexed by the dimensions of the bounding surface; Let be the components of the coordinate tensor of key points in the human skeleton along the three absolute coordinate axes. When respectively corresponding to the absolute coordinate system Axis physical coordinate values; These are the normal vector components of the bounding surface of the corresponding polytope; This represents the boundary offset value corresponding to the bounding surface of the polytope.

[0040] The algebraic signs of the components of the algebraic residual tensor have a continuous spatial geometric gradient meaning. When a certain component of the algebraic residual tensor... A negative value indicates that the corresponding key point of the human skeleton has been cut into the inner space of the corresponding boundary plane. For a specific convex multicellular structure... The dimensions of all its corresponding boundary planes If all residual components are activated to negative values, it directly indicates that the specific skeletal key point has entered the warning area.

[0041] At the output, the algebraic residual tensor The last two dimensions (i.e., the total number of corresponding convex polycells) With maximum number of bounding faces The two dimensions are flattened and reshaped into a unified boundary feature channel axis, generating a shape of The four-dimensional tensor is used as the input to the downstream feature transformation layer. This calculation method replaces the spatial inclusiveness judgment logic based on complex three-dimensional mesh topology search with matrix multiplication and addition operations.

[0042] In the matrix multiplication and addition calculation in step S202, taking the entry of a single keypoint into a convex multicellular warning region containing six boundary planes as an example, its spatial location trajectory and the corresponding algebraic residual tensor change trend are as follows: Figure 2 As shown. Figure 2The diagram illustrates the trajectory of the keypoint's 3D physical coordinates within a sliding window of the action time, as well as the numerical curves of the algebraic residual tensors corresponding to the six boundary planes as it approaches and crosses the boundary of the convex polytope warning region. When the keypoint is outside the convex polytope warning region, the algebraic residual components corresponding to the six boundary planes are all greater than 0. As the keypoint moves inward and crosses the boundary, the values ​​of the algebraic residual components corresponding to the planes intersecting the trajectory decrease monotonically, changing from positive to negative at the moment of crossing the boundary. The keypoint is determined to have entered the warning region if and only if the values ​​of the algebraic residual components of all six boundary planes corresponding to the convex polytope are less than 0. This calculation method replaces the spatial inclusiveness judgment logic with matrix multiplication and addition operations.

[0043] This step solves the technical problem of high forward propagation delay and computational graph blockage caused by the traditional geometric inclusive boundary detection relying on complex 3D mesh topology search, through the matrix linear mapping mechanism of multi-cell half-space inequalities. It achieves the technical effect of introducing discrete joint continuous geometric edge gradients while maintaining low computational complexity in parallel matrix operations, and significantly improving the spatial perception accuracy of behavioral features.

[0044] S3 performs a linear mapping on the algebraic residual tensor to obtain the spatial prior feature tensor while keeping the key point dimension unchanged.

[0045] In this embodiment, step S3 is as follows: By using a linear mapping layer containing activation functions to transform the channel dimension of the algebraic residual tensor, a spatial prior feature tensor is obtained. Specifically, the input to this step is the algebraic residual tensor obtained by flattening and reshaping the last two dimensions in step S2. The algebraic residual tensor is applied using the linear mapping layer containing the activation function. Multiply the last feature channel dimension by a matrix to reduce the channel dimension from... Transformed into the preset spatial prior feature channel axis dimension The specific computational equations for the linear mapping layer containing the activation function are as follows: (5) In the formula: The generated spatial prior feature tensor has the following shape: ; The input is the algebraic residual tensor; Let be the trainable weight matrix of the linear mapping layer containing the activation function; This is the corresponding trainable bias vector; This represents a batch normalization layer, used to stabilize the numerical distribution of features; This indicates a rectified activation function with leakage.

[0046] As an equivalent implementation, the linear mapping layer containing the activation function can also be replaced by a multilayer perceptron network layer, a piecewise linear embedding layer, or a depthwise separable convolutional layer; the leaky rectified activation function can also be replaced by a GeLU activation function, a SiLU activation function, or an ELU activation function.

[0047] In addition, the dimensions of key points remain unchanged during the dimensional transformation process.

[0048] Specifically, during the matrix transformation of the channel dimension in the linear mapping layer, the batch processing axis... Time sliding window axis and the key point axis of the human skeleton Each is treated as an independent batch dimension and does not participate in the channel-level fusion calculation or global average pooling compression. The key point dimension is maintained throughout the entire process. The corresponding tensor axes are independent, so that the spatial geometric edge residual components corresponding to each physical joint point, after linear transformation, remain consistent with the spatiotemporal index of the physical joint point itself, thus avoiding confusion between the spatial boundary features of different physical joint points.

[0049] This step solves the technical problem of missing local high-risk out-of-bounds movements caused by the loss of geometric edge gradients of discrete joints due to global average pooling when external spatial constraints are introduced into neural networks, by maintaining the feature dimension transformation mechanism that keeps the key point dimension unchanged. It achieves the technical effect of transforming discrete algebraic constraints into spatial prior feature tensors that can interact with limb movement features without destroying the graph network structure.

[0050] S4 uses the spatial prior feature tensor as the query tensor, and performs cross-attention fusion with the key point temporal features through a cross-attention fusion layer, with the key point dimension as the independent operation unit, to output the fused features.

[0051] In this embodiment, step S4 is specifically as follows: S401, the spatial prior feature tensor is mapped through the query projection matrix to generate the query tensor, and the key point temporal features are mapped through the key projection matrix and the value projection matrix respectively to generate the corresponding key tensor and value tensor. Specifically, the input to this step is the spatial prior feature tensor output from step S3. and the key point temporal features output in step S101 To achieve spatial alignment of the channel axes of heterogeneous features, the key point temporal features are... Perform a dimensional rearrangement, changing its axis order from Adjusted to Generate keypoint temporal features with adjusted dimensions. .

[0052] Subsequently, three independent learnable weight matrices are applied to the spatial prior feature tensor. Temporal features of key points after dimensional adjustment The last feature channel axis linearly projects the feature channel dimension onto the set hidden layer dimension. (In this embodiment, it is set to 128), which corresponds to the generation of the query tensor. The key tensor and the value tensor The specific formula for calculating projection mapping is as follows: (6) In the formula: The input spatial prior feature tensor has the following shape: ; The keypoint temporal features after dimensional rearrangement have the following shape: ; To query the projection matrix; The key projection matrix; The projection matrix is ​​the value. The defined output channel dimension; the generated query tensor With the key tensor All shapes are The generated value tensor The shape is .

[0053] S402, perform a scaled dot product operation on the query tensor and the key tensor, and obtain the attention weight matrix through a normalized exponential function; Specifically, the query tensor generated in step S401 With the key tensor The structure is reshaped into a tensor structure with the batch axis and keypoint axis as the joint batch dimension, and the time axis and channel axis as the matrix transformation dimensions, denoted as follows: and .

[0054] For any given batch index and key node index Extract its corresponding component matrix and . Component matrix With component matrix Matrix multiplication is performed on the transpose of the matrix to calculate the time-series correlation matrix; subsequently, a scaling factor is used. The matrix elements are scaled by division and then normalized along the time axis using a normalized exponential function to generate discrete temporal attention weight matrices for each physical node. The specific calculation equations are as follows: (7) (8) In the formula: In the first The first batch The scaled dot product temporal correlation matrix for each key point is discrete, and its size is... ; The normalized exponential function represents the effect along the time axis of the time-series correlation matrix; The output is a discrete temporal attention weight matrix for each keypoint, with size [missing information]. This represents the weight distribution of the key point in the time dimension.

[0055] Figure 3 This is a two-dimensional matrix heatmap of the discrete temporal attention weight matrix of a specific key point within a sliding window of the behavior time when the target person makes an abnormal out-of-bounds action. Figure 3 The horizontal and vertical axes in the graph correspond to frame indices within a time-sliding window, and the grayscale intensity within the heatmap represents the correlation weight values ​​between different time frames. In the forward propagation of the cross-attention fusion layer, the scaling dot product operation is applied to the time axis, since the human skeleton keypoint axis... Matrix multiplication that does not participate in the scaling dot product operation, within the time domain interval where an out-of-bounds error occurs, is the query tensor generated from the spatial prior feature tensor. The key tensor generated with the temporal features of the key points By performing a scaled dot product, the corresponding temporal attention weight matrix increases in the matrix element positions corresponding to the out-of-bounds time frames, while the matrix element weight values ​​in other frames approach 0. This feature mapping method avoids feature confusion between different physical nodes and prevents feature loss due to global average pooling of spatial dimensions for local out-of-bounds actions.

[0056] S403 uses an attention weight matrix to perform weighted aggregation of value tensors to obtain fused features; Further, the value tensor generated in step S401 is... Similarly, it is reshaped into a component matrix. The temporal attention weight matrix output in step S402 is used. With the component matrix Perform matrix multiplication to complete the weighted temporal aggregation of the discrete keypoint temporal features, and output the feature component matrix: (9) In the formula, To solve the first The first batch The feature component matrices of each keypoint are then combined, and inverse dimensionality rearrangement is performed to output the fused features of the standard four-dimensional topology. .

[0057] In the scaling dot product operation and weighted aggregation process, the key point dimensions are calculated independently and global average pooling is not performed.

[0058] Specifically, in the matrix multiplication and addition calculations in steps S401 to S403, the key point axes of the human skeleton... It is always treated as an independent batch dimension and does not undergo spatial global average pooling. During the forward propagation of the cross-attention fusion layer, the scaled dot product operation is applied to the time axis. This is because the human skeleton keypoint axis... Matrix multiplication is excluded from the scaling dot product operation, allowing the scaling dot product operation to be performed independently within each keypoint channel. This achieves temporal alignment between the spatial prior feature tensor and the temporal features of the keypoints under the same keypoint index. This avoids feature confusion between different physical nodes and prevents feature loss due to global average pooling of spatial dimensions for local out-of-bounds actions.

[0059] This step solves the technical problem of missed local out-of-bounds actions caused by spatiotemporal misalignment between action features and spatial boundary constraint features due to global average pooling in multimodal fusion by using a cross-attention fusion mechanism with key point dimensions as independent computational units. It achieves the technical effect of dynamically reweighting the temporal features of physical nodes using spatial boundary constraint features and improving the accuracy of local out-of-bounds action recognition.

[0060] S5 outputs the behavior category confidence score based on the fused features, and performs a logical judgment on the behavior category confidence score, the fall arrestor rope attachment detection status, and the visual confidence score of the human skeleton key point coordinate tensor with the preset safety rules, and outputs the anomaly detection decision result.

[0061] In this embodiment, the specific steps of step S5 are as follows: S501, outputs the confidence level of the behavior category based on the fused features; Specifically, the input to this step is the fusion feature output from step S403. Before performing classification mapping, while maintaining the batch axis dimension... Under the premise of not changing, the time axis dimension Key Dimensions With feature channel dimension Flattened into a two-dimensional matrix Subsequently, the two-dimensional matrix... The input is linearly transformed into a fully connected feature layer, and the output is a dense feature vector. .

[0062] The dense feature vector The input is fed into a cascaded classification network for computation. The forward computation equation of the classification network is expressed as follows: (10) (11) In the formula: It is a dense feature vector with the shape of ; This is the first-level classification weight matrix; This is the corresponding first-layer classification bias vector; This represents a one-dimensional batch normalization layer, used to stabilize the numerical distribution of intermediate features; It is a linear rectification activation function; This indicates a regularized dropout layer with a random dropout rate set to 0.2. These are the hidden layer feature vectors; This is the output weight matrix for the second layer; This is the corresponding output layer bias vector; This is an element-level normalized activation function that maps the output value to the interval [0,1]. For the output confidence score of the behavior category, each independent component represents the confidence score value corresponding to the unsafe behavior category. This represents the total number of pre-defined categories of unsafe behaviors.

[0063] As an equivalent implementation, the classification network performing the transformation can also be equivalently replaced by a convolutional classifier head, a tree-based ensemble classifier, or a support vector machine regression head.

[0064] S502 logically determines the behavior category confidence, fall arrestor rope attachment detection status, and visual confidence of human skeleton key point coordinate tensor with preset safety rules, and outputs the anomaly detection decision result. Specifically, the logical decision includes two parallel decision branches, which verify the confidence level of the behavior category by introducing geometric topological constraints. The first decision branch compares the algebraic relationship between the confidence level of the behavior category and a preset classification probability threshold; the second decision branch performs joint Boolean logic operations by combining the spatial state of the algebraic residual tensor, the fall arrestor attachment detection state, and the global visual confidence level. The specific logical decision calculation equation is as follows: (12) In the formula: For the anomaly detection decision results output, the one for the first... The final Boolean decision state bit for unsafe behaviors indicates that the judgment of an exception is true when the value is 1 and an exception activation command is output; when the value is 0, it indicates that the safe state is normal. The confidence score of the output behavior category is the first one. The confidence score of the class; For the preset target number The probability threshold for classifying unsafe behaviors; It is an existence quantifier that acts on the frame dimension of the time sliding window of the behavior, indicating that at least one time frame exists within the time sliding window; The existence quantifier that acts on the dimension of human skeletal keypoints indicates that at least one specific skeletal keypoint exists in the topological structure of human skeletal keypoints. As an existence quantifier acting on the convex polytope dimension, it indicates that at least one specific convex polytope region exists in a finite set of convex polytopes; It is a universal quantifier that acts on the dimensions of the boundary planes of a single convex polycell, indicating all the boundary planes contained in that particular convex polycell; For the component elements in the algebraic residual tensor calculated in step S202, when they are less than zero, the key point nodes are indicated to enter the inner space of the corresponding boundary plane. The value of 0 indicates that the fall arrestor cable is not attached, as determined in step S1034. The global visual confidence score calculated in step S1023; This is the preset low-confidence failure threshold.

[0065] It should be noted that, in the second decision branch, the contemporary residual tensor components satisfy... "At this time" means that at least one time frame within the action time sliding window, if the algebraic residual tensor corresponding to any human skeletal keypoint has negative components in all boundary plane dimensions, it indicates that the at least one human skeletal keypoint is located inside the warning region (i.e., the algebraic residual values ​​relative to all boundary planes of at least one convex multicellular region are less than zero). In terms of spatial geometry and topology, this specific skeletal keypoint has entered the geometric envelope of the warning region. Between different convex multicellular regions, the data flow uses a logical "OR" relationship for parallel branching computation.

[0066] As an equivalent approach, the control logic that executes the decision-making process can also be replaced by a fuzzy logic control rule table, a conditional random field discriminator, or a multi-level expert system decision tree.

[0067] S503 also includes parameter fine-tuning steps: S5031, lock the pre-trained weight matrix of the temporal feature extraction network and the cross-attention fusion layer used to extract the temporal features of key points; Specifically, the weights of the forward multiplication matrix are locked. For the temporal feature extraction network described in step S101, the spatial graph convolution weights and temporal convolution kernel parameters corresponding to the spatiotemporal graph convolution processes of each layer are locked; for the cross-attention fusion layer described in step S4, its query projection matrix is ​​locked. Key projection matrix and value projection matrix The aforementioned locked parameters are uniformly used as pre-trained weight matrices in the topology mapping described later, and do not participate in gradient value updates during backpropagation.

[0068] As an equivalent implementation, the parameter locking can also be replaced by a partial layer parameter masking mechanism or a specific layer static weight locking function to keep the main network parameters unchanged.

[0069] S5032 utilizes a low-rank incremental weight matrix, which is the product of a low-rank reduced-dimensional matrix and a low-rank increased-dimensional matrix, to perform feature forward mapping together with the pre-trained weight matrix. Furthermore, a low-rank adaptive path is introduced in parallel alongside the matrix multiplication path of the pre-trained weight matrix. To provide a standardized mathematical representation of the general feature mapping process, any matrix from the pre-trained weight matrix is ​​used as the target weight matrix. The low-rank incremental weight matrix is ​​constructed in parallel by the product of the low-rank reduced-dimensional matrix and the low-rank increased-dimensional matrix. The low-rank reduced-dimensional matrix is ​​described above. Its parameters are initialized using a Gaussian distribution for random filling; the low-rank up-dimensional matrix The initial state of all matrix elements within it is set to all zeros. The eigenrank is defined as follows: The dimension is much smaller than that of the original pre-trained weight matrix (in this embodiment, the intrinsic rank is set). The transformation equation for the hybrid forward eigenmap is as follows: (13) In the formula: This represents any one of the target weight matrices in the pre-trained weight matrices that were locked in step S5031; The input feature dimension of the target weight matrix is... The output feature dimension of the target weight matrix; The input feature vector; To construct the generated low-rank incremental weight matrix; This is the low-rank reduced-dimensional matrix; Let be the low-rank, high-dimensional matrix; The eigenrank constant of the low-rank matrix is ​​defined. A preset constant scaling factor is used to adjust the numerical magnitude ratio of the feature vector online. This is the output mapped feature vector.

[0070] As an equivalent implementation, the low-rank adaptive mapping layer can also be replaced by the AdaLoRA dynamic rank allocation mapping layer, the LyCORIS matrix block, or an incremental mapping layer based on tensor product.

[0071] S5033 constructs a fine-tuning dataset based on abnormal data samples with erroneous judgments, and uses the backpropagation algorithm to update only the parameters of the low-rank reduced-dimensional matrix and the low-rank increased-dimensional matrix.

[0072] Furthermore, abnormal snapshot data samples identified as erroneous in step S502 during the operation are collected and packaged as sample data to be added to the fine-tuning dataset. When using the backpropagation algorithm to calculate the feature gradient, the gradient propagation range is limited, and only the low-rank dimensionality reduction matrix is ​​processed during the optimization and update of network parameters. and the low-rank up-dimensional matrix The parameters are updated numerically, and the pre-trained weight matrix... The parameters are kept constant, thus limiting the number of weight parameters that need to be updated to less than 1% of the total number of global network parameters, thereby completing closed-loop parameter optimization for a specific construction site.

[0073] The parameter fine-tuning step accounts for a significant proportion of the convergence of the loss function and the number of updatable parameters on the sample dataset. Figure 4 As shown. Figure 4 It contains two sets of corresponding numerical trajectories: one set shows the change in the loss function value as the number of backpropagation iterations progresses, and the other set shows the proportion of updatable parameters during the fine-tuning process. Figure 4 As can be seen from the curve, the intrinsic rank is introduced through parallel connection. Low-rank dimensionality reduction matrix With low-rank increasing-dimensional matrix The loss function value decreases monotonically and tends to stabilize as the number of iterations increases, and the absolute value of its final convergence error is lower than the set convergence threshold; at the same time, due to the pre-trained weight matrix Constraint updates, during the parameter update process, the low-rank reduced-dimensional matrix With the low-rank up-dimensional matrix The number of updatable parameters accounts for less than 0.85% of the total number of network parameters, thus limiting the computational boundary of weight updates.

[0074] As an equivalent implementation, the parameter fine-tuning step can also be replaced by a prefix fine-tuning architecture, a prompt fine-tuning module, or an adapter fine-tuning structure.

[0075] This step solves the technical problems of traditional anomaly detection networks, such as unstable prediction probabilities leading to missed detections when facing sudden environmental changes, and computational power blockage caused by global parameter updates, by using a logical judgment mechanism that jointly verifies the output probability of the neural network with hard space safety rules, combined with an online low-rank incremental fine-tuning method. It achieves the beneficial technical effect of ensuring the determinism of anomaly alarms while realizing low system energy consumption and closed-loop updates.

[0076] Accordingly, embodiments of this application also disclose a pose temporal evolution and anomaly detection system that takes into account spatial semantic priors, such as... Figure 5 As shown, the system includes: The data acquisition module is used to acquire the coordinate tensor of key points of the human skeleton, the temporal features of key points, the detection status of the fall arrestor rope attachment, and the visual confidence of the coordinate tensor of key points of the human skeleton. The algebraic residual module is used to perform matrix multiplication operations on the coordinate tensor of key points of the human skeleton with the preset multi-cell boundary parameter matrix, and subtract the corresponding boundary bias vector to obtain the algebraic residual tensor of each key point relative to each boundary plane. The spatial encoding module is used to map the algebraic residual tensor using a linear mapping layer to obtain a spatial prior feature tensor while keeping the key point dimension unchanged. The feature fusion module is used to perform cross-attention fusion with keypoint temporal features through a cross-attention fusion layer, using the spatial prior feature tensor as the query tensor and the keypoint dimension as the independent operation unit, and outputs the fused features. The decision reasoning module is used to output the confidence level of the behavior category based on the fused features, and to perform logical judgment on the behavior category confidence level, the fall arrestor rope attachment detection status and the visual confidence level together with the preset safety rules, and output the anomaly detection decision result.

[0077] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.

[0078] Embodiments of this application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the pose temporal evolution and anomaly detection method taking into account spatial semantic priors.

[0079] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the pose temporal evolution and anomaly detection method taking into account spatial semantic priors when running.

[0080] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0081] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0082] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0083] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0085] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for pose temporal evolution and anomaly detection considering spatial semantic priors, characterized in that, The method includes: Obtain the coordinate tensor of key points of human skeleton, the temporal features of key points, the detection status of fall arrest rope attachment, and the visual confidence of the coordinate tensor of key points of human skeleton. The coordinate tensors of key points of the human skeleton are multiplied by a matrix multiplication operation with a preset multi-cell boundary parameter matrix, and the corresponding boundary bias vector is subtracted to obtain the algebraic residual tensor of each key point relative to each boundary plane. A linear mapping is performed on the algebraic residual tensor to obtain a spatial prior feature tensor, while keeping the dimension of the key points unchanged; Using the spatial prior feature tensor as the query tensor, cross-attention fusion is performed with the key point temporal features through a cross-attention fusion layer, with the key point dimension as the independent operation unit, and the fused features are output. The behavior category confidence score is output based on the fused features. The behavior category confidence score, the fall arrestor rope attachment detection status, and the visual confidence score of the human skeleton key point coordinate tensor are logically judged with the safety rules, and the detection result is output.

2. The method according to claim 1, characterized in that, The process of performing matrix multiplication between the human skeleton key point coordinate tensor and the preset multi-cell boundary parameter matrix to obtain the algebraic residual tensor is performed on the human skeleton key point coordinate tensor of each frame within the behavior time sliding window.

3. The method according to claim 1, characterized in that, The boundary parameter matrix and boundary bias vector of the multi-cell structure are determined through the following steps: Analyze the geometric topology of the warning area in the building information model and decompose the warning area into a finite set of convex multicells jointly defined by linear half-space inequalities; Extract the normal vector and boundary offset value of the bounding surface of each convex polycell, and construct the polycell boundary parameter matrix and boundary offset vector respectively.

4. The method according to claim 1, characterized in that, The spatial prior feature tensor is obtained by linearly mapping the algebraic residual tensor, including: By using a linear mapping containing an activation function to transform the channel dimension of the algebraic residual tensor, a spatial prior feature tensor is obtained, in which the dimension of key points remains unchanged during the dimension transformation process.

5. The method according to claim 1, characterized in that, The cross-attention fusion with keypoint temporal features, where the keypoint dimension is an independent computational unit, includes: The spatial prior feature tensor is mapped through the query projection matrix to generate the query tensor, and the key point temporal features are mapped through the key projection matrix and the value projection matrix respectively to generate the corresponding key tensor and value tensor. The query tensor and the key tensor are scaled and dot-producted, and the attention weight matrix is ​​obtained by normalizing the exponential function. By treating keypoint dimensions as independent operations, the value tensors are weighted and aggregated using an attention weight matrix to obtain fused features.

6. The method according to claim 1, characterized in that, The logical determination includes: An anomaly is determined when the confidence level of any behavior category is greater than the corresponding preset classification probability threshold; or... An anomaly is determined when at least one key point in the contemporary residual tensor has negative components in all boundary plane dimensions to indicate that the key point is located inside the warning area, the fall arrestor cable is not attached, and the visual confidence level is lower than a preset threshold. When an anomaly is determined, the anomaly detection decision result is set to an anomaly activation state.

7. The method according to claim 1, characterized in that, Obtaining the visual confidence score includes: The image frame is input into the keypoint detection network to obtain the predicted probability distribution of each skeletal keypoint on the pixel plane; Extract the highest probability response value corresponding to each skeletal key point as the single-node location confidence score; The global visual confidence score is obtained by averaging the location confidence scores of each single node within the behavior time sliding window.

8. The method according to claim 1, characterized in that, Obtaining the status of fall arrestor cable attachment detection includes: The object detection network is used to identify the location information of safety hooks, fall arrestors, and steel beam anchorage points in images; The coordinates of the key joints of the target worker are located using the human skeleton key point coordinate tensor, and the local region of interest containing the coordinates of the key joints is determined. Extract the connectivity path information of the fall arrestor cables within the local region of interest; Based on the spatial proximity between the safety hook and the steel beam anchorage point, and the topological connectivity between the key joint points and the safety hook, the detection status of the fall arrestor cable is determined.

9. The method according to claim 1, characterized in that, It also includes parameter fine-tuning steps: Lock the pre-trained weight matrices of the temporal feature extraction network and the cross-attention fusion layer; The low-rank incremental weight matrix, which is the product of the low-rank reduced-dimensional matrix and the low-rank increased-dimensional matrix, is used together with the pre-trained weight matrix to perform feature forward mapping. A fine-tuning dataset is constructed based on outlier data samples with incorrect judgments, and the parameters of the low-rank reduced-dimensional matrix and the low-rank increased-dimensional matrix are updated only using the backpropagation algorithm.

10. The method according to claim 1, characterized in that, The key point temporal features are extracted using a spatiotemporal graph convolutional network.

Images