Cable anti-external damage human behavior recognition method and system based on spatio-temporal graph convolution

Abstract

The present application belongs to the field of cable monitoring and protection, and discloses a cable anti-external damage human behavior recognition method and system based on space-time graph convolution. The method acquires image data and electrical data through a multispectral camera and a sensor respectively; pre-processes the image data to construct a skeletal motion topology graph containing time series dynamic correlation; uses an improved space-time graph convolution network to jointly model multi-scale global space-time features of actions through multi-layer space-time convolution, and outputs a behavior recognition probability; and uses a cross-modal adversarial discriminator to output a fused behavior recognition probability, and judges the risk level and formulates a graded response strategy. The present application realizes joint determination of behavior recognition probability and electrical characteristics, solves the problem of low recognition accuracy of damage behavior in a complex background, and can effectively prevent human-caused cable external damage accidents.

Description

Technical Field

[0001] The present invention belongs to the field of cable monitoring and protection, and specifically relates to a method and system for identifying human behavior for preventing cable damage based on spatiotemporal graph convolution. Background Art

[0002] With the deepening implementation of the new urbanization strategy, underground cable networks have become the core arteries of urban energy supply, and their safe operation is directly related to the stability of urban lifeline systems. However, amidst the rapid development of municipal construction, accidents involving external forces damaging buried cables are becoming more frequent and less visible. This situation places more stringent demands on cable protection technology, necessitating the development of intelligent protection devices and systems with precise identification methods and real-time warning capabilities.

[0003] In recent years, academia and industry have conducted extensive exploratory research in the field of cable protection, fostering a multifaceted approach. Existing approaches integrate computer vision, IoT sensing, and other technologies to build protection systems. However, practical applications have shown that current technologies are still insufficiently adaptable to complex construction scenarios.

[0004] Currently, RGB video-based cable damage monitoring systems generally use two-dimensional convolutional neural networks for behavior recognition. Essentially, these methods model motion features in a two-dimensional pixel space. This approach suffers from three problems: In complex urban scenarios, interference factors such as object occlusion and sudden changes in illumination can easily make human detection difficult, disrupting the spatiotemporal continuity of actions; most existing methods fail to consider the specific spatiotemporal patterns of cable damage behavior, resulting in low accuracy in identifying specific cable damage behaviors and misjudging the risk level of these behaviors; and existing systems often utilize a cloud-based centralized computing architecture. The latency associated with video streaming and processing causes response times to exceed the time threshold for handling high-risk events. Existing systems have a high false alarm rate for typical damage behaviors, such as excavation and metal striking, making it difficult to meet the real-time protection needs of urban power grids. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for cable anti-extrinsic damage human behavior recognition based on spatiotemporal graph convolution to overcome the shortcomings of the existing technology.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The cable anti-extrinsic human behavior recognition method based on spatiotemporal graph convolution has the following steps:

[0008] S1: Acquire image data and electrical data through a multispectral camera and sensor respectively;

[0009] S2: Preprocess the image data and construct a skeletal motion topology map containing dynamic associations of time series;

[0010] S3: Utilizes an improved spatiotemporal graph convolutional network (ST-GCN) to jointly model the multi-scale global spatiotemporal features of actions through multiple layers of spatiotemporal convolutions and output the behavior recognition probability. The improved spatiotemporal graph convolutional network includes an improved spatiotemporal graph convolutional layer and a channel-spatiotemporal attention module. The design of the improved spatiotemporal graph convolutional layer includes a spatial graph partitioning strategy, temporal dimension modeling, and dynamic adjacency matrix fusion.

[0011] S4: Using a cross-modal adversarial discriminator to output the fusion behavior recognition probability, determine the risk level, and formulate a graded response strategy; the cross-modal adversarial discriminator includes:

[0012] Generator G is used to map image features and electrical features into a common embedding space to generate joint features:

[0013] ;

[0014] Where, is the joint feature, is the image feature, which is extracted from the image data; It is the electrical feature, which is extracted from the electrical data;

[0015] Discriminator : To determine whether the feature pair comes from the real data distribution, the loss function is:

[0016] ;

[0017] Where, is the discriminator loss, is the mathematical expectation, A fake electrical signature with added Gaussian noise.

[0018] Further preferably, the preprocessing of the image data in step S2 refers to: fusing the three-channel data into a three-channel fusion feature map :

[0019] ;

[0020] Where, Visible light images , thermal infrared images , polarization degree The weight coefficient is dynamically adjusted according to the light intensity.

[0021] Further preferably, in step S2, a skeletal motion topology graph including time series dynamic association is constructed by a three-dimensional human posture estimation algorithm.

[0022] Further preferably, in step S2, a pre-trained 3D human pose estimation algorithm is used to extract the 3D coordinates of 17 joints from the three-channel fusion feature map generated by preprocessing, and a spatiotemporal topological map is constructed; a loss function is designed as a skeletal trajectory continuity constraint to penalize the variation of joint positions in adjacent frames;

[0023] Calculate the joint Euler angle based on the three-dimensional coordinates of the joint and preset biomechanical limits based on human anatomy and kinematics standards 、 contrast, is the minimum allowable angle of the current degree of freedom, is the maximum allowable angle of the current degree of freedom;

[0024] Define the physical loss function :

[0025] ;

[0026] Where t is the index of the frame and i is the joint number;

[0027] Constrain the joint Euler angles within the biomechanical limits through backpropagation;

[0028] Designing a second-order smoothing loss Implement motion continuity constraints.

[0029] Further preferably, the improved spatiotemporal graph convolutional layer:

[0030] In the spatial dimension, the neighborhood nodes of each joint are divided into three categories through the spatial graph partitioning strategy:

[0031] Root node: the current joint itself;

[0032] Centripetal nodes: adjacent joints pointing to the root node;

[0033] Centrifugal nodes: adjacent joints that are far away from the root node;

[0034] Each partition is assigned an independent learnable weight matrix to capture the local features of different motion patterns and obtain a spatial adjacency matrix;

[0035] Temporal dimension modeling refers to: using a one-dimensional temporal convolution kernel in the temporal dimension to perform temporal modeling on the motion trajectory of the same joint in consecutive frames to capture the temporal changes of the action;

[0036] The spatial adjacency matrix and the temporal adjacency matrix are fused into a dynamic adjacency matrix by setting a learnable weight coefficient; the inverse square root of the degree matrix is ​​used as the normalization factor to perform symmetric normalization on the dynamic adjacency matrix.

[0037] Further preferably, the channel-spatiotemporal attention module calculates the spatial attention weights , temporal attention weight and channel attention weights , feature adaptive enhancement is achieved by multiplying the space-time-channel element by element of the triple attention weights, and then combined with the original features Perform weighted fusion to form multi-scale global spatiotemporal features that are sensitive to destructive behavior;

[0038] ;

[0039] in, It is a multi-scale global spatiotemporal feature.

[0040] Further preferably, the cross-modal adversarial discriminator adopts alternating training, and in each round of iteration, the discriminator is first updated 3 times and then the generator is updated once;

[0041] The first discriminator updates the input real data pair , calculate the discriminator output , calculate the true loss , back propagation updates the discriminator parameters ;

[0042] The second discriminator updates the fake electrical signature generated by the input and forged joint features , For the added Gaussian noise, calculate the forgery discrimination loss , back propagation updates the discriminator parameters ;

[0043] The third discriminator update mixes the real and fake electrical features and calculates the total loss , update the discriminator parameters To maximize the total loss ; Then, freeze the discriminator parameters , generate fake joint features , calculate the generator loss , back propagation updates the generator parameters .

[0044] In step S4, the process of outputting the fusion behavior recognition probability and implementing the graded response of the external failure with the electrical data at the current moment is as follows:

[0045] Define the cable status anomaly detection function and quantify the current mutation amount More than 30% or insulation resistance is lower than the safety threshold Abnormal electrical events:

[0046] ;

[0047] in, is the current, is the rated current; is the insulation resistance;

[0048] When the improved spatiotemporal graph convolutional network outputs the behavior recognition probability When the electrical parameter check is activated, c represents the behavior category; the electrical parameter acquisition unit monitors the cable current mutation in real time. and insulation resistance ; Electrical anomaly confidence level Defined as:

[0049] ;

[0050] Establishing spatiotemporal alignment constraints :

[0051] ;

[0052] in, The improved spatiotemporal graph convolutional network detects the starting time of dangerous behavior. is the electrical anomaly triggering time, s represents seconds;

[0053] Determine the risk level:

[0054] ;

[0055] Trigger responses based on risk level.

[0056] Further optimization is performed to construct an edge-side incremental learning module under the federated learning collaborative framework to update the parameters of the three-dimensional human pose estimation algorithm and the improved spatiotemporal graph convolutional network in a hierarchical manner.

[0057] The present invention also provides a cable anti-extrinsic damage human behavior recognition system based on spatiotemporal graph convolution, comprising a multispectral camera, an electrical parameter acquisition unit, an edge computing terminal, a spatiotemporal graph convolution reasoning module, a fusion analysis module, and a hierarchical response device;

[0058] The multispectral camera is connected to the edge computing terminal, and the multispectral camera is used to collect image data;

[0059] The electrical parameter acquisition unit acquires the cable surface current fluctuation and insulation resistance in real time;

[0060] The spatiotemporal graph convolutional reasoning module uses an improved spatiotemporal graph convolutional network to jointly model the multi-scale global spatiotemporal features of the action through multiple layers of spatiotemporal convolutions and outputs the behavior recognition probability;

[0061] The edge computing terminal integrates an edge-side incremental learning module and a 3D human pose estimation algorithm. The 3D human pose estimation algorithm constructs a skeletal motion topology graph containing dynamic associations of time series. The edge-side incremental learning module optimizes the parameters of the improved spatiotemporal graph convolutional network and the 3D human pose estimation algorithm.

[0062] The fusion analysis module uses a cross-modal adversarial discriminator to output the recognition probability of fusion behavior and determine the risk level;

[0063] The hierarchical response device specifies a hierarchical response strategy according to the risk level.

[0064] The present invention has the following advantages:

[0065] High-Precision Behavior Recognition: Utilizing an improved spatiotemporal graph convolutional network, this approach uses multi-layer spatiotemporal convolutional modeling to comprehensively capture the multi-scale, global spatiotemporal characteristics of actions. By leveraging spatial graph partitioning strategies, temporal dimension modeling, and dynamic adjacency matrix fusion, along with enhanced features from the channel-spatiotemporal attention module, the system significantly improves the accuracy of identifying high-risk behaviors such as digging and tapping, overcoming the low accuracy of traditional two-dimensional convolutional neural networks in complex behavior recognition scenarios.

[0066] Strong anti-interference capability: The multispectral camera uses three-light fusion imaging technology, which combines visible light, thermal infrared, and polarized light to capture images, automatically switching imaging modes based on lighting conditions. It can accurately capture human outlines and movements in various environments, effectively addressing interference factors such as occlusion and sudden changes in lighting in complex urban scenes, and ensuring stable behavior recognition.

[0067] Fast response and low false alarm rate: A visual-electrical bidirectional verification mechanism is built, combining image and electrical features with a cross-modal adversarial discriminator. Through spatiotemporal alignment constraints and risk level determination, the entire chain of computation is completed within 200ms. This allows for rapid power cutoff in high-risk situations and timely warnings in low-risk situations, significantly reducing false alarm rates and meeting the real-time protection needs of urban power grids.

[0068] Good adaptability and scalability: This system builds an edge-side incremental learning module within a federated learning collaborative framework, hierarchically updating the parameters of the 3D human pose estimation algorithm and the improved spatiotemporal graph convolutional network. This system can adapt to behavioral changes brought about by new construction equipment, continuously optimize recognition capabilities, and enhance the system's practicality and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flow chart of the cable anti-extrinsic human behavior recognition method based on spatiotemporal graph convolution;

[0070] Figure 2 This is the structure diagram of the improved spatiotemporal graph convolutional network;

[0071] Figure 3 It is a flowchart of the visual-electrical bidirectional verification mechanism and graded alarm response. DETAILED DESCRIPTION

[0072] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Based on these detailed descriptions, those skilled in the art will be able to clearly understand the present application. Without violating the principles of the present application, the features of the various embodiments may be combined to obtain new implementations, or certain features of certain embodiments may be substituted to obtain other preferred implementations.

[0073] like Figure 1 As shown in the figure, the cable anti-extrinsic human behavior recognition method based on spatiotemporal graph convolution has the following steps:

[0074] S1: Acquire image data and electrical data through a multispectral camera and sensor respectively;

[0075] S2: Preprocess the image data and construct a skeletal motion topology map containing dynamic correlation of time series;

[0076] S3: Utilize the improved spatiotemporal graph convolutional network (ST-GCN) to jointly model the multi-scale global spatiotemporal features of actions through multiple layers of spatiotemporal convolutions and output the behavior recognition probability;

[0077] S4: Use the cross-modal adversarial discriminator to output the fusion behavior recognition probability, determine the risk level and formulate a graded response strategy.

[0078] In step S1, image data is acquired through a multispectral camera. To improve the robustness of video behavior feature detection, the multispectral camera collects video data using visible light and thermal infrared dual-mode imaging technology, with an image sampling frequency of 30fps. The video data is transmitted to the edge computing terminal via the PoE power supply interface. The multispectral camera includes:

[0079] The polarized light channel uses a polarization sensor to capture the metal reflective characteristics through polarizers in four directions: 0°, 45°, 90°, and 135°, and calculate the degree of polarization. :

[0080] ;

[0081] In the formula Represents the light intensity values ​​captured by the polarizers in directions of 0°, 45°, 90°, and 135° respectively.

[0082] Visible light channel and thermal infrared channel: Synchronously collect visible light images and thermal infrared images to resolve interference such as backlight and haze.

[0083] The pre-processing of the image data in step S2 is to fuse the three-channel data into a three-channel fusion feature map :

[0084] ;

[0085] Where, is a visible light image, For thermal infrared images, Visible light images , thermal infrared images , polarization degree The weight coefficient is dynamically adjusted according to the light intensity (Lux value) to improve the detection rate of video behavior features in strong light scenes.

[0086] In step S2, a skeletal motion topology graph containing dynamic associations of time series is constructed using a three-dimensional human posture estimation algorithm, such as Figure 2 The specific steps are as follows:

[0087] S2-1: Extract 3D bone data and construct spatiotemporal topology map:

[0088] Using the pre-trained 3D human pose estimation algorithm, the 3D coordinates of 17 joints are extracted from the three-channel fusion feature map generated by pre-processing to construct a spatiotemporal topological map. ; where the vertex set , Indicates the Frame No. The three-dimensional coordinates of the joints, T is the duration; the edge set , including spatial edges , connecting adjacent joints based on the human anatomy, Indicates the Frame No. The three-dimensional coordinates of the joint; and the timing edge , associate the same joint in three consecutive frames The motion trajectory in Indicates the Frame No. The three-dimensional coordinates of the joints, is the frame interval; dynamic adjacency matrix Contains joint space connections and temporal motion associations, where It is a spatial adjacency matrix that encodes the natural connection relationship of human joints; is the temporal adjacency matrix, encoding the temporal continuity of joint motion; is the learnable weight coefficient;

[0089] S2-2: Design a loss function as a continuity constraint for the skeleton trajectory, penalizing the magnitude of changes in joint positions between frames: Traditional methods only smooth joint coordinates, which may produce postures that violate human kinematics, such as reverse bending of the elbow joint. To suppress sudden changes in the skeleton trajectory caused by occlusion or blind spots, physical constraints are added through the inverse kinematic model:

[0090] First, calculate the joint Euler angle according to the three-dimensional coordinates of the joint and preset biomechanical limits based on human anatomy and kinematics standards 、 contrast, is the minimum allowable angle of the current degree of freedom, The maximum allowed angle of the current degree of freedom.

[0091] Define the physical loss function :

[0092] ;

[0093] Where t is the index of the frame and i is the joint number;

[0094] The joint Euler angles are constrained within the biomechanical limits by backpropagation.

[0095] To solve the problem of bone jitter caused by video jitter, a second-order smoothing loss is designed Implementing motion continuity constraints:

[0096] ;

[0097] Where, Indicates the Frame No. The three-dimensional coordinates of the joints, Indicates the Frame No. The three-dimensional coordinates of each joint and the second-order smoothing loss constraint acceleration are minimized. In the scenario of handheld vibrating tools, the acceleration minimization processing can greatly reduce the jitter of the bone trajectory, effectively suppress the volatility of the bone motion trajectory, and improve the stability of key action recognition.

[0098] Using the improved spatiotemporal graph convolutional network (ST-GCN) to jointly model the spatiotemporal features of the action through multiple layers of spatiotemporal convolution, refer to Figure 2 The specific implementation steps are as follows:

[0099] S3-1: The design of the improved spatiotemporal graph convolutional layer includes spatial graph partitioning strategy, temporal dimension modeling, and dynamic adjacency matrix fusion;

[0100] In the spatial dimension, the neighborhood nodes of each joint are divided into three categories through the spatial graph partitioning strategy:

[0101] Root node: the current joint itself;

[0102] Centripetal nodes: adjacent joints pointing to the root node;

[0103] Centrifugal nodes: adjacent joints that are far away from the root node;

[0104] Each partition is assigned an independent learnable weight matrix to capture the local characteristics of different motion patterns.

[0105] Time dimension modeling means: using a one-dimensional time convolution kernel in the time dimension, and the convolution kernel size is , perform temporal modeling on the same joint motion trajectory of consecutive frames to capture the temporal changes of the action:

[0106] ;

[0107] in, is the weight of the k-th time convolution kernel, For the Layer features, For the The feature input of the layer at time step t+k, For the The offset of the frame.

[0108] Set the learnable weight coefficients to fuse the spatial adjacency matrix with the temporal adjacency matrix:

[0109] ;

[0110] in is the spatial adjacency matrix, is the temporal adjacency matrix, The weight coefficients can be learned for the spatial adjacency matrix and the temporal adjacency matrix, respectively, to control the strength of spatial and temporal associations.

[0111] The inverse square root of the degree matrix is ​​used as the normalization factor By dynamic adjacency matrix Symmetric normalization is performed to ensure that the features remain numerically stable during graph convolution and to avoid gradient explosion or disappearance.

[0112] S3-2: Feature enhancement using channel-spatiotemporal attention module:

[0113] In the spatial dimension, we first construct dynamic attention weights based on the spatiotemporal correlation of joint motion trajectories:

[0114] Spatial attention uses two learnable parameter matrices to concatenate and map the current joint feature and the maximum response feature of its neighborhood, realizing feature extraction and nonlinear transformation, and generating spatial attention weights. The spatial weights are generated by the Sigmoid function. , used to enhance the motion characteristics of key joints;

[0115] The expression is: ;

[0116] In the formula represents the spatial attention weight, and Two learnable parameter matrices representing spatial attention, ‖ represents concatenation, For the The set of neighboring nodes of a joint, Indicates the The characteristics of the joints, Indicates the The characteristics of the neighborhood node set of a joint, Express Max pooling operation, is the Sigmoid function.

[0117] Temporal attention uses LSTM network to model temporal dependencies and calculates frame-level temporal attention weights. To highlight the action keyframe, the expression is:

[0118] ;

[0119] in, is the learnable parameter matrix of temporal attention, For the The temporal feature sequence of each joint in the time window 1 to T, LSTM is a long short-term memory network; T is the duration;

[0120] Channel attention captures the statistical characteristics of feature channels through global average pooling and uses a learnable parameter matrix Generate channel attention weights To suppress the noise feature dimension, the expression is:

[0121] ;

[0122] in, is the learnable parameter matrix of channel attention, is the original feature;

[0123] The process of feature fusion is to achieve feature adaptive enhancement by multiplying the space-time-channel elements of the triple attention weights, and then combine them with the original features. Perform weighted fusion to form multi-scale global spatiotemporal features that are sensitive to destructive behavior;

[0124] ;

[0125] in, It is a multi-scale global spatiotemporal feature.

[0126] This mechanism enables the model to have high feature focusing accuracy on subtle but critical motion patterns in complex backgrounds.

[0127] S3-3: Using spatiotemporal feature classification layer design, multi-scale global spatiotemporal features After global average pooling, the data is input into the fully connected layer and the behavior recognition probability is output through the Softmax function:

[0128] ;

[0129] Where, is the behavior recognition probability, is the classification weight matrix, represents the dimension, C is the number of behavior categories, and D is the feature dimension; is the bias term.

[0130] In step S4, to achieve joint judgment of behavioral risk and electrical parameter anomalies, a multimodal adversarial training framework is built. This framework combines transient electrical parameter analysis with lightweight current sensors and insulation resistance detection modules to establish a bidirectional verification mechanism for the visual behavior recognition and electrical linkage model. The mathematical modeling includes the following core elements:

[0131] Reference Figure 3 ,To address the modal differences between image data and electrical data, a cross-modal adversarial discriminator is designed:

[0132] Generator : Map image features and electrical features into a common embedding space to generate joint features:

[0133] ;

[0134] Where, is the joint feature, Image features are extracted from image data and come from the ST-GCN network; Electrical features are extracted from electrical data from sensors;

[0135] Discriminator : Determine whether the feature pair comes from the real data distribution. The loss function is:

[0136] ;

[0137] Where, is the discriminator loss, is the mathematical expectation, which is the calculation of the expected value of all samples in the data distribution; In order to add fake electrical features of Gaussian noise and force the generator to learn noise invariance, alternating training is adopted. In each iteration, the discriminator is updated 3 times and then the generator is updated once to avoid mode collapse.

[0138] In each iteration, the discriminator is updated 3 times:

[0139] The first discriminator updates the input real data pair , calculate the discriminator output , calculate the true loss , back propagation updates the discriminator parameters ;

[0140] The second discriminator updates the fake electrical signature generated by the input and forged joint features , For the added Gaussian noise, calculate the forgery discrimination loss , back propagation updates the discriminator parameters ;

[0141] The third discriminator update mixes the real and fake electrical features and calculates the total loss ,renew To maximize ;

[0142] In each iteration, the generator is also updated once:

[0143] Freeze the discriminator parameters , generate fake joint features , calculate the generator loss , back propagation updates the generator parameters .

[0144] The covariance matrix of constrained image features and electrical features approaches zero, retaining the unique information of each mode. In cable external damage detection, especially when the insulation layer is slightly damaged and the electrical signal is weak, the complementary role of image features is significant.

[0145] First, define the cable state abnormality detection function to quantify the current mutation amount More than 30% or insulation resistance is lower than the safety threshold Abnormal electrical events:

[0146] ;

[0147] in, is the current, is the rated current; is the insulation resistance;

[0148] When the improved spatiotemporal graph convolutional network (ST-GCN) outputs the behavior recognition probability When the system activates the electrical parameter check, c represents the behavior category. The electrical parameter acquisition unit monitors the cable current mutation in real time. and insulation resistance . Electrical anomaly confidence level Defined as:

[0149] ;

[0150] Furthermore, the spatiotemporal alignment constraints are established , the starting time of the dangerous behavior detected by the forced improved spatiotemporal graph convolutional network Electrical abnormality trigger time Synchronize within a ±0.5 second time window to avoid miscorrelating non-causal events;

[0151] ;

[0152] Where s stands for seconds.

[0153] Determine the risk level:

[0154] ;

[0155] Trigger responses based on risk level.

[0156] Through strict spatiotemporal correlation and fusion of multimodal features, the false alarm rate of traditional single-modal systems is greatly reduced, while ensuring that the entire link calculation is completed within 200ms to meet the real-time protection needs of urban power grids.

[0157] A graded response mechanism is implemented for different risk levels: for low-risk events, the system will activate the on-site sound and light alarm devices for early warning, and upload the alarm records to the central monitoring platform; when a high-risk situation is detected, the graded alarm system will trigger the power cut-off function of the intelligent circuit breaker within 200ms, and simultaneously transmit the fault location data in real time to the emergency repair terminal.

[0158] Furthermore, to adapt to new construction equipment, such as hydraulic shears and laser cutters, and effectively identify different types of damage, an edge-side incremental learning module was constructed within the federated learning collaborative framework to hierarchically update the parameters of the 3D human pose estimation algorithm and the improved spatiotemporal graph convolutional network:

[0159] Frozen layer: The HRNet (3D human pose estimation algorithm) backbone network parameters are fixed to retain general feature extraction capabilities and avoid catastrophic forgetting;

[0160] Fine-tunable layer: Only the weights of the last two layers of graph convolution of the improved spatiotemporal graph convolutional network (ST-GCN) are updated. The update formula is:

[0161] ;

[0162] Where, is the gradient, is the learning rate, is the elastic coefficient, For the newly generated dataset, is the learnable parameter matrix of temporal attention, To federate learning of global model parameters and prevent local models from deviating too far.

[0163] Build a federated learning collaborative framework, differential privacy parameter aggregation, and each edge node Based on local data Calculating gradients Conduct local training;

[0164] Add Laplace noise to protect privacy:

[0165] ;

[0166] Where, For edge nodes The gradient, Edge nodes after adding Laplace noise The gradient, is the Laplace noise generator, is the sensitivity, which can be set to 0.1. Budget for privacy and satisfy differential privacy.

[0167] A cable anti-extrinsic human behavior recognition system based on spatiotemporal graph convolution, including a multispectral camera, an electrical parameter acquisition unit, an edge computing terminal, a spatiotemporal graph convolution reasoning module, a fusion analysis module, and a hierarchical response device;

[0168] The multispectral camera adopts three-light fusion imaging technology, supports collaborative shooting of visible light, thermal infrared, and polarized light, enables visible light high-definition imaging when there is sufficient light during the day, and automatically switches to thermal imaging mode at night or in low visibility conditions such as haze. It enhances the contour extraction capability through the infrared radiation characteristics emitted by the human body. Its image sampling frequency is 30fps, and it is powered by PoE and connected to the edge computing terminal.

[0169] The electrical parameter acquisition unit acquires the cable surface current fluctuation and insulation resistance in real time;

[0170] The improved spatiotemporal graph convolutional network (ST-GCN) is used to jointly model the multi-scale global spatiotemporal features of actions through multiple layers of spatiotemporal convolutions and output the behavior recognition probability.

[0171] The edge computing terminal uses an embedded intelligent chip, integrated with an edge-side incremental learning module and a 3D human posture estimation algorithm. It can use the depth visual data acquired by a multispectral camera to analyze the 3D spatial coordinates of 17 key skeletal nodes of the human body in real time and construct a skeletal motion topology map containing dynamic time series associations. This skeletal motion topology map provides structured feature expression for subsequent behavior recognition through dual modeling of joint connection relationships in the spatial dimension and action continuity in the temporal dimension.

[0172] The fusion analysis module solves the modal differences between video and electrical signals by building a multimodal adversarial training framework, analyzes multimodal data together, and uses current sensors and insulation resistance detection modules to collect cable surface current fluctuations and insulation resistance in real time, and combines them with edge computing nodes to perform transient analysis of electrical parameters. When video behavior recognition detects cable damage behavior, it automatically activates the spatiotemporal alignment verification of the cable status monitoring unit, and implements double verification through the ±0.5s electrical anomaly time window and the matching degree calculation of the dangerous action video clip, controlling the system response delay within 200ms.

[0173] The hierarchical response device includes an audible and visual warning unit, a remote alarm interface, and an intelligent circuit breaker controller, and adopts a hierarchical response strategy:

[0174] Low risk: If a dangerous action (such as digging or metal striking) is detected, a strong flashing light will be activated on site and the alarm information will be uploaded to the monitoring center simultaneously.

[0175] High risk: When video analysis confirms that someone has damaged cables and the electrical anomaly data exceeds the safety threshold, the power supply to the corresponding section is automatically cut off and the location information is pushed to the emergency repair terminal.

[0176] The overall system architecture collects video data through multispectral cameras, and the edge computing terminal extracts three-dimensional skeleton data and constructs spatiotemporal topology. The spatiotemporal graph convolution reasoning module completes behavioral feature analysis and classification. Combined with the two-way verification mechanism of the electrical parameter acquisition unit, it finally triggers the response of the hierarchical response device.

[0177] The above description merely represents preferred embodiments of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above disclosure to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A cable anti-extrinsic human behavior recognition method based on spatiotemporal graph convolution is characterized by the following steps: as follows: S1: Acquire image data and electrical data through a multispectral camera and sensor respectively; S2: Preprocess the image data and construct a skeletal motion topology map containing dynamic associations of time series; S3: Using an improved spatiotemporal graph convolutional network to jointly model the multi-scale global spatiotemporal features of the action through multiple layers of spatiotemporal convolutions and output the behavior recognition probability; the improved spatiotemporal graph convolutional network includes an improved spatiotemporal graph convolutional layer and a channel-spatiotemporal attention module. The design of the improved spatiotemporal graph convolutional layer includes a spatial graph partitioning strategy, time dimension modeling, and dynamic adjacency matrix fusion; The improved spatiotemporal graph convolution layer: In the spatial dimension, the neighborhood nodes of each joint are divided into three categories through the spatial graph partitioning strategy: Root node: the current joint itself; Centripetal nodes: adjacent joints pointing to the root node; Centrifugal nodes: adjacent joints that are far away from the root node; Each partition is assigned an independent learnable weight matrix to capture the local features of different motion patterns and obtain a spatial adjacency matrix; Temporal dimension modeling refers to: using a one-dimensional temporal convolution kernel in the temporal dimension to perform temporal modeling on the motion trajectory of the same joint in consecutive frames to capture the temporal changes of the action; Set the learnable weight coefficient to fuse the spatial adjacency matrix and the temporal adjacency matrix into a dynamic adjacency matrix; The inverse square root of the degree matrix is ​​used as the normalization factor to perform symmetric normalization on the dynamic adjacency matrix; The channel-spatiotemporal attention module calculates the spatial attention weights respectively , temporal attention weight and channel attention weights , feature adaptive enhancement is achieved by multiplying the space-time-channel element by element of the triple attention weights, and then combined with the original features Perform weighted fusion to form multi-scale global spatiotemporal features; ； in, is the multi-scale global spatiotemporal feature: S4: Using a cross-modal adversarial discriminator to output the fusion behavior recognition probability, determine the risk level, and formulate a graded response strategy; the cross-modal adversarial discriminator includes: Generator G is used to map image features and electrical features into a common embedding space to generate joint features: ； Where, is the joint feature, is the image feature, which is extracted from the image data; It is the electrical feature, which is extracted from the electrical data; Discriminator : To determine whether the feature pair comes from the real data distribution, the loss function is: ； Where, is the discriminator loss, is the mathematical expectation, To add a fake electrical signature of Gaussian noise; In step S4, the process of outputting the fusion behavior recognition probability and implementing the graded response of the external failure with the electrical data at the current moment is as follows: Define the cable status anomaly detection function and quantify the current mutation amount More than 30% or insulation resistance is lower than the safety threshold Abnormal electrical events: ； in, is the current, is the rated current; is the insulation resistance; When the improved spatiotemporal graph convolutional network outputs the behavior recognition probability When the electrical parameter check is activated, c represents the behavior category; the electrical parameter acquisition unit monitors the cable current mutation in real time. and insulation resistance ; Electrical anomaly confidence level Defined as: ； Establishing spatiotemporal alignment constraints : ； in, The improved spatiotemporal graph convolutional network detects the starting time of dangerous behavior. is the electrical anomaly triggering time, s represents seconds; Determine the risk level: ， Trigger responses based on risk level.

2. The cable anti-breakage human behavior recognition method according to claim 1 is characterized in that: The pre-processing of the image data in step S2 is to fuse the three-channel data into a three-channel fusion feature map : ； Where, Visible light images , thermal infrared images , polarization degree The weight coefficient is dynamically adjusted according to the light intensity.

3. The cable anti-breakage human behavior recognition method according to claim 1 is characterized in that: In step S2, a skeletal motion topology graph including time series dynamic association is constructed by a three-dimensional human posture estimation algorithm.

4. The cable anti-breakage human behavior recognition method according to claim 3 is characterized in that: In step S2, a pre-trained 3D human pose estimation algorithm is used to extract the 3D coordinates of 17 joints from the three-channel fusion feature map generated by pre-processing, and a spatiotemporal topological map is constructed; Design a loss function as a continuity constraint for the skeleton trajectory, penalizing the change in joint positions between adjacent frames; Calculate the joint Euler angle based on the three-dimensional coordinates of the joint and preset biomechanical limits based on human anatomy and kinematics standards 、 contrast, is the minimum allowable angle of the current degree of freedom, is the maximum allowable angle of the current degree of freedom; Define the physical loss function : ； Where t is the index of the frame and i is the joint number; Constrain the joint Euler angles within the biomechanical limits through backpropagation; Designing a second-order smoothing loss Implement motion continuity constraints.

5. The cable anti-breakage human behavior recognition method according to claim 1 is characterized in that: The cross-modal adversarial discriminator adopts alternating training, updating the discriminator 3 times and then updating the generator once in each iteration; The first discriminator updates the input real data pair , calculate the discriminator output , calculate the true loss , back propagation updates the discriminator parameters ; The second discriminator updates the fake electrical signature generated by the input and forged joint features , For the added Gaussian noise, calculate the forgery discrimination loss , back propagation updates the discriminator parameters ; The third discriminator update mixes the real and fake electrical features and calculates the total loss , update the discriminator parameters To maximize the total loss ; Then, freeze the discriminator parameters , generate fake joint features , calculate the generator loss , back propagation updates the generator parameters .

6. The cable anti-breakage human behavior recognition method according to claim 1 is characterized in that: An edge-side incremental learning module is constructed under the federated learning collaborative framework to hierarchically update the parameters of the 3D human pose estimation algorithm and the improved spatiotemporal graph convolutional network.

7. A cable anti-extrinsic damage human behavior recognition system based on spatiotemporal graph convolution, used to implement the cable anti-extrinsic damage human behavior recognition method according to claim 1, characterized in that: It includes a multispectral camera, an electrical parameter acquisition unit, an edge computing terminal, a spatiotemporal graph convolution reasoning module, a fusion analysis module, and a hierarchical response device; The multispectral camera is connected to the edge computing terminal, and the multispectral camera is used to collect image data; The electrical parameter acquisition unit acquires the cable surface current fluctuation and insulation resistance in real time; The spatiotemporal graph convolutional reasoning module uses an improved spatiotemporal graph convolutional network to jointly model the multi-scale global spatiotemporal features of the action through multiple layers of spatiotemporal convolutions and outputs the behavior recognition probability; The edge computing terminal integrates an edge-side incremental learning module and a 3D human posture estimation algorithm, which constructs a skeletal motion topology graph containing dynamic associations of time series; The edge-side incremental learning module optimizes the parameters of the improved spatiotemporal graph convolutional network and 3D human pose estimation algorithm; The fusion analysis module uses a cross-modal adversarial discriminator to output the recognition probability of fusion behavior and determine the risk level; The hierarchical response device specifies a hierarchical response strategy according to the risk level.

Images