Power transmission corridor abnormal behavior real-time detection method based on space-time graph convolution network

Abstract

The application discloses a power transmission corridor abnormal behavior real-time detection method based on a space-time graph convolution network, and particularly relates to the field of intelligent monitoring of power transmission corridor videos, and solves the problem of unstable attribution and misjudgment caused by zooming of overlapping shielding of construction machinery approaching wires of multiple circuits on the same tower. The application extracts the target area of the construction machinery and the pixel position of the key points of the mechanical arm frame by frame, constructs a wire anchor point set, clusters the wire anchor point set according to circuits to form circuit wire cluster nodes, calculates the shortest pixel distance in a sliding time window, and generates a gap scale normalized distance from a gap scale, constructs a heterogeneous space-time graph containing candidate risk edges and cross-frame time edges, and adds shielding labels and scale mutation labels. Based on the lock signal coefficient, the candidate risk edge weight is gated, and then the circuit is input into a circuit mutual exclusion normalized attention mechanism space-time graph convolution network to output a circuit attribution lock state, complete abnormal approaching judgment and evidence package output, and output the circuit risk distribution when the circuit is not locked and trigger a cloud review request.

Description

Technical Field

[0001] This invention relates to the field of intelligent video monitoring of power transmission corridors, and more specifically, to a real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks. Background Technology

[0002] Video monitoring of power transmission corridors is commonly used for online early warning of external damage risks. The front end typically deploys fixed bullet cameras or pan-tilt-zoom (PTZ) cameras to cover the tower base, crossing areas, and corridor boundaries. Closed-loop handling is achieved through platform-side alarms, work orders, and joint prevention and control processes. In actual operation, dual-circuit or multi-circuit sections on the same tower are common. The conductors are distributed approximately parallel within the frame, forming a dense background along with insulator strings, hardware, and tower materials. Changes in imaging conditions such as PTZ patrol, zoom reset, slight wind-induced swaying, and rain, fog, and backlighting can cause significant fluctuations in the conductor edges and target appearance features within a short period. When construction machinery such as cranes, aerial work platforms, and excavators operate under the tower or within the corridor, their booms, hooks, and other slender components are prone to overlapping projections with conductors of different circuits under conditions of changing viewing angles, obstruction, and vibration. Furthermore, key points of these components may appear to jump or be missing within several frames.

[0003] Existing technologies mostly use target detection combined with multi-target tracking to obtain mechanical trajectories, and make judgments based on pixel distance, no-entry zone crossing, motion trend, or proximity threshold based on geometric models within each frame or short time window. Some solutions introduce wire extraction, prior line models, or point cloud information for ranging and classification. At the same time, there are also methods that use time smoothing or rule constraints to reduce jitter.

[0004] The aforementioned method works in scenarios with single loops or high conductor separation. However, in multi-loop scenarios on the same tower, the image domain distance difference between conductor clusters may be on the same order of magnitude as detection noise, zoom scale changes, and keypoint drift caused by occlusion. This leads to frequent alternation of the "nearest loop" between adjacent frames. The alternation process is often accompanied by detection category jitter, short-term loss and reconnection of tracking identity, resulting in discontinuous or incorrectly spliced ​​loop sequences corresponding to the same mechanical component within the time window. This, in turn, causes inconsistencies in loop pointing for the same event on the platform side, affecting alarm correlation, work order dispatch, and review and location. Furthermore, it may create the risk of duplicate alarms or missed alarms during high-frequency operation periods. If this inconsistency is not constrained, it will reduce the system's traceability and consistency for specific loop risks.

[0005] Therefore, there is a need for a detection method that can maintain the consistency of loop attribution and reduce false alarms when multiple loops on the same tower are projected and overlapped, and when there are occlusions and zoom disturbances. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks. This method extracts the target area of ​​construction machinery and the key pixel positions of the robotic arm frame by frame, constructs a set of conductor anchor points, and clusters these anchor points into loop conductor cluster nodes. Within a sliding time window, the shortest pixel distance is calculated and normalized using a gap scale to generate the gap scale net distance. A heterogeneous spatiotemporal graph containing candidate risk edges and cross-frame time edges is constructed with occlusion markers and scale mutation markers. Based on a locking coefficient, the weights of the candidate risk edges are gated, and the input loops are subjected to mutual exclusion normalization attention mechanism. The spatiotemporal graph convolutional network outputs the loop's locked state, completing the anomaly approximation judgment and evidence packet output. When not locked, the output loop risk distribution triggers a cloud-based review request, thus solving the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: S1: Process the multi-loop video stream on the same tower frame by frame, and extract the target area of ​​the construction machinery and the key point pixel position of the robotic arm; cluster the set of conductor anchor points according to the loop to obtain the loop conductor cluster node; S2: Calculate the shortest pixel distance from the key point of the robotic arm to the node of the loop wire cluster within the sliding time window, obtain the gap scale according to the anchor point spacing, and normalize it to the gap scale net distance. Based on the sliding time window, construct a heterogeneous spatiotemporal graph containing candidate risk edges and cross-frame time edges, and add occlusion markers and scale change markers to cross-frame time edges. S3: Extract edge evidence of loop pointing to fluctuation and edge evidence of loop belonging stability for candidate risk edges within the sliding time window. After fusion, obtain the locking coefficient. Based on this, perform gating weighting on the edge weights of candidate risk edges. Input the gated heterogeneous spatiotemporal graph into a spatiotemporal graph convolutional network with a loop mutual exclusion normalization attention mechanism to output the loop belonging locking state. S4: When the loop ownership lock state is established, select the gating candidate risk edge weight sequence corresponding to the locked loop, and combine the occlusion mark and scale change mark to determine the abnormal approximation and generate evidence package. When the loop ownership lock state is not established, output the loop risk distribution and non-convergence mark, and trigger the cloud review request.

[0008] Furthermore, the video stream of the multi-loop section on the same tower is processed frame by frame. The construction machinery target detection model outputs a set of construction machinery target areas and outputs category confidence information. The component key point extraction model outputs the root point, joint point and end point of the robotic arm to form a set of key points of the robotic arm within the set of construction machinery target areas, and outputs key point observation confidence markers and key point visibility markers.

[0009] Furthermore, the candidate region segmentation of the conductor yields the conductor probability map, skeletonization and straight line fitting yield the conductor segment set, structural point detection yields the tower head suspension point set and the insulator string connection point set, the endpoints of the conductor segment set and the structural points are merged to form the conductor anchor point set, and the anchor point observation confidence mark and anchor point visibility mark are output. The conductor anchor point set is clustered according to direction consistency, strip position identification, and tower head suspension point constraint to generate the loop conductor cluster node set.

[0010] Furthermore, within the sliding time window, the target area set of construction machinery for each frame is input, and the trajectory number of the construction machinery is generated by Kalman prediction and Hungarian matching. The set of key points of the robotic arm is indexed according to the trajectory number of the construction machinery and the key point type. The list of anchor points within each loop conductor cluster node in the loop conductor cluster node set is generated by pairing adjacent loops to generate a gap scale, and the visibility summary of loop anchor points is calculated.

[0011] Furthermore, the shortest pixel distance sequence is calculated based on the set of key points of the robotic arm and the set of nodes of the loop conductor cluster, and normalized to the gap marker net distance sequence. At the same time, relative orientation information and anchor point visibility information are generated. A heterogeneous spatiotemporal graph is constructed based on the nodes of the construction machinery, the key point nodes of the robotic arm, the loop conductor cluster nodes, and the anchor point nodes of the conductor. Candidate risk edges are kept in parallel, and the set of candidate risk edge sequences and the set of cross-frame time edge sequences are output. The cross-frame time edges carry occlusion markers and scale change markers.

[0012] Furthermore, within the sliding time window, the candidate risk edge sequence set and the cross-frame time edge sequence set are read. The frame index is filtered based on the occlusion marker, scale change marker, anchor point visibility information, and loop anchor point visibility, and a valid frame set is formed. The frame index with the scale change marker of one is used as the segment boundary, and the sequence is renumbered for each segment.

[0013] Furthermore, monotonic regression is performed on the gap mark net distance sequence corresponding to the effective frame set to obtain the trend sequence and generate the residual sequence. Frequency domain component analysis is performed on the residual sequence to extract the dominant ripple amplitude and form the gap mark ripple ratio with the background amplitude benchmark. The optimal loop index and the suboptimal loop index are determined based on robust aggregation. The difference sequence is subjected to sliding median filtering, and the symbol flip count and robust interval are statistically analyzed to form the bundle bias stability ratio.

[0014] Furthermore, evidence mapping is performed based on the gap-mark ratio and the beam-bias-stability ratio, and the locking-confidence coefficient is generated by fusing the Depster synthesis rule. The locking-confidence coefficient is used to gating the edge weights of candidate risk edges and generate the gating post-edge weight sequence. The gating post-edge weight sequence and the attention distribution of the spatiotemporal graph convolutional network output loop of the heterogeneous spatiotemporal graph input loop are mutually exclusive. Based on the attention convergence interval, the persistence determination of the locking-confidence coefficient, the occlusion marker count, and the scale change marker count, the loop-attribution locking state and the locking loop index are generated.

[0015] Furthermore, based on the loop ownership and locking status, the locked branch is divided into locked and unlocked branches. The locked branch has a fixed locked loop index and extracts the gating back weight sequence. The effective frame index is screened by combining the occlusion mark and the scale change mark, and a set of effective segments is formed. Based on the gating back weight sequence and the relative orientation information, the approximation intensity index, persistence index, and orientation consistency index are calculated, and the abnormal approximation judgment result is output.

[0016] Furthermore, the locked branch extracts the keyframe index set within the effective frame index and generates an evidence package. The evidence package includes the locked loop index, the keyframe index set, the key point trajectory summary, and the gated back weight evolution summary. The key point trajectory summary is generated using a polyline simplification strategy. The unlocked branch outputs the loop risk distribution vector and the unlocked review trigger mark, and generates a cloud review request. The cloud review request uses the same field structure as the evidence package, but the locked loop index field is set to empty.

[0017] The technical effects and advantages of the real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks in this invention are as follows: This invention focuses on the abnormal approach identification of construction machinery in multi-loop sections of the same tower, forming a closed-loop link from "target positioning and conductor structure characterization" to "distance scale unification and relationship modeling" and then to "loop attribution locking and risk assessment". The target area of ​​the construction machinery and the key points of the robotic arm locate the risk source to the specific component action. The conductor anchor point set and the loop conductor cluster nodes decompose the protected object into an identifiable loop structure. The gap scale converts the pixel distance fluctuation caused by the zoom and viewpoint change into a comparable gap scale clearance, so that the degree of approach at different time periods and different lens states can be continuously tracked on the same scale, thereby reducing loop pointing jumps and misjudgments caused by the overlapping of multi-loop conductor projections.

[0018] Based on this, the gap mark ratio and beam offset stability ratio distinguish between "loop alternating jitter" and "stable approximation," and the candidate risk edges are gated by the locking confidence coefficient, so that the loop attribution locking is completed based on the more credible loop relationship in the spatiotemporal reasoning stage. When the loop attribution locking state is established, the abnormal approximation judgment combines the occlusion mark, scale change mark and orientation consistency constraint to avoid false alarms triggered by occluded segments and zoom segments. At the same time, a structured evidence package is output, which allows the reviewers to directly locate and lock the loop and key frame and reproduce the approximation process. When the loop attribution locking state is not established, the loop risk distribution vector and the unlocked review trigger mark separate the uncertain situation from the on-site judgment, reduce the risk of erroneous handling and ensure the consistency of the subsequent review entry fields. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1: Figure 1 This invention presents a real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks, comprising: S1: Process the multi-loop video stream on the same tower frame by frame, and extract the target area of ​​the construction machinery and the key point pixel position of the robotic arm; cluster the set of conductor anchor points according to the loop to obtain the loop conductor cluster node; S2: Calculate the shortest pixel distance from the key point of the robotic arm to the node of the loop wire cluster within the sliding time window, obtain the gap scale according to the anchor point spacing, and normalize it to the gap scale net distance. Based on the sliding time window, construct a heterogeneous spatiotemporal graph containing candidate risk edges and cross-frame time edges, and add occlusion markers and scale change markers to cross-frame time edges. S3: Extract edge evidence of loop pointing to fluctuation and edge evidence of loop belonging stability for candidate risk edges within the sliding time window. After fusion, obtain the locking coefficient. Based on this, perform gating weighting on the edge weights of candidate risk edges. Input the gated heterogeneous spatiotemporal graph into a spatiotemporal graph convolutional network with a loop mutual exclusion normalization attention mechanism to output the loop belonging locking state. S4: When the loop ownership lock state is established, select the gating candidate risk edge weight sequence corresponding to the locked loop, and combine the occlusion mark and scale change mark to determine the abnormal approximation and generate evidence package. When the loop ownership lock state is not established, output the loop risk distribution and non-convergence mark, and trigger the cloud review request.

[0022] In the corridor video of the multi-circuit section of the same tower, the construction machinery and the conductors are in the same frame and frequently obscure each other. The conductors in the tower head area are multiple slender structures that are almost parallel. It is difficult to depict the geometric relationship between the robotic arm components and the conductor loops by simply relying on the whole machine target frame. It is also difficult to stably distinguish the multi-circuit conductors into loop objects by simply relying on conductor extraction. Therefore, it is necessary to simultaneously complete the positioning of the construction machinery and the key point positioning of the robotic arm in the same frame, and organize the conductor structure points into a set of conductor anchor points and a cluster node of loop conductors that can distinguish the loops.

[0023] S101 frame reading and pixel coordinates are unified.

[0024] In the case of multi-loop segment video streams on the same tower, zooming, jitter and occlusion occur alternately under frame-by-frame conditions. Step S2 needs to directly reuse the key point coordinates and anchor point coordinates in the same coordinate system. Therefore, the pixel coordinates are fixed in step S1.

[0025] The video stream reads image frames in chronological order and assigns frame numbers. The pixel coordinates use the top left corner as the origin, with the horizontal direction to the right being the direction of increasing x-coordinate and the vertical direction to the bottom being the direction of increasing y-coordinate. All coordinate values ​​are uniformly expressed in pixels.

[0026] Pixel coordinate out-of-bounds handling uses a cropping rule, which is executed in the order of operations: first, it checks if the x-coordinate is less than zero; if so, it is rewritten to zero. Then, it checks if the x-coordinate is greater than the maximum column coordinate of the image; if so, it is rewritten to the maximum column coordinate. The y-coordinate is processed in the same order: first, it checks if the y-coordinate is less than zero and rewrites it to zero; then, it checks if the y-coordinate is greater than the maximum row coordinate of the image and rewrites it to the maximum row coordinate.

[0027] The cropped pixel coordinates are then used in subsequent sub-steps to ensure that the coordinates of the target area of ​​the construction machinery, the key points of the robotic arm, and the anchor points of the guide wires are all within the same pixel domain.

[0028] S102 Construction Machinery Target Area Set Generation and Overlap Suppression.

[0029] In a multi-circuit scenario on the same tower, limiting the target area of ​​the construction machinery can reduce the interference of the background of the conductor on the extraction of key points. Step S2 requires limiting the source of key point nodes with the target area set of the construction machinery. Therefore, the target area set is determined in step S1.

[0030] Image frames are input into the construction machinery target detection model, which outputs a list of candidate target regions. Each candidate target region is represented by the coordinates of its upper left and lower right corner pixels. The candidate target regions also output target presence confidence information and category confidence information. The target presence confidence information ranges from zero to one, and the category confidence information ranges from zero to one. The category label is limited to a preset set of construction machinery categories.

[0031] Candidate target region filtering is performed in the order of operations. First, the comprehensive confidence information of the candidate target region is calculated, which is equal to the product of the target existence confidence information and the category confidence information. Then, candidate target regions with comprehensive confidence information lower than the corresponding preset threshold are eliminated. The preset threshold is determined by offline evaluation using labeled samples and written into the configuration file before deployment.

[0032] Overlap suppression is performed in the order of operations. First, the overlap area ratio is calculated for any two candidate target regions. The overlap area ratio is equal to the intersection area of ​​the two target regions divided by the union area of ​​the two target regions. Both the intersection area and the union area are in pixel area. Then, it is determined whether the overlap area ratio exceeds the corresponding preset threshold. If it does, only the candidate target region with higher comprehensive confidence information is retained and the other candidate target region is removed.

[0033] The target region set for construction machinery consists of candidate target regions after overlap suppression. Each target region for construction machinery records the upper left pixel coordinates, lower right pixel coordinates, category label, target existence confidence information, category confidence information, and comprehensive confidence information. All confidence information is in the range of zero to one.

[0034] Example: A crane appears in the picture. The construction machinery target detection model outputs two highly overlapping candidate target regions of the crane. The overlap area ratio exceeds the corresponding preset threshold. The candidate target region with higher comprehensive confidence information enters the construction machinery target region set, and the other candidate target region is eliminated.

[0035] S103 Robotic Arm Key Point Set Generation and Visibility Marking.

[0036] In the multi-loop section of the same tower, the problem of loop alternation caused by the overlap of the boom end and the conductor projection is mainly triggered by the drift of the end point and the joint point. Step S2 needs to construct key point nodes based on the pixel position sequence of the key points of the robotic arm. Therefore, the generation of key points must have deterministic coordinates and visibility markers.

[0037] For each construction machinery target area, a component key point extraction model is input. The component key point extraction model outputs three types of key point heat maps within the construction machinery target area. The three types of key points correspond to the root point, joint point, and end point of the robotic arm. The pixel values ​​of the heat map range from zero to one.

[0038] The keypoint pixel coordinates are determined according to the order of operations: first, the location of the maximum pixel value is searched in the corresponding heatmap; then, the pixel coordinates of the location of the maximum pixel value are used as the keypoint pixel coordinates; the keypoint observation confidence flag is taken as the maximum pixel value, with a value range of zero to one.

[0039] The visibility markers of key points are determined according to the order of operations. First, it is determined whether the observation confidence marker of the key point is not lower than the corresponding preset threshold. If it is not lower, the visibility marker of the key point is set to one; if it is lower, the visibility marker of the key point is set to zero. Then, the coordinate determination rules are applied to the key points whose visibility markers are zero, and the pixel coordinates of the key points are rewritten as the pixel coordinates of the geometric center of the target area of ​​the construction machinery. The pixel coordinates of the geometric center of the target area of ​​the construction machinery are calculated according to the order of operations. First, the average value of the horizontal coordinates of the upper left corner and the horizontal coordinates of the lower right corner is taken as the horizontal coordinate of the center, and then the average value of the vertical coordinates of the upper left corner and the vertical coordinates of the lower right corner is taken as the vertical coordinate of the center.

[0040] The set of key points for the robotic arm is output one by one according to the target area of ​​the construction machinery. For each target area of ​​the construction machinery, the key point pixel coordinates of the root point, joint point, and end point, the key point observation confidence flag, and the key point visibility flag are output. The key point observation confidence flag has a value range of zero to one, and the key point visibility flag has a value range of zero or one.

[0041] Example: In the image, the end of the crane boom is obscured by the tree canopy. The peak value of the heat map of the end point is not obvious in the component key point extraction model. The key point observation confidence mark is lower than the threshold. The end point visibility mark is set to zero. The end point pixel coordinates are rewritten as the geometric center pixel coordinates of the construction machinery target area. This makes it easier for step S2 to form a continuous sequence within the time window and identify the obscured segment through the visibility mark.

[0042] Extraction of anchor point set and annotation of anchor point attributes for S104 conductor.

[0043] In a multi-loop scenario on the same tower, the conductor position and tower head structural point are used for loop segmentation and loop clustering. Step S2 requires calling the loop conductor cluster anchor point set to calculate the shortest pixel distance and generate a gap scale. Therefore, the conductor anchor point set must contain both conductor segment endpoints and structural points, and provide visibility markers and observation confidence markers.

[0044] The image frame is input to the conductor candidate region segmentation model, which is implemented using a semantic segmentation network, such as DeepLabv3+, U-Net, and HRNet-OCR. The model outputs a conductor probability map, where pixel values ​​range from zero to one. The conductor probability map is binarized sequentially: first, pixels with values ​​at or above a preset threshold are designated as conductor pixels; then, the remaining pixels are designated as non-conductor pixels, resulting in a conductor binary mask.

[0045] The binary mask of the conductor is input into the skeletonization algorithm to obtain a single-pixel wide skeleton. The skeleton is input into the straight line fitting algorithm to obtain the set of conductor segments. The straight line fitting algorithm adopts random sampling consistency straight line fitting or probabilistic Hough straight line transformation. Each line segment in the set of conductor segments is represented by the starting pixel coordinates and the ending pixel coordinates.

[0046] The image frame is input to a structure point detection model (e.g., Keypoint R-CNN, CenterNet, and HRNet). This model outputs structure point coordinates through heatmap peak search and non-maximum suppression. The model outputs a set of tower head hanging points and a set of insulator string connection points. Each structure point records its pixel coordinates and an observation confidence flag, with the confidence flag ranging from zero to one. The set of conductor segment endpoints consists of all the start and end points of the conductor segment set. The observation confidence flag for each endpoint is calculated in the following order: first, the set of conductor probability values ​​within a fixed neighborhood of the endpoint's pixel coordinates is taken from the conductor probability map; then, the median of this set is used as the endpoint observation confidence flag, also ranging from zero to one.

[0047] The conductor anchor point set is obtained by merging the tower head attachment point set, the insulator string connection point set, and the conductor segment endpoint set. Each anchor point records the anchor point type, pixel coordinates, anchor point observation confidence flag, and anchor point visibility flag. The anchor point visibility flag is determined according to the calculation order: first, it is determined whether the anchor point observation confidence flag is not lower than the corresponding preset threshold. If it is lower, the anchor point visibility flag is set to zero and the judgment ends; if it is not lower, it enters the consistency judgment.

[0048] Consistency determination is performed in the order of operations. First, the dominant guide line segment near the anchor point is determined. The selection rule for the dominant guide line segment is to search for the line segment with the smallest Euclidean distance to the anchor point pixel coordinates in the set of guide line segments and select this line segment as the dominant guide line segment. Next, the direction of the dominant guide line segment is calculated. The direction of the dominant guide line segment is obtained by subtracting the pixel coordinates of the starting point of the line segment from the pixel coordinates of the ending point of the line segment. Then, the image gradient direction of the anchor point neighborhood is calculated. The image gradient direction is calculated by using the Sobel operator to calculate the horizontal gradient and vertical gradient and form a gradient vector. Then, the direction consistency between the gradient vector and the direction vector of the dominant guide line segment is calculated pixel by pixel in the anchor point neighborhood. The direction consistency is taken as the cosine value of the angle between the two vectors, and the value of the cosine angle ranges from negative one to one. Then, the median of the direction consistency of all neighborhoods is taken as the anchor point neighborhood consistency index. Finally, it is determined whether the anchor point neighborhood consistency index is not lower than the corresponding preset threshold. If it is not lower, the anchor point visibility flag is set to one; otherwise, the anchor point visibility flag is set to zero.

[0049] The anchor point observation confidence flag ranges from zero to one, the anchor point visibility flag ranges from zero to one, and the output of the conductor anchor point set is used for loop clustering in step S1 and can be directly called in step S2.

[0050] S105 conductor anchor points are clustered by loop and a set of loop conductor cluster nodes is established.

[0051] In a multi-loop section on the same tower, the conductors of each loop are distributed in parallel strips in the image. Loop clustering requires the consistency of conductor segment direction and strip distribution, and the tower head hanging point is used as a constraint on the loop structure to avoid misclassification of loop clusters due to simply relying on the conductor endpoints.

[0052] Loop clustering is performed in the order of operations. First, the direction angle of each line segment in the set of conductor segments is calculated. The direction angle is determined by the direction vector formed by the start and end points of the line segment. Then, the median of the direction angles of all line segments is taken as the dominant direction angle. Then, the set of parallel line segments is filtered. The filtering rule is that the difference between the line segment direction angle and the dominant direction angle does not exceed the corresponding preset threshold.

[0053] In the set of parallel line segments, a strip position identifier is calculated for each line segment. The strip position identifier is calculated in the order of operation. First, a reference line is constructed at the center of the image, and the direction of the reference line is taken as the dominant direction angle. Then, the pixel coordinates of the midpoint of the line segment are taken. Then, the signed vertical distance from the midpoint of the line segment to the reference line is calculated. The signed vertical distance is in pixels, and the sign is determined by the relationship between the midpoint and the reference line.

[0054] Sort all signed vertical distances in ascending order of value, and perform one-dimensional split clustering. The split clustering is performed in the order of operation. First, create the first cluster starting from the smallest distance. Then, compare the differences between adjacent distances in turn. When the difference exceeds the corresponding preset threshold, start a new cluster. Otherwise, it is assigned to the current cluster. The resulting set of clusters is called the wire-strip cluster set.

[0055] Assign the set of wire anchor points to the set of wire strip clusters. The assignment rule is to calculate the signed vertical distance from the anchor point pixel coordinates to the reference line and assign the anchor point to the wire strip cluster with the closest signed vertical distance.

[0056] Structural constraint correction is performed using the tower head attachment point set. The structural constraint correction is performed in the order of operation: first, the number of tower head attachment points in each conductor strip cluster is counted; then, conductor strip clusters lacking tower head attachment points are identified; for conductor strip clusters lacking tower head attachment points, a merging rule is applied, which merges the conductor strip cluster with the nearest adjacent conductor strip cluster. The nearest neighbor is determined by the smallest difference in the center values ​​of the signed vertical distance between the two clusters; after merging, the center value of each cluster is recalculated and the check for missing tower head attachment points is repeated until all conductor strip clusters contain at least one tower head attachment point or the number of conductor strip clusters no longer changes.

[0057] The set of loop conductor cluster nodes is generated from the modified set of conductor strip clusters. The loop index is numbered in ascending order of the signed vertical distance center value. Each loop conductor cluster node records the loop index and the list of anchor points within the cluster. The list of anchor points within the cluster includes anchor point type, pixel coordinates, anchor point observation confidence flag, and anchor point visibility flag.

[0058] Example: In the image, the double-loop conductors on the same tower are distributed in two parallel strips. After screening the parallel line segments, two groups of signed vertical distance concentration areas are formed. The split clusters are formed into two clusters. The tower head hanging points are distributed into the two clusters respectively. The structural constraint correction does not trigger merging. The loop index is numbered in a fixed order from top to bottom or from bottom to top, resulting in two loop conductor cluster nodes and each carrying a list of anchor points within the cluster.

[0059] The system outputs the target area and category confidence information of the construction machinery frame by frame. Within the target area of ​​the construction machinery, the pixel position sequence of the root point, joint point and end point of the robotic arm is output. At the same time, within the same frame, the set of conductor anchor points consisting of tower head hanging point, insulator string connection point and end point of long straight conductor segment is obtained. The set of conductor anchor points completes loop clustering and forms loop conductor cluster nodes based on the spatial arrangement relationship of conductors and the structural constraints of the same tower. Visibility markers and observation confidence markers are added to the set of conductor anchor points.

[0060] In multi-loop scenarios on the same tower, the pixel distance from the key point of the robotic arm to the wire fluctuates with the zoom of the gimbal, the shooting distance and the angle of view. The same approximation behavior may present different pixel scales in different frames. The adjacent loop projections cause the key point to be close to the multiple loops, resulting in loop pointing swing. Therefore, it is necessary to transform the geometric distance into a sequence of gap marker net distances with consistent scale within a fixed length sliding time window, and maintain the parallel candidate relationship and cross-frame consistency constraints of the key point to all loops in the form of a heterogeneous spatiotemporal graph.

[0061] The S201 sliding time window is aligned with the construction machinery trajectory number.

[0062] In multi-loop sections on the same tower, the movement of construction machinery within the camera's view can cause multiple target areas for the machinery to appear in different frames. Step S3 requires that the candidate risk edges from key points to loops form a continuous sequence. Therefore, within the sliding time window, a stable trajectory number needs to be established for the same construction machinery, and the key point index of the robotic arm needs to be bound. The sliding time window has a fixed length and includes the current frame and several historical frames. The frames within the sliding time window are arranged chronologically.

[0063] The trajectory number of the construction machinery is generated through multi-target tracking. The multi-target tracking input is the set of target regions of the construction machinery in each frame. The multi-target tracking adopts the association method of Kalman prediction plus Hungarian matching. The association threshold is a joint threshold of the overlap area ratio threshold and the center distance threshold. The overlap area ratio ranges from zero to one, and the center distance is in pixels.

[0064] The trajectory number generation rules include four items. The first item is to use Kalman prediction to obtain the predicted target area for each trajectory in the previous frame. The second item is to calculate the overlap area ratio and center distance between each construction machinery target area and each predicted target area in the current frame. The third item is to enter the Hungarian matching matrix and complete a one-to-one match for combinations that meet the joint threshold. The fourth item is to create a new trajectory number for unmatched construction machinery target areas. Unmatched trajectories enter the mismatch count and terminate when the mismatch count exceeds the corresponding preset threshold.

[0065] The binding rule for the set of key points of the robotic arm is that each construction machinery trajectory number corresponds to a set of robotic arm root points, robotic arm joint points, and robotic arm end points in each frame. The key point index is determined by the construction machinery trajectory number and the key point type. The key point visibility flag takes a value of zero or one, and the key point observation confidence flag takes a value of zero to one.

[0066] S202 Gap Scale Generation and Loop Anchor Point Visibility Summary.

[0067] In a multi-loop section of the same tower, the pixel distance changes with zoom. Directly using the pixel distance will amplify the loop alternation phenomenon. Therefore, each frame needs to extract a stable scale from the same tower structure to form a gap scale, and simultaneously output the loop anchor point visibility summary for use by candidate risk edge features.

[0068] The gap scale generation prioritizes the use of tower head attachment points from adjacent loops, and the anchor point selection rules are executed in the following order: 1. For each loop conductor cluster node, select the tower head attachment points and retain the tower head attachment points with a visibility marker of one; 2. Adjacent loops are processed in pairs. For the first loop, the tower head anchor point whose projected coordinates along the dominant direction of the conductor are closest to the median projected coordinates are selected. For the second loop, the tower head anchor point whose projected coordinates are smallest are selected from the tower head anchor point of the first loop, thus forming an anchor point pair. 3. The anchor point uses Euclidean distance for pixel spacing. The unit of Euclidean distance is pixels, and the Euclidean distance is always greater than zero. IV. Within the same frame, the pixel spacing of all anchor points is aggregated using the median to form a gap scale, with the unit of the gap scale being pixels and greater than zero.

[0069] Missing rollback rules are executed in the following order: 1. When there are insufficient anchor points on the tower head, use insulator strings to connect the points and perform the same anchor point pair selection and median aggregation process; 2. When there are insufficient connection points for insulator strings, the spacing of the strip center line of the loop segment set is used as the anchor point to pixel spacing. The strip center line is obtained by fitting the set of midpoints of the loop segment. 3. When the strip centerline is unavailable, use the previous frame's gap ruler and add a gap ruler availability mark of zero.

[0070] The loop anchor visibility summary is performed once per frame for each loop conductor cluster node. The loop anchor visibility summary is the median of the visibility marker set within the loop anchor set, and the value is either zero or one.

[0071] Example: In the dual-circuit image of the same tower, the visibility mark of the hanging point on one side of the tower head is zero due to the tree canopy obscuring it, while the visibility mark of the hanging point on the other side of the tower head is one. The insufficient hanging points of the tower head trigger the first backtracking and use the insulator string to connect the points to form an anchor point pair. The gap scale continues to output in pixel units.

[0072] S203 Shortest Pixel Distance and Gauge Clear Distance Calculation.

[0073] In a multi-circuit section on the same tower, the overlap between the end of the boom and the projection of the conductor can cause the anchor point distance to jump. Therefore, the shortest pixel distance needs to refer to both the set of loop anchor points and the set of loop segments, and the line segment distance is used to maintain continuity when anchor points are missing or fluctuate.

[0074] The rules for generating the loop segment set are determined in step S2. The loop segment set comes from the set of conductor segments and the set of loop conductor cluster nodes output in step S1. The allocation logic is to assign each conductor segment to the loop conductor cluster node that is closest to the strip position identifier according to the strip position identifier. The calculation method of the strip position identifier is the same as the loop clustering in step S1. The loop segment set is updated with each frame.

[0075] The shortest pixel distance is calculated frame-by-frame for each keypoint type and each loop conductor cluster node for each construction machinery trajectory number.

[0076] The calculation process for the distance from keypoints to the loop anchor set includes three steps: the first step is to traverse all anchor points in the loop anchor set; the second step is to calculate the Euclidean distance between the pixel coordinates of the keypoints and the pixel coordinates of the anchor points and record the minimum value; and the third step is to output the distance from the keypoints to the loop anchor set, in pixels and not less than zero.

[0077] The calculation process for the distance from the keypoint to the loop segment set includes four steps. The first step is to traverse all segments in the loop segment set. The second step is to construct the segment direction based on the segment's start and end points and calculate the projection point from the keypoint to the segment. The third step is to take the projection point as the nearest point when it falls between the start and end points, take the start point as the nearest point when it falls outside the start point, and take the end point as the nearest point when it falls outside the end point. The fourth step is to calculate the Euclidean distance from the keypoint to the nearest point and record the minimum value, and output the distance from the keypoint to the loop segment set in pixels, which is not less than zero.

[0078] The shortest pixel distance is the smaller of the distance from the keypoint to the set of loop anchor points and the distance from the keypoint to the set of loop segments, in pixels and not less than zero.

[0079] The gap marker clearance calculation process includes two steps. The first step is to read the gap marker of the same frame. The second step is to divide the shortest pixel distance by the gap marker to obtain the gap marker clearance. The gap marker clearance is dimensionless and not less than zero.

[0080] The relative orientation information calculation process includes three steps. The first step is to determine the nearest loop point. The nearest loop point is obtained from the anchor point or the closest point of the line segment that generates the shortest pixel distance. The second step is to calculate the displacement vector of the key point pointing to the nearest loop point. The third step is to input the horizontal and vertical components of the displacement vector into the arctangent two-parameter function to obtain the angle. The relative orientation information value range is from the negative circumference to the positive circumference.

[0081] Anchor point visibility information is taken from the anchor point visibility marker with the minimum distance from the keypoint to the loop anchor point set, and the value is zero or one. When the shortest pixel distance is generated by the loop segment set distance, the anchor point visibility information is taken from the loop anchor point visibility summary.

[0082] Example: In the image, the end point of the crane is close to the two-circuit conductors. In a certain frame, the connection point of the insulator string is flooded by strong light, causing the anchor point set distance to suddenly increase. The set distance of the return line segment remains stable. The shortest pixel distance is determined by the set distance of the return line segment. The gap marker net distance sequence remains continuous.

[0083] S204 Heterogeneous Spatiotemporal Graph Construction and Temporal Consistency Marker Generation.

[0084] In the multi-loop section of the same tower, mutually exclusive loop reasoning requires parallel candidate risk edges, and occlusion and zooming require time consistency marking. Therefore, it is necessary to construct a heterogeneous spatiotemporal graph and add markings to cross-frame connections within the sliding time window.

[0085] The nodes of the heterogeneous spatiotemporal graph include the complete construction machinery node, the key point node of the robotic arm, the loop conductor cluster node, and the conductor anchor point node. The complete node of the construction machinery is generated from the target area set of the construction machinery according to the trajectory number of the construction machinery and carries the pixel coordinates and confidence information of the target area; The key point nodes of the robotic arm are generated from the set of key points of the robotic arm according to the construction machinery trajectory number and carry the pixel coordinates of the key points, the observation confidence markers of the key points, and the visibility markers of the key points; The loop conductor cluster node is generated from the loop conductor cluster node set and carries the loop index and loop anchor visibility summary; The traverse anchor node is generated from the traverse anchor set and carries the anchor pixel coordinates, anchor observation confidence flag, and anchor visibility flag; The component structure edges are connected to the key point nodes corresponding to the root point nodes of the robotic arm and the joint points of the robotic arm according to the same construction machinery trajectory number, and the key point nodes corresponding to the joint points of the robotic arm are connected to the end point nodes of the robotic arm. Candidate risk edges connect all loop wire cluster nodes to each key node of the robotic arm and maintain parallel candidate relationships. The edge features of the candidate risk edges include gap marker clearance, relative orientation information, anchor point visibility information, and loop anchor point visibility summary. The gap marker clearance is a non-negative dimensionless value, the relative orientation information ranges from the negative circumference to the positive circumference, the anchor point visibility information is zero or one, and the loop anchor point visibility summary is zero or one. Cross-frame time edges connect nodes of the same type with the same construction machinery trajectory number in adjacent frames; Occlusion markers are generated for key nodes of the robotic arm. The occlusion markers are the inverse of the key node visibility markers, and their values ​​are either zero or one. The scale mutation marker is generated for the nodes of the construction machinery. The scale mutation marker judgment process includes three steps: the first step is to calculate the diagonal length of the target area; the second step is to calculate the ratio of the diagonal length of the current frame to the diagonal length of the previous frame; and the third step is to set the scale mutation marker to one if the ratio exceeds the corresponding preset threshold, otherwise set it to zero. The scale mutation marker takes the value of zero or one.

[0086] S205 serialization output.

[0087] Within the sliding time window, output the gap scale sequence, gap scale clearance sequence, relative orientation information sequence, anchor point visibility information sequence, and loop anchor point visibility summary sequence.

[0088] Within the sliding time window, a set of candidate risk edge sequences is output. The set of candidate risk edge sequences is indexed by construction machinery trajectory number, key point type, and loop index and arranged in frame order. Within the sliding time window, a set of cross-frame time edge sequences is output. The set of cross-frame time edge sequences includes occlusion marker sequences and scale change marker sequences and is bound by node index keys.

[0089] Step S3 directly reads the gap marker net distance sequence with the same index key and performs monotonic trend fitting and frequency domain component analysis. Step S3 directly reads the gap marker net distance sequences corresponding to the optimal loop and the suboptimal loop to form a difference sequence and performs robust smoothing and flip statistics. Step S3 directly reads the occlusion marker sequence and the scale mutation marker sequence to determine the persistence of the lock-in coefficient and performs shielding and degradation rules.

[0090] Within a sliding time window, the shortest pixel distance sequence from the key points of the robotic arm to each loop conductor cluster node is calculated. The pixel spacing between anchor points is generated from the set of conductor anchor points according to a preset pairing rule to obtain a gap scale through robust statistics. The shortest pixel distance sequence is normalized into a gap scale net distance sequence according to the gap scale. Based on the time window data, a heterogeneous spatiotemporal graph containing the nodes of the construction machinery, the key points of the robotic arm, the loop conductor cluster nodes, and the conductor anchor points is constructed. Candidate risk edges carry gap scale net distance and relative orientation information as well as anchor point visibility information. Cross-frame time edges carry occlusion markers and scale change markers.

[0091] Within the overlapping area of ​​multi-loop conductor projections, the alternation of candidate risk edges between loops is not equivalent to true approximation. Occlusion and scale abrupt changes can cause breaks and false peaks in the gap marker distance sequence. Directly relying on the minimum distance of a single frame can easily misjudge jitter and occlusion as risk escalation. Therefore, it is necessary to extract evidence that can distinguish between alternating loop ripples and stable approximation from the candidate risk edge sequence within a sliding time window, and convert the evidence into a locking coefficient to gate the candidate risk edges. Then, loop mutual exclusion normalization attention is used to form the loop attribution locking state.

[0092] S301 Valid Frame Set Construction and Sequence Renumbering.

[0093] The occlusion marker sequence and the scale mutation marker sequence determine the available segments of the sequence, while the loop anchor visibility summary sequence and the anchor visibility information sequence determine whether the loop observation has a geometric reference. The joint screening of the four types of markers can avoid writing the jump caused by the observation gap into the ripple evidence.

[0094] The effective frame set is constructed independently for each candidate risk edge within a sliding time window. The filtering order is fixed in three steps: First, the frame index with occlusion marked as zero is retained. Second, the frame index with scale change marked as one is removed from the retained frame index, and the frame index with scale change marked as one is used as the segment boundary. Third, in each segment after segmentation, the frame index with loop anchor visibility summed to one and anchor visibility information of one is retained. When the number of retained frame indexes in a certain segment is insufficient to support frequency domain analysis, the entire segment is marked as invalid and exits the subsequent gap mark ratio calculation.

[0095] Sequence renumbering maps the retained frame index to consecutive serial numbers according to time sequence. The consecutive serial numbers are used for subsequent residual frequency domain component analysis. Renumbering only changes the index and does not change the sequence value of the gap marker distance.

[0096] S302 Gap Mark Ratio Calculation.

[0097] In multi-loop scenarios on the same tower, loop alternation is usually manifested as a quasi-periodic oscillation of the gap marker net distance sequence around the trend line. The combination of monotonic trend fitting and residual frequency domain component analysis is used to separate the trend approximation from the alternating ripple.

[0098] Monotonic trend fitting is performed on the gap marker net distance sequence corresponding to the effective frame set. The fitting objective is defined as generating a trend sequence that does not increase over time, such that the sum of the absolute values ​​of the difference between the gap marker net distance and the trend value in each frame is minimized. The fitting algorithm adopts monotonic regression and uses a solution process that merges adjacent violators in the pool. The residual sequence is formed by subtracting the trend value of the same frame from the gap marker net distance in each frame.

[0099] The residual sequence is subjected to discrete Fourier transform according to consecutive numbers to obtain the frequency grid amplitude sequence. The dominant ripple amplitude is defined as the frequency grid amplitude with the largest amplitude other than zero frequency. The background amplitude benchmark is defined as the median of the amplitude set after excluding the dominant ripple frequency grid and adjacent frequency grids. When the background amplitude benchmark set is empty, the background amplitude benchmark is taken as the median of the remaining amplitude set excluding zero frequency and the neighborhood of the dominant ripple. When the remaining set is still empty, the gap mark ripple ratio is marked as invalid and withdrawn from the subsequent evidence fusion of this candidate risk edge.

[0100] The gap-mark ratio is defined as the ratio of the dominant ripple amplitude to the background amplitude reference. The gap-mark ratio is a positive real number and is dimensionless.

[0101] Example: The end point of the crane boom swings left and right between the two circuit conductors. The gap mark distance sequence shows a downward trend and is superimposed with periodic oscillation. The monotonic trend fitting gives a smooth downward curve. The residual sequence shows a single dominant frequency peak. The gap mark ratio enters the high value range and is mapped by the evidence as uncertain evidence strength.

[0102] S303 beam eccentricity ratio calculation.

[0103] In a multi-loop scenario on the same tower, when the loop affiliation is stable, the difference in the gap marker distance between the optimal loop and the second-best loop remains in the same sign and the difference magnitude is stable. The combination of robust interval and sign reversal statistics is used to characterize the separability and stability of the loop.

[0104] The optimal loop index and the suboptimal loop index are determined by robust aggregation within the sliding time window. For each loop, robust aggregation takes the median gap distance of the slot marker within the set of valid frames. The optimal loop index is the loop index with the smallest robust aggregation value, and the suboptimal loop index is the loop index with the smallest robust aggregation value after removing the optimal loop.

[0105] The difference sequence is constructed on the set of frame indices where both the optimal and suboptimal loops are effective. The difference sequence is constructed frame by frame by subtracting the gap mark distance of the optimal loop from the gap mark distance of the suboptimal loop. The difference sequence is robustly smoothed by using a sliding median filter. The filter window length is a preset odd number and covers the symmetrical frame indices before and after the current frame.

[0106] The sign-flipping count is performed on the smoothed difference sequence, traversing two adjacent frames. When the signs of the difference values ​​in the two frames are opposite, the count is incremented by one. The robust interval is the median of the absolute values ​​of the smoothed difference sequence. The flipping penalty function consists of one plus the natural logarithm of the flipping count incremented by one. The bundle-partial stability ratio is defined as the ratio formed by the robust interval and the flipping penalty function. The bundle-partial stability ratio is a non-negative real number and is dimensionless.

[0107] Example: The end point of the crane boom continuously approaches along the first loop direction, the optimal loop index remains consistent within the sliding time window, the difference sequence remains positive for a long time and the flip count is zero, the bundle bias stability ratio enters the high value range and is mapped by the evidence to lock the evidence strength.

[0108] S304 Evidence Mapping, Evidence Fusion and Trust Coefficient Generation.

[0109] In a multi-loop scenario on the same tower, the gap mark ripple ratio reflects the alternating ripple intensity of the loop, the beam bias stability ratio reflects the separability stability of the loop, and evidence fusion is used to unify the two types of evidence to the lock-in coefficient while retaining the uncertain state.

[0110] The gap mark ratio mapping value is defined as the ratio of the gap mark ratio to the gap mark ratio plus one. The mapping value ranges from zero to less than one, and increases monotonically as the gap mark ratio increases. The beam offset stability ratio mapping value is defined as the ratio of the beam offset stability ratio to the beam offset stability ratio plus one. The mapping value ranges from zero to less than one, and increases monotonically as the beam offset stability ratio increases.

[0111] Evidence quality allocation uses two sets of propositions: locked propositions and uncertain propositions. For gap-scale ratio evidence, the gap-scale ratio mapping value is assigned to the uncertain proposition and the remainder is assigned to the locked proposition. For bundle-partial stability ratio evidence, the bundle-partial stability ratio mapping value is assigned to the locked proposition and the remainder is assigned to the uncertain proposition.

[0112] Evidence fusion adopts Depster's synthesis rule. The conflict coefficient is formed by the sum of the products of the two pieces of evidence on opposite propositions. The synthesis quality is formed by dividing the product of the same proposition by one and subtracting the conflict coefficient. When the conflict coefficient reaches the corresponding preset upper bound, the lock-in confidence coefficient is set to zero and all synthesis quality is allocated to the uncertain proposition. When the conflict coefficient is lower than the preset upper bound, the lock-in confidence coefficient takes the quality of the locked proposition in the synthesis quality. The lock-in confidence coefficient ranges from zero to one and is dimensionless.

[0113] The gating front weight is obtained by mapping the gap marker net distance. The mapping rule is that it is the reciprocal of the sum of the gap marker net distance and the gating front weight. The value range of the gating front weight is greater than zero and does not exceed one. The gating back weight is obtained by multiplying the locking signal coefficient and the gating front weight. The value range of the gating back weight is zero to one and is dimensionless.

[0114] S305 Loop Mutual Exclusion Normalized Attention Reasoning and Loop Attribution Locking Criterion.

[0115] In a multi-loop scenario on the same tower, key points of the same robotic arm need to form mutually exclusive choices among multiple loop wire clusters. Mutually exclusive normalized attention transforms parallel candidate risk edges into loop attention distributions, and loop attribution locking criteria bind attention convergence and locking confidence coefficients to the locking state.

[0116] The spatiotemporal graph convolutional network takes a heterogeneous spatiotemporal graph and a gated back edge weight sequence as input. The message passing consists of three paths: the first path passes component topology information along the component structure edges between the root node of the robotic arm, the key node corresponding to the joint of the robotic arm, and the end node of the robotic arm; the second path passes gated back edge weights and gap marker distance feature information along the candidate risk edges between the key node of the robotic arm and the loop conductor cluster node; and the third path passes occlusion marker and scale change marker information along the cross-frame time edges between nodes with the same name in adjacent frames. Each frame outputs the hidden state of the key node of the robotic arm and the hidden state of the loop conductor cluster node.

[0117] The loop mutual exclusion normalized attention calculates the attention score between the key point node of the same robot arm and all loop wire cluster nodes. The attention score is generated by the similarity function between the hidden state of the key point node and the hidden state of the loop wire cluster node. The similarity function adopts the vector inner product or cosine similarity. Then, normalization is performed on the loop dimension of the key point node of the same robot arm so that the loop attention distribution is in the range of zero to one and the loop attention distribution is summed to one.

[0118] The locked loop index is taken as the loop index corresponding to the maximum value of the loop attention distribution, and the attention convergence interval is taken as the difference between the maximum attention and the second largest attention. The loop belonging to the locked state is generated by three parallel conditions. The first condition is that the attention convergence interval continuously meets the persistence determination within the sliding time window. The second condition is that the locking confidence coefficient corresponding to the locked loop index continuously meets the persistence determination within the sliding time window. The third condition is that the frame count with occlusion mark as one and the frame count with scale change mark as one within the sliding time window meet the preset upper limit constraint. The persistence determination adopts the continuous frame count method or the proportion method and is fixed by the preset rules.

[0119] Example: When the end point of the crane boom enters the overlapping area of ​​the dual-loop projection on the same tower, the loop attention distribution alternates between the two loops. The attention convergence interval does not meet the continuous judgment for a long time, and the loop ownership lock state remains unlocked. When the end point of the boom leaves the overlapping area and gets close to one of the loops, the loop attention distribution concentrates on a single loop and meets the continuous judgment. The loop ownership lock state switches to locked and the locked loop index is output.

[0120] For each candidate risk edge, the gap scale ratio and bundle bias stability ratio are calculated within a sliding time window, and evidence mapping and evidence fusion are performed to obtain the locking confidence coefficient. The locking confidence coefficient is used to perform gating weighting on the edge weights of the candidate risk edges. The gated heterogeneous spatiotemporal graph is input into a spatiotemporal graph convolutional network with a loop mutual exclusion normalized attention mechanism for inference and outputs the loop attention distribution. Based on the continuous satisfaction of the locking confidence coefficient and the convergence judgment condition of the loop attention distribution, the loop belongs to the locked state or the unlocked state.

[0121] When the loop attribution lock state is established, the risk object has been located to a specific loop. The anomaly approach judgment needs to constrain the temporal consistency of the gating candidate risk edge weight sequence to eliminate false risks introduced by occlusion segments and scale change segments, and compress the judgment process into a verifiable evidence package field. When the loop attribution lock state is not established, the uncertainty of multiple loops still exists. The output needs to retain the loop risk distribution and trigger the verification entry to ensure the consistency of the handling link.

[0122] S401 Input Object Alignment and Locked Branch / Unlocked Branch Division.

[0123] Within the multi-loop section of the same tower, the construction machinery trajectory number and key point type are consistent from step S1 to step S3. In step S4, the same index key is used to align the gating weight sequence, loop attention distribution, locking coefficient, occlusion marker, and scale change marker to the same key point event to avoid multiple contradictory judgment outputs for the same event.

[0124] Within the sliding time window, the loop's locked status, locked loop index, loop attention distribution, locking confidence coefficient, and gating back weight sequence are read. Within the sliding time window, occlusion markers and scale change markers are read and aligned to the same frame index. When the loop's locked status is locked, the locked branch is entered and the locked loop index is fixed. When the loop's locked status is unlocked, the unlocked branch is entered and the full set of loop indices is retained. The locked and unlocked branches share the same output field name. The locked branch fills the locked loop index, while the unlocked branch sets the locked loop index field to null.

[0125] S402 locks the branch gate, then cleans the edge weight sequence and generates the effective segment set.

[0126] When there is occlusion and zoom in the overlapping area of ​​the projection of multi-circuit conductors on the same tower, the weight sequence after gating will show breaks and spikes. The cleaning rule needs to remove unobservable frames and divide observable frames into continuous segments. The continuous segments are used to construct persistence indicators and evidence summaries.

[0127] After fixing the locking loop index of the locked branch, extract the corresponding gated weight sequence of the locking loop and simultaneously extract the occlusion marker and scale change marker; the effective frame index filtering is performed in a fixed order. The first step is to retain the frame index with the occlusion marker of zero. The second step is to remove the frame index with the scale change marker of one from the retained frame index and use the frame index with the scale change marker of one as the segment boundary. The third step is to retain the frame index with the locking signal coefficient satisfying the preset lower limit constraint in each segment. The effective segment set is obtained by dividing the effective frame index according to the time continuity. Each segment records the start frame index and the end frame index. The segment-level representative measure is the median of the gated weights in the segment. The segment-level representative measure ranges from zero to one.

[0128] S403 Lock Branch Abnormal Approach Judgment and Decision Output.

[0129] The approximation of multi-circuit construction machinery on the same tower needs to reflect both the approximation intensity and continuity. High values ​​in a single frame are easily triggered by jitter at key points. Segment-level statistics and orientation consistency constraints can separate jitter spikes from the actual approximation.

[0130] The approximation strength index is calculated on the valid frame index. The calculation rule is fixed: sort the gating weights within the valid frame by value and take the gating weight corresponding to the preset quantile position as the approximation strength index. The preset quantile position is fixed by the configuration file and falls in the upper half interval. The approximation strength index value ranges from zero to one. The persistence index is calculated on the valid segment set. The calculation rule is fixed: count the lengths of consecutive segments that satisfy the corresponding preset threshold conditions at the segment level and take the longest consecutive segment length as the persistence index. The persistence index value is a non-negative integer. The azimuth consistency index is calculated using step S2. The relative azimuth information sequence is calculated using a fixed rule: the proportion of frames whose relative azimuth information falls into the same main sector on the valid frame index. The main sector is determined by the median direction of the relative azimuth information, and the sector width is fixed by a preset angle width rule. The azimuth consistency index ranges from zero to one. The abnormal approach judgment result adopts parallel criteria. When the approach intensity index meets the corresponding preset threshold condition, the persistence index meets the corresponding preset threshold condition, and the azimuth consistency index meets the corresponding preset threshold condition, the abnormal approach judgment result is output as one. Otherwise, the abnormal approach judgment result is output as zero.

[0131] Example: As the crane passes through the passage under the tower, the end point of the boom moves continuously toward the side of the locking loop conductor. The relative orientation information falls into the same sector for a long time. The weight after the gate control remains high in multiple consecutive effective segments. The abnormal approach judgment result is output and enters the evidence package generation process together.

[0132] S404 Evidence Package Field Generation and Keyframe Index Extraction.

[0133] On-site handling of multiple loops on the same tower requires locating and locking the loops and the approximation process. The evidence package fields need to cover the locked loop index, key point trajectory summary, gating back weight evolution summary, and key frame index set. The field generation rules need to be deterministic and robust to avoid repeatedly replaying the full-window video during the review stage.

[0134] The evidence package outputs a fixed lock loop index, a keyframe index set, a keypoint trajectory summary, and a gated back weight evolution summary. The keyframe index set extraction is performed in a fixed order: first, the frame index with the maximum gated back weight is found in the valid frame indices and added to the keyframe index set; second, the direction of change of the gated back weight in adjacent frames is calculated in the valid frame indices, and turning frame indices are filtered. Turning frame indices are determined by simultaneously satisfying a reversal of the change direction and a preset lower limit constraint on the change magnitude; third, the start and end frame indices of each valid segment are added to the keyframe index set. The keypoint trajectory summary uses the keypoint pixel coordinate sequence output in step S1. The trajectory summary generation adopts a Douglas-Pock polyline simplification strategy. Polyline simplification iteratively deletes intermediate points and retains key turning points according to fixed rules, with the tolerance fixed by a preset pixel tolerance rule. The gated back weight evolution summary outputs a segment-level summary field based on the valid segment set. The segment-level summary field contains a segment-level representative quantity and a segment-internal change direction marker. The segment-internal change direction marker is determined by comparing the gated back weight of the segment's start frame and the gated back weight of the segment's end frame.

[0135] Example: The end point of the boom first approaches the guide wire, pauses briefly, and then moves away. After gating, the weight first rises, then the platform descends. The key frame index set includes the maximum value frame index and the two turning frame indexes. The trajectory summary retains the stopping point and turning point and corresponds to the key frame index.

[0136] S405 Unlocked Branch Risk Distribution Output and Cloud Review Request Generation.

[0137] Within the overlapping area of ​​multi-loop conductor projections on the same tower, the loop attention distribution is prone to being multi-peaked and failing to converge for a long time. Unlocked branches need to output the loop risk distribution and trigger a review request. The field structure of the review request is consistent with the field structure of the evidence package, and the review entry point remains unified.

[0138] The unlocked branch reads the loop attention distribution and outputs the loop risk distribution vector. The loop risk distribution vector is arranged in loop index order, and each component has a value ranging from zero to one and sums to one. The unlocked review trigger flag generation uses two sets of conditions for parallel judgment. The first set of conditions is the unlocked high-risk condition, and the judgment order is fixed as follows: first, detect the frame index that meets the preset peak condition in the gating back weight sequence within the sliding time window, and then detect the loop attention distribution having two or more loop components at the same frame index that simultaneously meet the preset proportion condition. The second set of conditions is the uncertain persistence condition, and the judgment order is fixed as follows: first, calculate the attention impurity index, and then detect the attention impurity index. Within the sliding time window, the label satisfies the continuity judgment, and the count of the scale mutation markers meets the preset upper limit constraint. The calculation rule for the attention impurity index is fixed as follows: square the attention components of each loop, sum them, and then subtract the summation result from one to obtain the impurity value. The impurity value ranges from zero to one. When any set of conditions is met, the output of the unlocked review trigger marker is one, and the output of the rest is zero. The cloud review request carries the evidence package field structure and sets the locked loop index field to empty. It also carries the loop risk distribution vector. The key frame index set and key point trajectory summary are generated according to the aforementioned rules. The gated weight evolution summary is generated according to the loop index and the loop index is attached to each summary segment.

[0139] When the loop ownership lock state is established, the gating candidate risk edge weight sequence corresponding to the locked loop is selected as the judgment input. Combining the occlusion mark and scale change mark carried by the cross-frame time edge and the time consistency constraint of the gating edge weight, the anomaly approximation judgment is completed and an evidence package is generated. The evidence package includes the locked loop identifier, the robot arm key point trajectory summary, the time evolution summary of the key candidate risk edge weight, and the key frame index. When the loop ownership lock state is not established, the loop risk distribution and non-convergence mark are output and a cloud review request is triggered. The cloud review request carries the same data field structure as the evidence package.

[0140] Specifically, the above are merely preferred embodiments of this application and are not intended to limit this application.

[0141] The thresholds and parameters preset in this invention can be pre-calibrated through offline simulation testing, or set to fixed values ​​according to on-site operating procedures.

[0142] In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0143] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A real-time detection method for abnormal behavior in power transmission corridors based on spatiotemporal graph convolutional networks, characterized in that, Including the following steps: S1: Process the multi-loop video stream on the same tower frame by frame, and extract the target area of ​​the construction machinery and the key point pixel position of the robotic arm; cluster the set of conductor anchor points according to the loop to obtain the loop conductor cluster node; S2: Calculate the shortest pixel distance from the key point of the robotic arm to the node of the loop wire cluster within the sliding time window, obtain the gap scale according to the anchor point spacing, and normalize it to the gap scale net distance. Based on the sliding time window, construct a heterogeneous spatiotemporal graph containing candidate risk edges and cross-frame time edges, and add occlusion markers and scale change markers to cross-frame time edges. S3: Extract edge evidence of loop pointing to fluctuation and edge evidence of loop belonging stability for candidate risk edges within the sliding time window. After fusion, obtain the locking coefficient. Based on this, perform gating weighting on the edge weights of candidate risk edges. Input the gated heterogeneous spatiotemporal graph into a spatiotemporal graph convolutional network with a loop mutual exclusion normalization attention mechanism to output the loop belonging locking state. S4: When the loop ownership lock state is established, select the gating candidate risk edge weight sequence corresponding to the locked loop, and combine the occlusion mark and scale change mark to determine the abnormal approximation and generate evidence package. When the loop ownership lock state is not established, output the loop risk distribution and non-convergence mark, and trigger the cloud review request.

2. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 1, characterized in that, Step S1 includes: The video stream of the multi-loop section of the same tower is processed frame by frame. The construction machinery target detection model outputs the set of construction machinery target areas and outputs category confidence information. The component key point extraction model outputs the root point, joint point and end point of the robotic arm to form the set of key points of the robotic arm within the set of construction machinery target areas, and outputs the key point observation confidence mark and key point visibility mark.

3. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 2, characterized in that, Step S1 also includes: The candidate region segmentation of the conductor yields the conductor probability map. Skeletonization and straight line fitting yield the conductor segment set. Structural point detection yields the tower head suspension point set and the insulator string connection point set. The endpoints of the conductor segment set and the structural points are merged to form the conductor anchor point set, and anchor point observation confidence markers and anchor point visibility markers are output. The conductor anchor point set is clustered according to directional consistency, strip position identifier, and tower head suspension point constraint to generate a loop conductor cluster node set.

4. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 3, characterized in that, Step S2 includes: Within a sliding time window, input the target region set of construction machinery for each frame, and use Kalman prediction and Hungarian matching to generate the trajectory number of the construction machinery. The set of key points of the robotic arm is indexed according to the trajectory number of the construction machinery and the key point type. The list of anchor points within each loop conductor cluster node in the loop conductor cluster node set is generated by pairing adjacent loops to generate a gap scale, and the visibility summary of loop anchor points is calculated.

5. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 4, characterized in that, Step S2 also includes: The shortest pixel distance sequence is calculated based on the set of key points of the robotic arm and the set of nodes of the loop conductor cluster, and normalized to the gap marker net distance sequence. At the same time, relative orientation information and anchor point visibility information are generated. A heterogeneous spatiotemporal graph is constructed based on the nodes of the construction machinery, the key point nodes of the robotic arm, the loop conductor cluster nodes, and the anchor point nodes of the conductor. Candidate risk edges are kept in parallel, and the set of candidate risk edge sequences and the set of cross-frame time edge sequences are output. The cross-frame time edges carry occlusion markers and scale change markers.

6. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 5, characterized in that, Step S3 includes: Within the sliding time window, the candidate risk edge sequence set and the cross-frame time edge sequence set are read. The frame index is filtered based on the occlusion marker, scale change marker, anchor point visibility information, and loop anchor point visibility, and a valid frame set is formed. The frame index with the scale change marker of one is used as the segment boundary, and the sequence is renumbered for each segment.

7. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 6, characterized in that, Step S3 also includes: Monotonic regression is performed on the gap mark net distance sequence corresponding to the effective frame set to obtain the trend sequence and generate the residual sequence. Frequency domain component analysis is performed on the residual sequence to extract the dominant ripple amplitude and form the gap mark ripple ratio with the background amplitude benchmark. The optimal loop index and the suboptimal loop index are determined based on robust aggregation. The difference sequence is subjected to sliding median filtering, and the symbol flip count and robust interval are statistically analyzed to form the bundle bias stability ratio.

8. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 7, characterized in that, Step S3 also includes: Evidence mapping is performed based on the gap-mark ratio and the beam-bias-stability ratio, and the locking-confidence coefficient is generated by fusing the Depster synthesis rule. The locking-confidence coefficient is used to gate the edge weights of candidate risk edges and generate the gated edge weight sequence. The gated edge weight sequence and the attention distribution of the spatiotemporal graph convolutional network output loop of the heterogeneous spatiotemporal graph input loop are mutually exclusive. The loop-attribution locking state and the locking loop index are generated based on the attention convergence interval, the persistence determination of the locking-confidence coefficient, the occlusion marker count, and the scale change marker count.

9. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 8, characterized in that, Step S4 includes: Based on the loop ownership and locking status, the system divides the system into locked and unlocked branches. The locked branch has a fixed locked loop index and extracts the gating back weight sequence. It combines occlusion markers and scale change markers to filter valid frame indices and form a set of valid segments. Based on the gating back weight sequence and relative orientation information, it calculates the approach intensity index, persistence index, and orientation consistency index and outputs the abnormal approach judgment result.

10. The real-time detection method for abnormal behavior of power transmission corridors based on spatiotemporal graph convolutional networks according to claim 9, characterized in that, Step S4 also includes: The locked branch extracts the keyframe index set within the valid frame index and generates an evidence package. The evidence package includes the locked loop index, the keyframe index set, the key point trajectory summary, and the gated back weight evolution summary. The key point trajectory summary is generated using a polyline simplification strategy. The unlocked branch outputs the loop risk distribution vector and the unlocked review trigger mark, and generates a cloud review request. The cloud review request uses the same field structure as the evidence package, but the locked loop index field is set to empty.

Images