A method for power grid construction site safety analysis using intelligent video technology

Abstract

The application provides a method for safety analysis of power grid construction site by using intelligent video technology, which extracts personnel skeleton key point coordinate sequence frame by frame through deep learning skeleton point detection on the continuous video stream of the site, and then identifies each atomic operation action and accurately locates its start and end frame, so as to decompose the continuous video into a step sequence with timestamp; then the sequence is matched with the standard operation step set and its timing logic chain defined in the regulation knowledge base in advance, and the abnormal action of type inconsistency and the order violation of violating the sequence dependence relationship are automatically found by using the checking mechanism based on directed graph or timing logic; the abnormal points are further matched with the violation mode library to distinguish the known typical violation from the unknown new type of deviation, and finally it is determined whether the operation meets the safety requirements, realizing the safety compliance analysis of the whole process of the complex operation of the power grid site, and greatly improving the timeliness and accuracy of the violation discovery.

Description

Technical Field

[0001] This invention relates to a method for conducting on-site safety analysis of power grid construction using intelligent video technology. Background Technology

[0002] In the field of power grid construction and operation and maintenance, on-site safety management is the cornerstone of ensuring the stable operation of the power system and the safety of personnel. With the introduction of intelligent technologies, safety supervision through video analysis and other means has become an important development direction, aiming to transform the traditional model relying on manual supervision into an automated and intelligent proactive early warning system. However, many current video-based monitoring methods often focus on identifying isolated static scenarios such as whether personnel are wearing safety helmets or entering dangerous areas. This analytical model has obvious limitations. It is difficult to cope with a series of continuous, dynamic, and interconnected operational behaviors during the operation process, and it cannot judge whether a complex operation has been executed correctly and completely as a whole. This greatly reduces the depth and effectiveness of safety supervision.

[0003] The core technical challenge in automating the dynamics and compliance of operational processes lies in the "temporal logical relationship of action sequences." Power grid field operations consist of a series of steps strictly following the operation ticket, executed in a specific order. Each step corresponds to a specific equipment operation and state change, with a mandatory logical and temporal sequence between each step. Simply identifying a final action state is insufficient; the key is to determine whether the action occurred at the correct step node and whether its preconditions were met. This requires analyzing and verifying the temporal logical chain of the entire operational process.

[0004] The attribute of "the temporal logical relationship of action sequences" leads to another closely related technical factor: "identifying subtle differences between compliant actions and non-compliant patterns." In actual operations, compliant and non-compliant operations may differ only slightly in video presentation, such as a reversed order of operations, the omission of a key confirmation step, or the inclusion of a simplified action not permitted by procedures. These non-compliant patterns are often hidden within seemingly normal operational flows, and their identification requires not only comparison with preset steps but also a deep understanding of the specific patterns of non-compliant actions and their deviations from standard procedures. How to accurately decompose and match operational steps from continuous video streams, and on this basis, keenly identify various potential non-compliant patterns, constitutes a key obstacle to achieving intelligent security analysis.

[0005] Therefore, in power grid construction site safety monitoring, the ability to automatically and accurately verify whether a multi-step operation fully complies with the operation ticket procedures based on video streams, and to promptly detect risks such as disordered operation sequence, missing steps, or additional violations, has become a key issue in improving the level of proactive on-site safety management and preventing accidents caused by human error. Summary of the Invention

[0006] This invention proposes a method for safety analysis at power grid construction sites using intelligent video technology. It automatically determines in real time whether the operation procedures of workers at power grid sites strictly comply with safety regulations in terms of continuity, completeness, and correctness of sequence. This achieves intelligent, real-time, and objective safety compliance analysis of the entire process of complex operations at power grid sites, significantly improving the timeliness and accuracy of violation detection and effectively reducing blind spots in human supervision and the risk of missed detection.

[0007] In one aspect, this invention provides a method for conducting on-site safety analysis of power grid construction using intelligent video technology, comprising the following steps: S1. Acquire continuous video stream data of power grid on-site operations. For this video stream, use an action recognition algorithm based on deep learning skeleton point detection to extract the coordinate sequence of the skeleton key points of the workers frame by frame. S2, based on the preset set of operation steps, combined with the motion trajectory and spatial position relationship of the skeleton key point coordinate sequence, identifies each discrete atomic operation action in the video, and determines the start frame and end frame of each atomic operation action, thereby decomposing the continuous video stream into a sequence of step decomposition points with timestamps. S3 reads the set of standard operating steps corresponding to the current task and its prescribed timing logic chain from the preset procedure knowledge base. The timing logic chain clarifies the dependencies between each operating step and the allowed execution time window. S4. Input the sequence of decomposition points obtained from the video stream decomposition and the standard temporal logic chain into the logic checker. The logic checker uses a matching algorithm based on directed graph or temporal logic to compare the type and order of each action in the decomposition point sequence with the node and edge relationships defined in the logic chain. S5. If the logic checker finds that there is an action type that does not match the logic chain node in the step decomposition point sequence, it marks the action as a type anomaly point. If it finds that the order between actions violates the edge constraints of the logic chain, it marks the order relationship as an order violation point. S6, for the set of type anomalies and sequence violations output by the logic verifier, the deviation detector calls the preset violation pattern library to perform pattern matching, and analyzes whether the anomaly or violation belongs to a known typical violation operation pattern or an unknown new deviation. S7. Based on the overall matching result of the logic checker and the output of the deviation detector, calculate the consistency score between the current operation process and the standard sequential logic chain, and determine whether the process integrity of the current operation meets the safety procedure requirements based on the preset consistency threshold.

[0008] Preferably, step S1 includes:

[0009] Continuous video stream data collected by cameras in the power grid field operation area is acquired, and the video stream is input frame by frame into a skeleton point detection network to obtain a two-dimensional coordinate set of the skeletal key points of the worker in each frame; the two-dimensional coordinate sets of the skeletal key points of each frame are connected sequentially in time order to form a continuous sequence of skeletal key point coordinates of the worker reflecting the relative position of the joints and the movement trajectory.

[0010] The displacement distance and angle change of each joint between adjacent frames are calculated based on the coordinate sequence of key points of the skeleton, so as to obtain the limb movement speed characteristics and posture rotation characteristics of the operator, and obtain a complete time-series data record that can be used for subsequent power grid on-site operation safety status determination.

[0011] Preferably, step S2 specifically includes:

[0012] The coordinates of the skeletal key points in each frame of the video sequence are obtained. The smoothed trajectory sequence is obtained through filtering and used for subsequent motion feature extraction. The Euclidean distance and relative displacement vector between key points in adjacent frames are calculated to obtain the feature sequence representing the spatial relationship between key points.

[0013] For the feature sequence of the spatial correlation, calculate the consistency of displacement amplitude and direction of key points in consecutive frames, determine the intensity change position of the motion start and end, and obtain preliminary motion boundary candidate points; if there is an interval in the frame sequence between adjacent candidate points where the motion amplitude is continuously lower than the preset change range, then merge the candidate points at both ends of the interval to obtain the adjusted motion interval sequence.

[0014] By matching and verifying the adjusted action interval sequence with the spatial association features of key points, the start frame number and end frame number of each atomic operation action are determined, and the step decomposition point sequence with frame timestamps is output.

[0015] Preferably, step S3 specifically includes:

[0016] Obtain the set of standard operation steps corresponding to the current task from the preset procedure knowledge base, and at the same time read the temporal logic chain associated with the set of standard operation steps for link parsing to determine the sequential dependencies between each operation step, thereby obtaining the dependency constraint set and the sequential sequence.

[0017] Calculate the execution time window for each operation step to obtain the time boundary range of each step. If the time boundary ranges overlap or misalign, it is determined that the time window constraint matching fails and the execution of the current link is blocked; if the time boundary ranges meet the constraints, the step triggering order is determined.

[0018] By triggering the sequential and timing logic chain through the steps described above, the state transition of the standard operation step set is completed, resulting in an executable procedure flow sequence.

[0019] Preferably, step S4 specifically includes:

[0020] The video stream is acquired, and action decomposition points are extracted through keyframe recognition and pose estimation to form the decomposition point sequence. A standard temporal logic chain is pre-established, and a directed graph is constructed to represent the node and edge relationships.

[0021] The decomposed point sequence is input into a logic verifier, and a topological sorting algorithm is used to compare the node order in the graph with the action type and occurrence order in the sequence. If the matching result shows a difference, abnormal paths are identified, and inconsistent nodes and edges are extracted from the directed graph.

[0022] A verification report is generated based on the abnormal path to determine the consistency between the overall sequence and the standard timing logic chain.

[0023] Preferably, step S5 specifically includes:

[0024] For each action in the step decomposition point sequence, determine whether its action type matches the node type at the corresponding position in the logical chain. If they do not match, mark the action as a type anomaly point. Based on the directed edge constraints of the logical chain, obtain the sequential relationship between adjacent actions in turn, and determine whether the sequential relationship violates the directed edge constraints. If it does, mark the adjacent sequential relationship as a sequence violation point.

[0025] By summarizing all types of anomalies and sequential violations, a complete anomaly labeling result for the step decomposition point sequence is obtained.

[0026] Preferably, step S6 specifically includes:

[0027] The temporal correlation strength between each violation point is calculated by the set of anomalies and the sequence of violation points. The contextual correlation vector of the violation sequence is obtained and matched with the preset pattern feature library to determine whether there is a complete hit or partial similarity of a known typical violation template.

[0028] If a match is found, the deviation type label is output as "known violation"; if no known typical violation template is found, the features corresponding to the unmatched part are extracted from the context association vector, a new deviation label is generated and recorded in the historical deviation record.

[0029] The new deviation identifier is compared with existing records in the historical deviation record. If the comparison confirms that it is new content, the new deviation feature is added to the preset pattern feature library for subsequent real-time deviation pattern matching judgment.

[0030] Preferably, step S7 specifically includes:

[0031] The overall matching degree result output by the logic checker and the deviation quantization value output by the deviation detector are obtained. The overall matching degree result represents the degree of conformity between the current operation process and the standard timing logic chain, and the deviation quantization value represents the cumulative degree of deviation of each link from the standard timing.

[0032] The overall matching result and the deviation quantification value are weighted and fused using a pre-set weighting coefficient to obtain the consistency score;

[0033] The consistency score is compared with a preset consistency threshold. If the consistency score is greater than or equal to the preset consistency threshold, the current work process is determined to meet the safety procedure requirements. If the consistency score is less than the preset consistency threshold, the current work process is determined to not meet the safety procedure requirements.

[0034] The output judgment result serves as the conclusion of the process integrity and compliance for this operation.

[0035] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described thereon.

[0036] In another aspect, the present invention also provides a computer program product, comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method described herein.

[0037] The beneficial effects of this invention are as follows: By performing deep learning skeleton point detection on continuous video streams from the site, the coordinate sequence of key points of personnel skeletons is extracted frame by frame, thereby identifying each atomic operation action and accurately locating its start and end frames, thus decomposing the continuous video into a sequence of steps with timestamps. This sequence is then matched with a predefined set of standard operation steps and their temporal logic chains in a procedure knowledge base. Using a verification mechanism based on directed graphs or temporal logic, abnormal actions of inconsistent types and sequential violations that violate dependency relationships are automatically detected. For abnormal points, pattern matching is further performed through a violation pattern library to distinguish between known typical violations and unknown novel deviations. Finally, a consistency score between the work process and standard procedures is calculated, and a threshold is used to determine whether the operation meets safety requirements. This invention achieves intelligent, real-time, and objective safety compliance analysis of the entire process of complex operations on power grid sites, significantly improving the timeliness and accuracy of violation detection, effectively reducing blind spots and missed detection risks in human supervision, and providing strong technical support for safety monitoring at power grid construction sites. Detailed Implementation

[0038] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0039] In one aspect, this invention provides a method for conducting on-site safety analysis of power grid construction using intelligent video technology, comprising the following steps:

[0040] S1. Acquire continuous video stream data of power grid on-site operations. For this video stream, use an action recognition algorithm based on deep learning skeleton point detection to extract the coordinate sequence of the skeletal key points of the workers frame by frame.

[0041] Continuous video stream data collected by cameras in the power grid field operation area is acquired. The video stream is then input frame by frame into a skeleton point detection network to obtain a two-dimensional coordinate set of the skeletal key points of the worker in each frame. The two-dimensional coordinate sets of the skeletal key points of each frame are connected sequentially in time to form a continuous sequence of skeletal key point coordinates of the worker, reflecting the relative position of the joints and the movement trajectory. The displacement distance and angle change of each joint between adjacent frames are calculated based on the skeletal key point coordinate sequence to obtain the limb movement speed characteristics and posture rotation characteristics of the worker, resulting in a complete time-series data record that can be used for subsequent safety status determination of power grid field operations.

[0042] Action recognition algorithms based on deep learning skeleton point detection first perform human detection on each received image frame, and then perform skeleton point localization. A common implementation method is to use a bottom-up multi-human pose estimation algorithm. This algorithm first uses a convolutional neural network to extract feature maps of the entire image, and then uses confidence heatmaps and vector fields to predict the existence probability of each key point and the association direction between key points, thereby constructing a complete skeleton connection relationship. In power grid field operations, workers are often in non-standard postures, such as holding a voltage detector high with one hand, pulling wires taut with both arms, and leaning forward significantly to observe hardware. These postures differ significantly from standing and walking postures in daily behavior datasets. Therefore, it is necessary to construct a dedicated labeled dataset for the characteristics of power grid operations. This dataset contains video clips of various typical working conditions such as pole climbing, crossing, tension pole wire hanging, equipotential bonding operations, and live water washing. Each frame is labeled with 17 or more human key point coordinates, including key points of the head, neck, shoulders, elbows, wrists, hips, knees, ankles, and hands. After non-maximum suppression, the detected human bounding boxes retain only the boxes with the highest confidence. Skeleton point estimation is then performed on the regions within each box to obtain the independent skeletons of multiple individuals. Even with multiple workers present, the keypoint sequence of the target object can be distinguished. The extracted skeletal keypoint coordinate sequence is organized in two or three dimensions, with each frame corresponding to a fixed-length vector containing the x-coordinate, y-coordinate, and confidence value of all keypoints. The coordinate system has its origin at the top left corner of the image, with the x-coordinate increasing to the right and the y-coordinate increasing downwards. To ensure the spatiotemporal continuity of the sequence, when the confidence of a keypoint in a frame falls below a threshold, linear interpolation or Kalman filtering based on preceding and following frames can be used for smoothing and filling.

[0043] The system segments the continuous video stream, with each segment comprising a 5-second analysis unit. It performs a comprehensive analysis of the skeletal keypoint sequence within that time period, enabling real-time identification of typical violations. When the angle between the line connecting the knee and hip keypoints of a worker and the horizontal plane is consistently less than a certain angle and the duration exceeds a preset threshold, it indicates a possible low squatting or kneeling posture. This is then combined with the height of the hand keypoints to determine if it constitutes a dangerous low posture for live-line work, triggering an alarm. Through this method, the system can reliably extract the skeletal keypoint coordinate sequence of workers in the power grid field environment, providing reliable basic data support for subsequent safety risk identification based on action patterns.

[0044] S2, based on the preset set of operation steps, combined with the motion trajectory and spatial position relationship of the skeleton key point coordinate sequence, identifies each discrete atomic operation action in the video, and determines the start frame and end frame of each atomic operation action, thereby decomposing the continuous video stream into a sequence of time-stamped step decomposition points.

[0045] The coordinates of skeletal keypoints in each frame of the video sequence are obtained. A smoothed trajectory sequence for subsequent motion feature extraction is obtained through filtering. The Euclidean distance and relative displacement vector between keypoints in adjacent frames are calculated to obtain a feature sequence representing the spatial relationship between keypoints. For the feature sequence representing the spatial relationship, the consistency of the displacement amplitude and direction of keypoints in consecutive frames is calculated to determine the intensity change position of the motion start and end, obtaining preliminary motion boundary candidate points. If there is an interval in the frame sequence between adjacent candidate points where the motion amplitude is continuously lower than a preset range, the candidate points at both ends of this interval are merged to obtain an adjusted motion interval sequence. The adjusted motion interval sequence is matched and verified with the spatial relationship features of the keypoints to determine the start and end frame numbers of each atomic operation action, and a step decomposition point sequence with frame timestamps is output.

[0046] The representation of motion trajectory is mainly described by the temporal displacement changes of key points. Taking hand operation as an example, the relative motion vector sequence of the fingertip key point relative to the wrist key point can be calculated to form a local motion trajectory; at the same time, the absolute displacement trajectory of the wrist key point in the global coordinate system can also be calculated. Spatial positional relationships mainly focus on the relative geometric constraints between multiple key points. Common skeletal topologies can be selected to calculate features such as the Euclidean distance, included angle, and area changes of the triangles formed by the key points between adjacent joints. For example, during the screw-tightening process, when the screwdriver tip key point, the wrist key point, and the center point of the screw hole gradually approach a collinear state, it indicates that the tool has been aligned with the target position. This change in spatial collinearity can serve as an important segmentation clue.

[0047] To simultaneously utilize motion trajectory and spatial position relationships, a composite feature descriptor can be constructed. Within each time window, trajectory-related features such as velocity, acceleration, and curvature, as well as spatial relationship-related indicators such as distance change rate, angle change rate, and volume bounding box changes between key points, are extracted simultaneously. These two types of features are then concatenated or weighted and fused to form a comprehensive state representation vector for the current time point. For example, in a video of disassembling electronic components, the trajectory feature of tweezers briefly pausing after gripping a pin and then rapidly pulling it out, accompanied by the spatial feature of a rapid increase in the distance between the pin and the pad and a significant change in the gripping angle, typically corresponds to the beginning of the atomic action of "pulling out the pin." The identification of atomic operations can be achieved using a hybrid approach combining rule-based threshold judgment and temporal model-based methods. The rule-based judgment part determines the operation based on predefined criteria, such as when the trajectory curvature exceeds a certain threshold and the rate of change in distance between key points is higher than a set value, classifying it as a "twisting" or "pulling" action. The temporal model part can employ a bidirectional long short-term memory network or a temporal convolutional network to annotate the fused feature sequence frame by frame, outputting the probability distribution of which atomic operation the current frame belongs to. The outputs from both methods can be fused using weighted voting or posterior probability to obtain the final action category label.

[0048] Determining the start and end frames is one of the most critical steps in the entire method. First, the preliminary action range is obtained using the action classification results mentioned above. Then, a local search window is set near the boundary of each preliminary range, for example, extending 15 frames before and after. Within this window, the abrupt change index of the relationship between the tangential angle change rate of the trajectory and the spatial position is recalculated, which can be used as the precise boundary for action switching.

[0049] When decomposing a continuous video stream into a sequence of timestamped steps, the start and end frame numbers of each atomic action can be directly recorded and converted into corresponding video timestamps. Furthermore, for ease of subsequent use, short pauses between adjacent actions can be individually marked as "transition" or "waiting" states. Targeted step decomposition can be achieved simply by defining different atomic action granularities and corresponding trajectory and spatial relationship judgment rules based on specific business requirements.

[0050] S3 reads the set of standard operating steps corresponding to the current task and its prescribed timing logic chain from the preset procedure knowledge base. The timing logic chain clarifies the dependencies between each operating step and the allowed execution time window.

[0051] The system retrieves a set of standard operation steps corresponding to the current task from a pre-defined procedure knowledge base. Simultaneously, it reads the temporal logic chain associated with the standard operation step set for link parsing, determining the dependencies between each operation step to obtain a set of dependency constraints and a sequence of possible orders. It calculates the execution time window for each operation step to obtain the time boundary range of each step. If the time boundary ranges overlap or misalign, it determines that the time window constraint matching has failed, and the current link execution is blocked. If the time boundary ranges satisfy the constraints, the step triggering order is determined. Through the step triggering order and the temporal logic chain, the system completes the state transition of the standard operation step set, obtaining an executable procedure flow sequence.

[0052] The pre-stored procedure knowledge base contains operation procedures corresponding to various job task types. Each procedure is organized in a structured form, including step numbers, operation content, execution conditions, and associated temporal constraints. The reading process first locates the corresponding procedure record based on the unique identifier of the job task, such as the line maintenance task number or equipment maintenance work order number; then, all standard operation steps are extracted to form an ordered set of steps. The set of standard operation steps contains several independent but interrelated operation units. For example, in live-line work on overhead transmission lines, the set of steps may include safety measure deployment, high-altitude entry, insulator inspection, conductor replacement, tool retrieval, etc., with each step accompanied by clear input conditions and output status. The knowledge base uses a relational or graph database for storage, allowing the dependencies between steps to be directly reflected through edge relationships. The temporal logic chain is used to clarify the sequential dependencies and allowed execution time windows between each operation step. The temporal logic chain is constructed in the form of a directed graph, where nodes represent individual operation steps and directed edges represent the execution order. The construction process of the time-series logic chain includes two stages: dependency resolution and time feasibility check. In the dependency resolution stage, the precondition fields in the procedure are traversed, and the triggering conditions of each step are mapped to references to the completion status of other steps. In the time feasibility check stage, the time window of each step is compared with the actual workable time period based on the current time and the preset calendar, and the subset of steps that can be executed at present is marked.

[0053] When the task changes, such as when a defect review step is added temporarily, the system can incrementally update the logic chain, insert the new step into the appropriate position and re-verify the dependencies and time constraints, thereby maintaining the correctness of the overall execution order. This can effectively avoid safety hazards caused by incorrect step order or timeout operations, while improving the standardization and traceability of on-site operations.

[0054] S4. Input the sequence of decomposition points obtained from the video stream decomposition and the standard temporal logic chain into the logic checker. The logic checker uses a matching algorithm based on directed graph or temporal logic to compare the type and order of each action in the decomposition point sequence with the node and edge relationships defined in the logic chain.

[0055] The video stream is acquired, and action decomposition points are extracted through keyframe recognition and pose estimation to form a sequence of decomposition points. A standard temporal logic chain is pre-established, and a directed graph is constructed to represent the node and edge relationships. The sequence of decomposition points is input into a logic checker, and a topological sorting algorithm is used to compare the node order in the graph with the action type and occurrence order in the sequence. If the matching result shows a difference, abnormal paths are identified, and inconsistent nodes and edges are extracted from the directed graph. A check report is generated based on the abnormal paths to determine the consistency between the overall sequence and the standard temporal logic chain.

[0056] The video stream is first processed by the action decomposition module to obtain a sequence of step decomposition points. The action decomposition module uses a deep learning network based on temporal segmentation to analyze continuous video frames, dividing the video into independent action segments with clear beginnings and ends. Each action segment corresponds to a step decomposition point and is assigned an action type label, such as "wearing insulating gloves," "connecting the grounding wire," or "removing the grounding wire." The action decomposition process first extracts the spatiotemporal features of each frame, then uses a boundary detection mechanism to determine action switching points. When the feature changes of consecutive frames exceed a preset threshold, an action switch is considered to have occurred, thus forming the sequence of step decomposition points. It should be noted that each decomposition point in this sequence not only includes the action category but also records the approximate timestamp of the occurrence for subsequent sequential comparison. The action types in the step decomposition point sequence are labeled using a predefined finite set of actions. This set is determined according to the specific business domain; for example, in a live-line working scenario, the action set is limited to more than ten standard operating steps to avoid undefined irrelevant actions interfering with subsequent verification.

[0057] Based on the above steps, the resulting sequence of decomposed steps is input into the logic checker. The logic checker pre-stores a standard sequential logic chain, which is represented in the form of a directed graph. Nodes in the graph correspond to allowed action types, and directed edges between nodes indicate the allowed order of actions. For example, a directed edge from the "wearing insulating gloves" node to the "connecting the ground wire" node indicates that the next step is allowed only after the previous step is completed.

[0058] The directed graph is stored using an adjacency list, with each node maintaining a list of successor nodes. During verification, starting from the first decomposition point of the sequence, its action type is mapped to the corresponding node in the graph. Then, it checks whether the action type of the next decomposition point in the sequence appears in the current node's valid successor list. If so, the current node is updated as a successor node, and the comparison continues; if a mismatch occurs, it is determined to be an incorrect sequence or an incorrect action type. For example, if the sequence "remove grounding wire" is followed directly by "human body approaching the wire," the verifier will determine it as a violation of the operation sequence because the "remove grounding wire" node in the graph does not have an edge pointing to "human body approaching the wire." The logic verifier uses a timing logic-based matching method. The standard process is expressed as a linear timing logic formula, such as (wearing insulating gloves → connecting grounding wire), which means that once the action of wearing insulating gloves occurs, the action of connecting grounding wire must eventually occur. After receiving a complete or partial sequence of decomposition points, the verifier runs an automaton to verify the formula online.

[0059] Understandably, the choice between directed graph matching and sequential logic matching can be made based on process complexity and real-time requirements. In mobile verification devices with high real-time requirements, directed graph strict path matching, which has lower computational overhead, is preferred. In backend auditing systems, however, sequential logic automata, with their stronger expressive capabilities, can be used for more comprehensive compliance assessments. When verification fails, the system can trigger audible and visual alarms or send voice prompts to operators, enabling real-time intervention in the process and further ensuring operational compliance.

[0060] S5. If the logic checker finds that there is an action type that does not match the logic chain node in the step decomposition point sequence, it marks the action as a type anomaly point. If it finds that the order between actions violates the edge constraints of the logic chain, it marks the order relationship as an order violation point.

[0061] For each action in the step decomposition point sequence, determine whether its action type matches the node type at the corresponding position in the logical chain. If they do not match, mark the action as a type anomaly. Based on the directed edge constraints of the logical chain, sequentially obtain the order relationship between adjacent actions and determine whether the order relationship violates the directed edge constraints. If it does, mark the adjacent order relationship as an order violation point. Summarize all type anomalies and order violation points to obtain the complete anomaly marking result of the step decomposition point sequence.

[0062] After receiving the step decomposition point sequence, the logic validator first performs type matching verification. It compares the type identifier of each action in the sequence with the type identifiers of all nodes in the logic chain. If the type of an action has no matching entries in any node of the logic chain, it is determined to be a type anomaly, and the position and specific type of the action are recorded. Based on the type verification, the sequence of actions that passed the type verification is then subjected to sequence constraint verification. Specifically, this verification process involves traversing adjacent action pairs in the step decomposition point sequence and checking whether there is a corresponding directed edge between the previous and next actions. If there is no allowed edge relationship between adjacent actions, or if there is a reverse edge but the logic chain does not allow reversal, it is determined to be a sequence violation. By separately marking type anomalies and sequence violations, it is possible to clearly distinguish between action definition errors and execution timing errors.

[0063] S6, for the set of type anomalies and sequence violations output by the logic verifier, the deviation detector calls the preset violation pattern library to perform pattern matching, and analyzes whether the anomaly or violation belongs to a known typical violation operation pattern or an unknown new deviation.

[0064] The temporal correlation strength between each violation point is calculated by combining the set of anomalies and the sequence of violation points to obtain the contextual correlation vector of the violation sequence. This vector is then matched against a pre-set pattern feature library to determine whether there is a complete or partial similarity to a known typical violation template. If a match is found, the deviation type label is output as "known violation". If no known typical violation template is found, features corresponding to the unmatched portion are extracted from the contextual correlation vector to generate a new deviation identifier, which is then recorded in the historical deviation record. The new deviation identifier is compared with existing records in the historical deviation record. If the comparison confirms that it is new content, the new deviation feature is added to the pre-set pattern feature library for subsequent real-time deviation pattern matching judgment.

[0065] After receiving the set of anomalies by type and the set of violations by sequence from the logic checker, the deviation detector first arranges these anomalies according to their timestamps or process node order, forming an ordered sequence of anomalies. A pre-defined violation pattern library stores various known typical violation operation patterns. These patterns are represented as templates, which can be combinations of multiple violations in a fixed order or fuzzy sequence templates with a certain tolerance range. Each pattern in the violation pattern library includes attributes such as pattern name, a list of violation point types, order constraints between points, and frequency thresholds, facilitating rapid retrieval and comparison during subsequent matching.

[0066] The deviation detector employs a template-based sliding window matching method to scan abnormal sequences segment by segment. The window length is dynamically adjusted based on the number of points in the longest violation chain within the pattern. A successful match is determined when the abnormal point sequence within the window meets the preset similarity requirements for type and order constraints with a template in the pattern library. It should be noted that the similarity calculation comprehensively considers multiple dimensions such as point type consistency, order consistency, and time interval deviation. For example, 1 point is awarded for completely identical types, 0.7 points for reversed but adjacent sequences, and 0.4 points for time intervals exceeding a threshold. The final weighted sum is compared with a preset threshold to determine whether it is classified as a known pattern. When a pattern already existing in the violation pattern library is matched, the deviation detector marks the abnormal sequence as a "known typical violation operation pattern" and outputs the corresponding pattern identifier and confidence score for subsequent targeted verification by compliance personnel. When no matching item meeting the threshold is found in the violation pattern library for an abnormal sequence, the deviation detector classifies it as an "unknown novel deviation" and records the entire sequence, along with the occurrence time, business context, and detailed anomaly information, and pushes it to the manual review queue or incremental learning module. To improve the ability to capture new types of deviations, the system can periodically send the accumulated unknown deviation sequences into the pattern library update process. After expert confirmation, they are transformed into new violation pattern templates, thereby realizing the dynamic expansion of the violation pattern library.

[0067] S7. Based on the overall matching result of the logic checker and the output of the deviation detector, calculate the consistency score between the current operation process and the standard sequential logic chain, and determine whether the process integrity of the current operation meets the safety procedure requirements based on the preset consistency threshold.

[0068] The system acquires the overall matching degree result output by the logic checker and the deviation quantification value output by the deviation detector. The overall matching degree result represents the degree of conformity between the current work process and the standard timing logic chain, and the deviation quantification value represents the cumulative degree of deviation of each step from the standard timing. The overall matching degree result and the deviation quantification value are weighted and fused using a pre-set weighting coefficient to obtain a consistency score. The consistency score is compared with a preset consistency threshold. If the consistency score is greater than or equal to the preset consistency threshold, the current work process is determined to meet the safety procedure requirements. If the consistency score is less than the preset consistency threshold, the current work process is determined to not meet the safety procedure requirements. The judgment result is output as the conclusion of the work process integrity compliance for this operation.

[0069] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0070] In another aspect, the present invention also proposes a computer program product, comprising a computer program, characterized in that the computer program implements the above-described method when executed by a processor.

[0071] In particular, according to some embodiments of this disclosure, the processes described above can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0072] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a task data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated task data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0073] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital task data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0074] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine the network connection status of the switch production line management application in response to detecting a query operation on a production collaboration document in the switch production line management application; replace the webpage entry information corresponding to the production collaboration document with target entry file information and load target webpage resource information in response to determining that the network connection status of the switch production line management application indicates an offline state, so as to display the webpage of the production collaboration document offline in the switch production line management application, wherein the target entry file information is the file information of the entry file corresponding to the webpage of the production collaboration document downloaded in advance, and the target webpage resource information is the resource information corresponding to the webpage stored locally; in response to determining that the network connection status of the switch production line management application indicates an online state and that the webpage resource information corresponding to the production collaboration document is not stored locally, download the webpage resource information of the webpage from the production line document server, wherein the webpage resource information includes an entry file and resource information; display the webpage of the production collaboration document in the switch production line management application according to the webpage resource information, and store the webpage resource information in a local database.

[0075] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including product-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for conducting on-site safety analysis of power grid construction using intelligent video technology, characterized in that, Includes the following steps: S1. Acquire continuous video stream data of power grid on-site operations. For this video stream, use an action recognition algorithm based on deep learning skeleton point detection to extract the coordinate sequence of the skeleton key points of the workers frame by frame. S2, based on the preset set of operation steps, combined with the motion trajectory and spatial position relationship of the skeleton key point coordinate sequence, identifies each discrete atomic operation action in the video, and determines the start frame and end frame of each atomic operation action, thereby decomposing the continuous video stream into a sequence of step decomposition points with timestamps. S3 reads the set of standard operating steps corresponding to the current task and its prescribed timing logic chain from the preset procedure knowledge base. The timing logic chain clarifies the dependencies between each operating step and the allowed execution time window. S4. Input the sequence of decomposition points obtained from the video stream decomposition and the standard temporal logic chain into the logic checker. The logic checker uses a matching algorithm based on directed graph or temporal logic to compare the type and order of each action in the decomposition point sequence with the node and edge relationships defined in the logic chain. S5. If the logic checker finds that there is an action type that does not match the logic chain node in the step decomposition point sequence, it marks the action as a type anomaly point. If it finds that the order between actions violates the edge constraints of the logic chain, it marks the order relationship as an order violation point. S6, for the set of type anomalies and sequence violations output by the logic verifier, the deviation detector calls the preset violation pattern library to perform pattern matching, and analyzes whether the anomaly or violation belongs to a known typical violation operation pattern or an unknown new deviation. S7. Based on the overall matching result of the logic checker and the output of the deviation detector, calculate the consistency score between the current operation process and the standard sequential logic chain, and determine whether the process integrity of the current operation meets the safety procedure requirements based on the preset consistency threshold.

2. The method for on-site safety analysis of power grid construction using intelligent video technology according to claim 1, characterized in that, Step S1 includes: Continuous video stream data collected by cameras in the power grid field operation area is acquired, and the video stream is input frame by frame into a skeleton point detection network to obtain a two-dimensional coordinate set of the skeletal key points of the worker in each frame; the two-dimensional coordinate sets of the skeletal key points of each frame are connected sequentially in time order to form a continuous sequence of skeletal key point coordinates of the worker reflecting the relative position of the joints and the movement trajectory. The displacement distance and angle change of each joint between adjacent frames are calculated based on the coordinate sequence of key points of the skeleton, so as to obtain the limb movement speed characteristics and posture rotation characteristics of the operator, and obtain a complete time-series data record that can be used for subsequent power grid on-site operation safety status determination.

3. The method for on-site safety analysis of power grid construction using intelligent video technology according to claim 2, characterized in that, Step S2 specifically includes: The coordinates of the skeletal key points in each frame of the video sequence are obtained. The smoothed trajectory sequence is obtained through filtering and used for subsequent motion feature extraction. The Euclidean distance and relative displacement vector between key points in adjacent frames are calculated to obtain the feature sequence representing the spatial relationship between key points. For the feature sequence of the spatial correlation, calculate the consistency of displacement amplitude and direction of key points in consecutive frames, determine the intensity change position of the motion start and end, and obtain preliminary motion boundary candidate points; if there is an interval in the frame sequence between adjacent candidate points where the motion amplitude is continuously lower than the preset change range, then merge the candidate points at both ends of the interval to obtain the adjusted motion interval sequence. By matching and verifying the adjusted action interval sequence with the spatial association features of key points, the start frame number and end frame number of each atomic operation action are determined, and the step decomposition point sequence with frame timestamps is output.

4. The method for on-site safety analysis of power grid construction using intelligent video technology according to claim 3, characterized in that, Step S3 specifically includes: Obtain the set of standard operation steps corresponding to the current task from the preset procedure knowledge base, and at the same time read the temporal logic chain associated with the set of standard operation steps for link parsing to determine the sequential dependencies between each operation step, thereby obtaining the dependency constraint set and the sequential sequence. Calculate the execution time window for each operation step to obtain the time boundary range of each step. If the time boundary ranges overlap or misalign, it is determined that the time window constraint matching fails and the execution of the current link is blocked; if the time boundary ranges meet the constraints, the step triggering order is determined. By triggering the sequential and timing logic chain through the aforementioned steps, the state transition of the standard operation step set is completed, resulting in an executable procedure flow sequence.

5. A method for on-site safety analysis of power grid construction using intelligent video technology according to claim 4, characterized in that, Step S4 specifically includes: The video stream is acquired, and action decomposition points are extracted through keyframe recognition and pose estimation to form the decomposition point sequence. A standard temporal logic chain is pre-established, and a directed graph is constructed to represent the node and edge relationships. The decomposed point sequence is input into a logic verifier, and a topological sorting algorithm is used to compare the node order in the graph with the action type and occurrence order in the sequence. If the matching result shows a difference, abnormal paths are identified, and inconsistent nodes and edges are extracted from the directed graph. A verification report is generated based on the abnormal path to determine the consistency between the overall sequence and the standard timing logic chain.

6. A method for on-site safety analysis of power grid construction using intelligent video technology according to claim 5, characterized in that, Step S5 specifically includes: For each action in the step decomposition point sequence, determine whether its action type matches the node type at the corresponding position in the logical chain. If they do not match, mark the action as a type anomaly point. Based on the directed edge constraints of the logical chain, obtain the sequential relationship between adjacent actions in turn, and determine whether the sequential relationship violates the directed edge constraints. If it does, mark the adjacent sequential relationship as a sequence violation point. By summarizing all types of anomalies and sequential violations, a complete anomaly labeling result for the step decomposition point sequence is obtained.

7. A method for on-site safety analysis of power grid construction using intelligent video technology according to claim 6, characterized in that, Step S6 specifically includes: The temporal correlation strength between each violation point is calculated by the set of anomalies and the sequence of violation points. The contextual correlation vector of the violation sequence is obtained and matched with the preset pattern feature library to determine whether there is a complete hit or partial similarity of a known typical violation template. If a match is found, the deviation type label is output as "known violation"; if no known typical violation template is found, the features corresponding to the unmatched part are extracted from the context association vector, a new deviation label is generated and recorded in the historical deviation record. The new deviation identifier is compared with existing records in the historical deviation record. If the comparison confirms that it is new content, the new deviation feature is added to the preset pattern feature library for subsequent real-time deviation pattern matching judgment.

8. A method for on-site safety analysis of power grid construction using intelligent video technology according to claim 7, characterized in that, Step S7 specifically includes: The overall matching degree result output by the logic checker and the deviation quantization value output by the deviation detector are obtained. The overall matching degree result represents the degree of conformity between the current operation process and the standard timing logic chain, and the deviation quantization value represents the cumulative degree of deviation of each link from the standard timing. The overall matching result and the deviation quantification value are weighted and fused using a pre-set weighting coefficient to obtain the consistency score; The consistency score is compared with a preset consistency threshold. If the consistency score is greater than or equal to the preset consistency threshold, the current work process is determined to meet the safety procedure requirements. If the consistency score is less than the preset consistency threshold, the current work process is determined to not meet the safety procedure requirements. The output judgment result serves as the conclusion of the process integrity and compliance for this operation.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.