Power operator behavior identification early warning system and method based on video analysis

Abstract

The invention discloses a video analysis-based electric power operation personnel behavior identification and early warning system and method, which realize accurate identification and real-time early warning of electric power operation personnel behaviors by combining a video analysis technology with multi-modal data fusion, and effectively improve the safety management level of an operation site. Compared with a traditional safety supervision mode, the method employs a mode of combining deep learning target detection and time sequence behavior analysis, improves the recognition accuracy of operators and safety equipment, fuses the data of equipment worn by the operators with video data, improves the detection precision, and improves the safety supervision accuracy. And misjudgment caused by illumination change, shielding or complex environment is reduced. Besides, high-risk violation behaviors such as no safety helmet wearing, no safety belt fastening, violation climbing, high-altitude object throwing and the like are accurately recognized through the violation detection module, and different levels of alarm measures are adopted according to the severity of the violation behaviors in combination with an early warning feedback mechanism, so that the pertinence and response efficiency of early warning are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power operation safety monitoring, and particularly relates to a power operation personnel behavior recognition and early warning system and method based on video analysis. Background Art

[0002] In the field of power operation, safety management is the core element to ensure the life safety of personnel and the stable operation of equipment during power construction, maintenance, and overhaul. Power operation involves various high-risk work types such as high-altitude operation, live operation, and operation in complex environments. Operators need to strictly follow safety operation procedures to reduce the probability of safety accidents. With the development of intelligent monitoring technology, using technical means such as video analysis, artificial intelligence, and sensor fusion to intelligently identify the behavior of operators and give early warnings of violations has become an important direction to improve the efficiency of operation safety management.

[0003] Currently, the safety supervision of power operation sites mainly relies on manual inspections, fixed video monitoring, and the operators' conscious compliance with safety regulations. However, these traditional safety management methods have certain limitations: First, manual inspections are affected by human subjective factors, making it difficult to achieve all-weather and full-coverage operation safety supervision, and it is easy to have monitoring blind spots; Second, traditional video monitoring is mainly used for post-event retrospective, lacking real-time intelligent analysis capabilities and unable to actively detect the violation behaviors of operators; In addition, although some intelligent monitoring systems introduce object detection and behavior recognition technologies, they often rely on single visual data for analysis, and are easily interfered by factors such as light changes, occlusion, and complex operation environments, resulting in misjudgment or missed judgment. In addition, the current safety supervision means lack three-dimensional visualization management based on the operation scenario, making it difficult to intuitively present remote monitoring and safety assessment, which affects the management efficiency.

[0004] At power operation sites, due to the complex operation environment and variable human behaviors, the existing technologies still have certain limitations in the intelligent detection and early warning of operators' violation behaviors, especially in improving the accuracy of violation recognition, reducing the false alarm rate, and enhancing the operation safety visualization management ability. Therefore, how to improve the accuracy of operator behavior recognition in power operation safety monitoring, reduce misjudgment caused by light changes, occlusion, and environmental interference, and enhance remote monitoring and visualization management capabilities is still a technical problem to be solved urgently. Summary of the Invention

[0005] The purpose of the present invention is to provide a power operation personnel behavior recognition and early warning system and method based on video analysis to overcome the technical problem of poor accuracy in identifying the violation behaviors of existing operators.

[0006] To solve the above problems, the present invention adopts the following technical solutions: A behavior recognition and early warning system for power operation personnel based on video analysis, acquisition and processing module: used to collect real-time video data of the power operation site and preprocess the video data; Target analysis module: Based on the processed video data, use deep learning target detection algorithms to detect operation personnel, safety equipment and operation environment, generate detection results, and use human pose estimation algorithms and temporal behavior analysis algorithms to identify and analyze the behaviors of operation personnel; Violation judgment module: used to judge whether there are any violation operations by operation personnel based on the detection and recognition analysis results of the target analysis module and combined with power operation specifications; Feedback and optimization module: used for early warning of violation behaviors, remote monitoring, fusing the data of the equipment worn by operation personnel and video data, training a behavior recognition model in combination with historical data, and optimizing the violation behavior detection ability based on online learning technology.

[0007] Furthermore, the real-time video data of the power operation site is collected through fixed monitoring cameras, wearable camera devices and UAV camera devices; Preprocessing the video data includes performing video denoising processing using a denoising neural network, using an adaptive enhancement algorithm to improve the clarity of the target area, and performing time synchronization processing on the video data and multi-modal sensor data through the Precision Time Protocol.

[0008] Furthermore, the target detection algorithm includes YOLOv8 or EfficientDet.

[0009] Furthermore, the human pose estimation algorithm includes OpenPose or HRNet, and the temporal behavior analysis algorithm includes TSM, SlowFast network or Transformer-based temporal model.

[0010] Furthermore, the method used for fusing the data of the equipment worn by operation personnel and video data is Kalman filtering and Bayesian fusion.

[0011] Furthermore, the equipment worn by operation personnel includes an inertial measurement unit, a global positioning system and a radio frequency identification device.

[0012] Furthermore, the early warning of violation behaviors is divided into three levels, among which: Level 1 early warning is applicable to high-risk violation behaviors, including not wearing a safety helmet, not fastening a seat belt, falling, and the system immediately triggers an audible and visual alarm and notifies the monitoring center; Level 2 early warning is applicable to medium-risk violation behaviors, including incorrect operation postures and deviation of the operation range, and the system sends a mobile terminal notification to the operation personnel; The third-level warning applies to minor violations, including failure to follow operating specifications within a short period. The system records the violation information and incorporates it into subsequent safety assessments.

[0013] Furthermore, the remote monitoring implementation method is to construct a three-dimensional virtual operation environment based on BIM modeling technology or lidar scanning, and combine video data, the location information of operating personnel, and sensor data to real-time map the behavior status of operating personnel in the virtual environment.

[0014] In a second aspect, a method for identifying and warning the behavior of electric power operating personnel based on video analysis is provided, including the following steps: Collect real-time video data of the operation site and preprocess the real-time video data; Based on the processed video data, use deep learning object detection algorithms to identify operating personnel, safety equipment, and the operation environment; Use human pose estimation algorithms and temporal behavior analysis algorithms to identify and analyze the behavior of operating personnel; Combine operating personnel, safety equipment, the operation environment, and the behavior of operating personnel to determine violations; Warn against violations and conduct remote monitoring, fuse the data of the devices worn by operating personnel and video data, train a behavior recognition model in combination with historical data, and optimize the violation detection ability based on online learning technology.

[0015] Furthermore, the violation detection steps include static violation detection, dynamic violation detection, and regional violation detection, and classify the behavior of operating personnel based on a machine learning model.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a system for identifying and warning the behavior of electric power operating personnel based on video analysis. Through video analysis technology combined with multi-modal data fusion, it realizes the accurate identification and real-time warning of the behavior of electric power operating personnel, effectively improving the safety management level of the operation site. Compared with traditional safety supervision methods, the present invention adopts a combination of deep learning object detection and temporal behavior analysis, improving the accuracy of identifying operating personnel and safety equipment. At the same time, it fuses the data of the devices worn by operating personnel and video data, improving the detection accuracy and reducing misjudgments caused by light changes, occlusion, or complex environments. In addition, the present invention accurately identifies high-risk violations such as not wearing a safety helmet, not fastening a seat belt, illegal climbing, and throwing objects from a height through a violation detection module, and combines a warning feedback mechanism to take different levels of alarm measures according to the severity of the violation, improving the pertinence and response efficiency of the warning.

[0017] The system architecture of the present invention covers the complete process from data acquisition, behavior recognition, violation detection, data fusion, early warning feedback to digital twin, realizing intelligent safety supervision of power operation and improving the safety and management efficiency of the operation site.

[0018] At the same time, the modeling of the operation environment based on digital twin technology enables the behavior of operators to be mapped in real time in a three-dimensional visualization scene, enhancing the intuitiveness of remote monitoring and operation management; the present invention can effectively reduce the risk of operators' violations, reduce the occurrence of safety accidents, and improve the safety and supervision efficiency of power operation.

[0019] The system optimization module combines historical operation data and optimizes the behavior recognition model through deep learning training and online learning technology to improve the adaptive ability of the system.

[0020] The behavior recognition module fuses human pose estimation and temporal behavior analysis to improve the accuracy of dynamic behavior recognition.

[0021] The present invention also provides a method for behavior recognition and early warning of power operation personnel based on video analysis, which is applicable to scenarios such as high-altitude operation, transmission line maintenance, and substation maintenance. Through steps such as video acquisition, data preprocessing, target detection, behavior recognition, violation detection, multi-modal data fusion, early warning feedback, and system optimization, it realizes real-time monitoring of the behavior of operation personnel and violation early warning, so as to reduce the incidence of safety accidents and improve the operation safety and management efficiency of the power operation site. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 It is a structural diagram of a behavior recognition and early warning system for power operation personnel based on video analysis in an embodiment of the present invention; Figure 2 It is a flowchart of a method for behavior recognition and early warning of power operation personnel based on video analysis in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention clearer, the following specific embodiments are used to further elaborate on the present invention. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0023] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] In the description of the embodiments of the present invention, it should also be noted that unless otherwise clearly specified and limited, if the terms "set", "install", "connect", and "link" are used, they should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0025] A power operation personnel behavior recognition and early warning system based on video analysis, comprising: An acquisition and processing module: used to acquire real-time video data of the power operation site and preprocess the video data; Specifically, as Figure 1 shown, the acquisition and processing module includes a video acquisition module and a data preprocessing module: The main function of the video acquisition module is to obtain real-time video data of the operation site, ensure that the system has complete operation environment information and operation personnel behavior information, and provide a high-quality video source for subsequent target detection and behavior recognition. The design of this module needs to consider the complexity of different operation scenarios, including high-altitude operations, substation operations, line inspections, etc. Therefore, this module adopts a variety of video acquisition methods, including fixed monitoring, wearable camera devices, and unmanned aerial vehicle (UAV) inspection devices, to ensure full-range monitoring coverage; Fixed monitoring cameras: These devices are installed in key areas of the operation site, such as around high-voltage equipment, high-altitude operation areas, operation channels, etc. Their advantage is that they can provide stable monitoring images and are suitable for long-term, all-weather video monitoring; they are connected to the central server through wired or wireless networks to ensure the stable transmission of video data.

[0026] Wearable camera devices: Installed on the helmets or chests of operation personnel, they can obtain first-person perspective video data of the operation personnel. These devices can move with the operation personnel and are suitable for environments where fixed cameras cannot be installed, such as high-altitude operations and equipment internal maintenance; the video data is transmitted to the system in real time via wireless means (Wi-Fi, 5G) to ensure low-latency data updates.

[0027] Unmanned aerial vehicle (UAV) camera devices: Used for high-altitude inspections and long-distance monitoring. UAVs can fly autonomously to acquire video data of specific operation areas, especially suitable for scenarios that require large-scale monitoring such as transmission line inspections and high-tower operations; the video data collected by UAVs can be transmitted back to the system in real time via wireless networks or stored in local memories and uploaded to the server for processing after the task is completed.

[0028] Optionally, this module uses a high-resolution (1080P or 4K) camera device to ensure the clarity of video data. Meanwhile, the camera device has a night vision function and can work properly in low-light environments. In addition, to adapt to harsh weather conditions, the camera device has waterproof and dustproof capabilities and supports high dynamic range (HDR) technology to cope with complex lighting conditions such as strong light and shadows.

[0029] The main function of the data preprocessing module is to optimize the collected video data, including operations such as denoising, image enhancement, key frame extraction, and time synchronization, to improve the data quality and provide high-quality input for subsequent object detection and behavior recognition.

[0030] Among them, the denoising is carried out in the following ways: Remove noise through methods such as Gaussian filtering and median filtering, while retaining the video detail information; Use a convolutional neural network (CNN) to perform denoising training on the video data so that it can adapt to the noise characteristics of different scenarios and improve the image quality.

[0031] The image enhancement is carried out in the following ways: Use histogram equalization technology to improve the contrast of the image and make the target clearer; Use a super-resolution neural network (SRGAN) to optimize the low-resolution video and improve the image clarity.

[0032] The key frame extraction is carried out in the following ways: Based on optical flow analysis: By calculating the optical flow changes between video frames, filter out the key frames to improve the calculation efficiency; Based on deep learning: Use LSTM (long short-term memory network) to perform sequence analysis on the video and extract the most representative frames.

[0033] The time synchronization is carried out in the following ways: Use PTP (Precision Time Protocol) for time synchronization to ensure the time consistency of video, inertial measurement unit (IMU), GPS and other data; Use an adaptive correction algorithm to reduce the errors caused by data asynchronization and improve the accuracy of behavior recognition.

[0034] Object analysis module: Based on the processed video data, use deep learning object detection algorithms to detect operators, safety equipment, and the working environment, generate detection results, and use human pose estimation algorithms and temporal behavior analysis algorithms to identify and analyze the behaviors of operators; Specifically, as Figure 1 shown, the object analysis module includes an object detection module and a behavior recognition module, where: The main function of the target detection module is to identify key objects such as workers, safety equipment, and objects in the working environment, and provide high-precision detection results. The module uses deep learning target detection algorithms to ensure the real-time performance and accuracy of detection. Worker detection: Use algorithms such as YOLOv8 or EfficientDet to perform real-time detection of workers in the video, and identify information such as their positions and postures. Safety protection equipment detection: Detect whether workers are wearing safety helmets, safety belts, insulating gloves, etc., and judge whether the wearing complies with the specifications. Working environment detection: Identify obstacles, danger area warning signs, power equipment, etc. in the working area to ensure the safety of the working environment.

[0035] Since the sizes of workers and equipment vary greatly, the FPN (Feature Pyramid Network) and Anchor-Free detection methods are adopted to improve the detection accuracy of targets at different scales. The object detection algorithm based on Transformer (DETR) is used to improve the detection speed and reduce the consumption of computing resources at the same time.

[0036] The process of target detection is as follows: 1. The video data processed by the preprocessing module is input into the target detection model. 2. The target detection model uses a neural network to analyze the video data and outputs the target box and class information. 3. The target detection results are transmitted to the behavior recognition module and the violation detection module, serving as the basis for subsequent behavior analysis.

[0037] The behavior recognition module is one of the core components of the system. Its main task is to analyze the behaviors of workers, judge whether they comply with safety operation specifications, and identify potential violations or dangerous behaviors. Different from static target detection, the module adopts temporal behavior analysis technology, extracts the dynamic features of workers through consecutive video frames, and combines deep learning models for behavior classification to achieve accurate recognition of different working actions.

[0038] The core of behavior recognition is human pose estimation and temporal behavior analysis: Human pose estimation: Based on deep learning models such as OpenPose or HRNet, extract human key points from video frames, including the head, limbs, torso, etc. Build a human skeleton model through the positions and angles of these key points. Temporal behavior analysis: Adopt TSM (Temporal Shift Module), SlowFast network or Transformer-based temporal models to perform temporal modeling on the pose changes of workers and analyze whether the behaviors comply with working specifications.

[0039] The module can accurately identify the following behaviors: 1. Normal operation behaviors: including compliant operations such as standing work, squatting operations, tool use, etc.; 2. High-risk violation behaviors: Not wearing a safety helmet: Judging whether there is a safety helmet target on the key points of the head; Not wearing a seat belt: Analyzing the waist area of the operator and combining object detection to judge whether the seat belt is properly fastened; Illegal climbing: Through human key point analysis, identifying whether the operator climbs high-altitude structures in a non-standard way; Fall detection: Detecting sudden changes in the human skeleton, such as center of gravity shift, imbalance, etc., and verifying the fall event in combination with inertial sensor data; Single-handed operation: Identifying whether the operator uses a tool with one hand during high-altitude operations, which may cause the tool to slip out of the hand or operational errors; The behavior recognition module integrates human pose estimation and temporal behavior analysis, improving the accuracy of dynamic behavior recognition; combining a Transformer-based behavior classification model, it realizes efficient temporal modeling and improves the recognition ability for complex behaviors; multi-modal behavior analysis, combining video data and IMU sensor data, reduces recognition errors caused by occlusion or lighting changes.

[0040] Behavior recognition process: 1. The object detection module identifies the operator and extracts human key point data; 2. The temporal behavior analysis model analyzes the changes of key points in the time dimension; 3. Combining multi-modal data, comprehensively judging the behavior category of the operator; 4. Sending the recognition result to the violation detection module for violation determination.

[0041] Violation determination module: Used to judge whether the operator has violated operations based on the detection and recognition analysis results of the target analysis module and in combination with electrical operation specifications; The module combines the results of object detection, behavior recognition and multi-modal data fusion, classifies the violation operations, and generates a risk assessment report.

[0042] Violation behavior detection includes: Static violation detection: Through object detection algorithms, identifying whether the operator wears necessary safety equipment, such as safety helmets, seat belts, insulating gloves, etc.; Dynamic violation detection: Combining the behavior recognition results, judging whether the operator has violation actions, such as incorrect use of the seat belt, wrong climbing, falling, etc.; Area Violation Detection: Combine GPS or RFID data to determine whether the operator has entered an unauthorized dangerous area, such as a high-voltage equipment area.

[0043] This module uses the following technical means to improve the accuracy of violation detection: Rule Matching: Based on the power safety operation standards, formulate rules for judging violation behaviors, such as "seat belt not connected to a fixed point = violation".

[0044] Machine Learning Model: Train a deep learning model to classify different behavior patterns and improve the detection accuracy of complex violation behaviors.

[0045] Anomaly Detection Algorithm: Based on historical data, use anomaly detection methods (such as PCA, Autoencoder) to identify abnormal behaviors of operators.

[0046] Feedback Optimization Module: Used for violation behavior warning, remote monitoring, fusing the data of the devices worn by operators and video data, training a behavior recognition model in combination with historical data, and optimizing the violation behavior detection ability based on online learning technology.

[0047] Specifically, as Figure 1 shown, the feedback optimization module includes a multi-modal data fusion module, a warning feedback module, a digital twin system, and a system optimization module, where: The main task of the multi-modal data fusion module is to fuse video data with the multi-modal sensor data of operators to improve the accuracy of violation behavior recognition. By combining visual information and non-visual data, misjudgments caused by factors such as lighting and occlusion can be effectively reduced.

[0048] The main data sources of the multi-modal sensor data of operators are: Inertial Measurement Unit (IMU): Obtain the acceleration and angular velocity information of the operator to judge whether the posture change is abnormal.

[0049] Global Positioning System (GPS): Determine the geographical location of the operator, compare it with the scope of the safe area, and identify whether there is a violation of entering a dangerous area.

[0050] Radio Frequency Identification (RFID): Used for identity verification to ensure that the operator's identity meets the requirements of the operation permit.

[0051] The technologies adopted by the multi-modal data fusion module are: Time Synchronization Technology: Use the Precision Time Protocol (PTP) to synchronize different data sources to ensure data consistency; Data Fusion Algorithm: Use methods such as Kalman filtering and Bayesian fusion to fuse visual data and sensor data to improve the recognition accuracy; Abnormal data correction: Through the sensor redundancy mechanism, abnormal data is detected and compensated or corrected to reduce the misjudgment rate.

[0052] The task of the warning feedback module is to take corresponding warning and intervention measures according to the violation detection results to ensure the safety of operators. The warning levels of this system are divided into three levels: Level 1 warning (high-risk violation): If the safety helmet is not worn or the safety belt is not fastened, the system immediately triggers an audible and visual alarm and sends an emergency alarm to the monitoring center; Level 2 warning (medium violation): If the operation posture is incorrect, the system sends a reminder to the operator's mobile device to urge them to adjust the operation method; Level 3 warning (minor violation): If the standard operation is not followed within a short period of time, the system records the violation behavior and provides a risk analysis report after the operation ends.

[0053] The warning methods are as follows: Audible and visual alarm: The equipment at the operation site (such as warning lights, buzzers) emits an alarm signal to remind the operator to pay attention to safety; Mobile notification: The operator's intelligent device (such as a smart watch, mobile phone APP) receives a warning message and provides operation suggestions; Remote monitoring warning: The monitoring center system receives the violation alarm and can take emergency measures, such as remotely shutting down the operation equipment.

[0054] Digital twin technology is used to establish a three-dimensional virtual model of the operation site and map the behavior of operators to the virtual environment in real time to achieve remote visual monitoring.

[0055] The technologies adopted by the digital twin system are as follows: Three-dimensional modeling: Based on BIM (Building Information Modeling) or lidar scanning technology, a digital twin model of the power operation environment is constructed.

[0056] Real-time mapping: Project the behavior data (position, posture, violation behavior) of operators into the digital twin environment to achieve remote monitoring.

[0057] Risk prediction: Based on historical data, predict possible safety accidents and provide preventive suggestions.

[0058] The system architecture of the present invention covers the complete process from data acquisition, behavior recognition, violation detection, data fusion, warning feedback to digital twin, realizing intelligent safety supervision of power operations and improving the safety and management efficiency of the operation site.

[0059] The system optimization module includes: (1) Historical data training Train the historical violation data using a deep learning model to improve the adaptive ability of the model.

[0060] (2) Online learning and update Through the incremental learning algorithm, the system can automatically learn new types of violation behaviors and improve the recognition accuracy.

[0061] The present invention also provides a method for identifying and warning the behavior of power operation personnel based on video analysis, including the following steps: Collect real-time video data of the operation site and preprocess the real-time video data; Based on the processed video data, use the deep learning object detection algorithm to identify operation personnel, safety equipment, and the operation environment; Use the human pose estimation algorithm and the temporal behavior analysis algorithm to identify and analyze the behavior of operation personnel; Combine the operation personnel, safety equipment, operation environment, and the behavior of operation personnel to determine violations; Warn against violation behaviors and conduct remote monitoring, fuse the data of the devices worn by operation personnel and video data, train the behavior recognition model in combination with historical data, and optimize the violation behavior detection ability based on online learning technology.

[0062] Specifically, as Figure 2 shown, the detailed steps are: (1) Collect real-time video data of the operation site through the video acquisition module and transmit it to the central server; (2) Preprocess the video data, including denoising, image enhancement, key frame extraction, and time synchronization; (3) Use the target detection module to detect operation personnel, safety equipment, and the operation environment and generate detection results; (4) Use the behavior recognition module to analyze the actions of operation personnel and identify normal operation behaviors and violation behaviors; (5) Integrate video data, IMU data, GPS data, and RFID data through the multi-modal data fusion module to improve the accuracy of violation behavior recognition; (6) Determine whether there are violation behaviors of operation personnel through the violation detection module; (7) According to the severity of the violation behavior, take corresponding warning measures through the warning feedback module; (8) Map the behavior of operation personnel to a three-dimensional virtual environment through the digital twin system to achieve remote monitoring; (9) Combine historical data and use the system optimization module to train the behavior recognition model to improve the adaptive ability of the system.

[0063] The present invention also provides an application of a power operation personnel behavior recognition and early warning system based on video analysis in transmission line maintenance operations: 1. Application background During the maintenance of transmission lines, operators need to perform equipment maintenance, line maintenance, etc. in a high-altitude environment. The working environment is complex and the risks are relatively high. Common safety hazards include: operators not wearing safety helmets and seat belts correctly; not using anti-fall devices as required during high-altitude operations; improper postures during operations, with risks of imbalance or falling; tools falling in the high-altitude operation area, which may cause harm to ground personnel; safety operation mistakes caused by operator fatigue or inattention; unauthorized personnel entering high-risk operation areas.

[0064] Traditional operation safety management methods rely on manual inspections and on-site monitoring, which have supervision blind spots and are difficult to detect and warn against violations in real time. To improve the efficiency of safety monitoring, this system is applied to the transmission line maintenance site to achieve intelligent monitoring and early warning of operator behavior.

[0065] 2. Specific implementation steps Step 1: Deploy video acquisition equipment (1) Installation of fixed monitoring cameras Install high-definition monitoring cameras on power towers near transmission lines and on-site ground stations in the operation area to ensure that key operation areas can be covered.

[0066] The monitoring cameras have high magnification and night vision capabilities to ensure normal video data acquisition in different lighting environments.

[0067] (2) Deployment of wearable camera devices Each operator wears an intelligent safety helmet with a miniature camera integrated on it to record the operation process in real time.

[0068] The wearable device can transmit video data back to the monitoring center via a wireless network (Wi-Fi / 5G).

[0069] (3) Drone inspection In large-scale transmission line inspection tasks, use drones equipped with high-definition cameras to provide additional monitoring perspectives in the air, especially suitable for mountainous areas or long-distance transmission lines where fixed cameras cannot be installed.

[0070] Step 2: Preprocessing of video data (1) Video denoising and enhancement Since the high-altitude environment may be affected by factors such as sand and wind, and lighting changes, the system first performs noise reduction processing on the video data to improve the video quality, and uses an adaptive enhancement algorithm to optimize the clarity of key areas (such as operators and line equipment) to improve the accuracy of subsequent analysis.

[0071] (2) Time synchronization Due to the diverse sources of on-site data (fixed cameras, wearable cameras, drone cameras), the system uses the Precision Time Protocol (PTP) to correct the time of all video streams, ensuring that data from different sources can be processed synchronously.

[0072] Step 3: Detection of workers' targets (1) Identification of workers The YOLOv8 object detection algorithm is used to accurately identify workers in the video frame, and a unique identification ID is assigned to each worker; Combined with GPS data, the specific location of the workers is determined and marked in real time on the system interface.

[0073] (2) Detection of safety equipment wearing situation The wearing situation of safety helmets of each worker is detected through a deep learning classification model. If it is detected that the helmet is not worn or worn improperly, the system automatically marks the risk level.

[0074] The target segmentation algorithm is used to analyze the waist area of the workers to judge whether the safety belt is fastened correctly. If the safety belt is not fastened, an alarm is triggered.

[0075] Step 4: Behavior recognition and violation detection (1) Pose analysis The OpenPose human key point detection algorithm is used to analyze the bone positions of the workers and calculate the angle and displacement changes between the key points.

[0076] If it is detected that the worker is in an abnormal pose (such as a hanging state, loss of balance), the system automatically judges it as a fall risk and triggers a warning.

[0077] (2) Analysis of violation behaviors Detection of unauthorized climbing: The system analyzes the movement trajectory of the workers. If it is detected that the workers climb an unauthorized tower structure, the system automatically identifies it as a violation behavior.

[0078] Detection of high-altitude throwing: Through object trajectory tracking, it is analyzed whether there is a tool or equipment falling off. If high-altitude throwing is detected, the system immediately notifies the ground personnel to take evasive action.

[0079] Step 5: Multimodal data fusion (1) Combining with IMU sensors The inertial measurement unit (IMU) worn by the workers continuously monitors acceleration and angular velocity information. The system combines video data analysis to determine whether the workers are in a dangerous pose or have fallen.

[0080] If a sudden posture change is detected (such as the body tilt angle exceeding the safety threshold), the system immediately issues an alarm.

[0081] (2) Combine GPS location information Compare the GPS coordinates of the operator with the preset safe operation area. If it is found that the person enters an unauthorized high-risk area, the system immediately sends a notice to the monitoring center.

[0082] (3) Combine RFID identity authentication In specific operation areas (such as beside high-voltage power equipment), set up an RFID access control system to ensure that only authorized personnel can enter.

[0083] If the video system detects a person without an RFID identity authentication device entering the operation area, the system triggers an alarm and notifies the on-site management personnel.

[0084] Step 6: Early warning feedback According to the severity of the violation, the system triggers different levels of early warnings: Level 1 early warning (high-risk violation) Trigger on-site audible and visual alarms to warn the operator to immediately rectify the violation.

[0085] The monitoring center receives an emergency notice and activates the emergency response mechanism.

[0086] The system records the event and archives it in the operation safety database.

[0087] Level 2 early warning (medium violation) Send a notice through a smartwatch or mobile application to remind the operator to adjust the operation method.

[0088] The staff in the monitoring center remotely reminds the operator to pay attention to safety by voice.

[0089] Level 3 early warning (minor violation) The violation is recorded by the system and incorporated into the post-operation safety analysis report.

[0090] If the same operator violates the regulations multiple times, the system generates a safety score report for subsequent training and assessment.

[0091] Step 7: Digital twin analysis (1) Virtual operation environment construction Based on the three-dimensional model of the transmission line, construct a digital twin environment and map the position, posture, and behavior of the operator to the virtual space.

[0092] The monitoring personnel can view the operation situation in real time in the virtual environment and conduct remote monitoring.

[0093] (2) Risk Prediction Through historical data analysis, the system can predict potential safety risks during operations and remind operators in advance to adjust their operation methods.

[0094] For example, by analyzing operation videos from the past week, the system found that the risk of falls at a certain location was relatively high and could arrange for additional safety inspections in advance.

[0095] 3. Application Effects Improve operation safety: This system can monitor the behavior of operators in real time, reduce accidents caused by illegal operations, and improve the level of operation safety management.

[0096] Enhance operation management efficiency: Through automated behavior recognition and early warning, reduce the dependence on manual inspections and improve the efficiency of safety management.

[0097] Data-driven safety optimization: The system automatically records illegal behaviors and provides detailed safety analysis reports, which helps with subsequent training and optimization of operation processes.

[0098] 4. Conclusion This application example demonstrates the practical application of the present invention in the maintenance of transmission lines, giving full play to the intelligent behavior recognition and early warning functions based on video analysis, and significantly enhancing the safety management ability of the power operation site.

[0099] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited thereto. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solutions of the present invention, including any other suitable combination of each technical feature. These simple modifications and combinations should also be regarded as the content disclosed by the present invention and fall within the protection scope of the present invention.

Claims

1. The power worker behavior recognition and early warning system based on video analysis is characterized by: include; Collection and processing module: used to collect real-time video data of the power operation site and pre-process the video data; Target analysis module: Based on the processed video data, the deep learning target detection algorithm is used to detect the operators, safety equipment and working environment, generate detection results, and the human posture estimation algorithm and time series behavior analysis algorithm are used to identify and analyze the behavior of the operators; Violation determination module: used to determine whether the operator has violated the regulations based on the detection and identification analysis results of the target analysis module and the power operation specifications; Feedback optimization module: used for violation warning, remote monitoring, integrating data from equipment worn by operators and video data, training behavior recognition models with historical data, and optimizing violation detection capabilities based on online learning technology.

2. According to claim 1, a power worker behavior recognition and early warning system based on video analysis is characterized in that: The real-time video data of the power operation site is collected by fixed monitoring cameras, wearable cameras and drone cameras; The preprocessing of the video data includes performing video denoising processing using a denoising neural network, improving the clarity of the target area using an adaptive enhancement algorithm, and performing time synchronization processing on the video data and multimodal sensor data through a precise time protocol.

3. The electric power worker behavior recognition and early warning system based on video analysis according to claim 1 is characterized in that: The target detection algorithm includes YOLOv8 or EfficientDet.

4. The electric power worker behavior recognition and early warning system based on video analysis according to claim 1 is characterized in that: The human body posture estimation algorithm includes OpenPose or HRNet, and the timing behavior analysis algorithm includes TSM, SlowFast network or Transformer-based timing model.

5. The electric power worker behavior recognition and early warning system based on video analysis according to claim 1 is characterized in that: The method used for fusing the data from the equipment worn by the operator and the video data is Kalman filtering and Bayesian fusion.

6. The electric power worker behavior recognition and early warning system based on video analysis according to claim 5 is characterized in that: The equipment worn by the operator includes an inertial measurement unit, a global positioning system and a radio frequency identification device.

7. The electric power worker behavior recognition and early warning system based on video analysis according to claim 1 is characterized in that: The violation warning is divided into three levels: Level 1 warning applies to high-risk violations, including not wearing a helmet, not wearing a seat belt, and falling. The system immediately triggers an audible and visual alarm and notifies the monitoring center; Level 2 warning applies to medium-risk violations, including incorrect working posture and deviation from the working scope. The system sends a mobile notification to the operator. The third-level warning applies to minor violations, including failure to follow operating specifications within a short period of time. The system records the violation information and includes it in subsequent safety assessments.

8. The electric power worker behavior recognition and early warning system based on video analysis according to claim 1 is characterized in that: The remote monitoring implementation method is to build a three-dimensional virtual working environment based on BIM modeling technology or lidar scanning, and combine video data, worker location information, and sensor data to map the worker's behavior status in real time in the virtual environment.

9. A method for identifying and warning the behavior of power workers based on video analysis, characterized in that: The electric power operator behavior recognition and early warning system based on video analysis according to any one of claims 1 to 8 comprises the following steps: Collect real-time video data at the work site and pre-process the real-time video data; Based on the processed video data, deep learning target detection algorithms are used to identify operators, safety equipment, and working environment; Use human posture estimation algorithm and time series behavior analysis algorithm to identify and analyze the behavior of operators; Determine violations based on the operator, safety equipment, operating environment, and operator behavior; Provide early warnings for violations and conduct remote monitoring, fuse data from equipment worn by operators with video data, train behavior recognition models based on historical data, and optimize violation detection capabilities based on online learning technology.

10. The method for identifying and warning the behavior of electric power workers based on video analysis according to claim 9, characterized in that: The violation detection steps include static violation detection, dynamic violation detection, and regional violation detection, and the operator behavior is classified based on a machine learning model.

Images