Intelligent monitoring and shooting method and system for bird threading and computer readable storage medium

Abstract

The invention discloses an intelligent monitoring and shooting method and system for bird threading and a computer readable storage medium, and relates to the technical field of computer vision and power equipment online monitoring. The objective of the invention is to solve the short-circuit fault problem caused by bird threading of an ultra-high voltage transmission line and overcome the defects of passive defense, high false alarm rate and insufficient intelligence in the prior art. The method comprises the steps of video stream acquisition and preprocessing, intelligent demarcation and geometric correction of a monitoring area, threading behavior recognition based on trajectory analysis, event triggering and data uploading, and data management and closed-loop optimization. The system adopts a cloud side-end collaborative architecture, comprises a front-end acquisition and intelligent processing unit, a communication transmission unit, a rear-end platform and management unit and a power supply sub-unit, and realizes real-time and accurate recognition and early warning of bird threading behaviors through a lightweight deep learning model integrated with an attention mechanism and an improved multi-target tracking algorithm. The closed-loop optimization mechanism continuously improves the model performance through knowledge distillation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and online monitoring technology of power equipment. More specifically, it relates to an intelligent monitoring method and system for birds crossing conductor gaps in ultra-high voltage transmission lines, which integrates lightweight deep learning, multi-target tracking, and trajectory behavior analysis, as well as a computer-readable storage medium. This method is specifically designed to address the risk scenario of large birds crossing conductor gaps in ultra-high voltage transmission lines, enabling accurate early warning and evidence collection. Background Technology

[0002] Ultra-high voltage (UHV) transmission lines form the backbone of the national power grid, and their safe and stable operation is of paramount importance. However, these lines often traverse ecologically rich areas such as mountains, lakes, and wetlands, which are also frequented by birds. When large birds (such as eagles, cranes, and storks) fly across the conductors (i.e., cross the lines), their large wingspans make them highly susceptible to simultaneously touching conductors at different potentials (such as phase wires and ground wires) or conductors and tower components, causing short-circuit grounding faults, triggering power outages, and resulting in significant economic losses and social impacts on the power grid.

[0003] Currently, bird damage control along power transmission lines mainly relies on bird deterrents (such as reflectors and windmill-type bird deterrents), bird spikes, and manual inspections. However, these methods have significant shortcomings: 1. Passive defense, lack of early warning capability: Existing methods are mostly passive defenses, which cannot provide early warning before birds approach or cross the line. They can only deal with the situation after the incident occurs, and cannot prevent it in advance. 2. Limited effectiveness and easy adaptation: The effectiveness of bird deterrents weakens as birds become accustomed to them; 3. Low efficiency of manual inspection: UHV lines are long and widely distributed, and manual inspection is labor-intensive, costly and time-consuming. In addition, it is difficult to detect bird crossing behavior that occurs in real time. 4. The existing monitoring system has a low level of intelligence: some lines are equipped with video monitoring devices, but most of them are timed shooting or image change detection, which are easily affected by changes in light and cloud shadows, generating a large amount of invalid data. Manual playback and identification are required. It cannot automatically and accurately identify bird crossing events in real time, resulting in a high false alarm rate and poor practicality.

[0004] In addition, some existing power line monitoring technologies focus on general abnormal behavior recognition or foreign object detection, without being customized for the specific scenario of birds crossing wires. This results in poor scenario adaptability, insufficient recognition accuracy, and high false alarm rates. Furthermore, the application of related algorithm formulas has not been optimized in combination with the movement characteristics of birds and the structural features of UHV lines, making it difficult to meet the needs of real-time and accurate recognition of wire crossing behavior.

[0005] Therefore, there is an urgent need for a system and method that can automatically, in real time and accurately identify bird crossing behavior in areas of ultra-high voltage transmission line spans based on video surveillance, and provide early warning and evidence collection. Summary of the Invention

[0006] The present invention aims to solve the technical problems mentioned in the background art and provide a method, system and computer-readable storage medium for intelligent monitoring of birds threading wires, so as to provide a low-cost and simple-to-deploy solution. Its core lies in achieving high-precision and low-false-alarm intelligent sensing of birds threading wires through a carefully designed resource-aware algorithm pipeline.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a smart monitoring method for birds threading wires, applied to ultra-high voltage transmission lines, comprising the following steps: S1: Video stream acquisition and preprocessing: Deploy cameras on ultra-high voltage transmission towers to continuously acquire video streams covering the conductor area of ​​the span, and preprocess the video streams; S2: Intelligent delineation and geometric correction of monitoring area: Delineate one or more virtual monitoring areas covering conductors with different potentials in the video image, and perform geometric correction on the area to eliminate perspective distortion; S3: Track analysis-based wire-threading behavior recognition: A lightweight deep learning model is used to analyze the corrected virtual monitoring area in real time to detect bird targets and form a motion trajectory through multi-target tracking technology; based on the continuous state transitions of the motion trajectory between multiple virtual monitoring areas, the bird's wire-threading behavior is determined. S4: Event Triggering and Data Upload: Once the threading behavior is detected, an alarm is triggered, and video is recorded and images are captured. The alarm information containing event metadata is then uploaded to the backend platform. S5: Data Management and Closed-Loop Optimization: The alarm data is stored, managed and analyzed on the backend platform, and the lightweight deep learning model is incrementally learned and optimized based on manually labeled data.

[0008] Furthermore, in step S2, the geometric correction is a perspective transformation. By solving the perspective transformation matrix M that adapts to the distribution characteristics of UHV conductors, the irregular monitoring area in the image is mapped into a regular rectangular processing area. Physical parameter constraints such as conductor spacing and tower structure are incorporated into the matrix solution process to improve the correction accuracy. The delineation of the virtual monitoring area is based on the Hough transform or LSD line segment detector automatically identifying the conductors, combined with the safety distance threshold of conductor gaps at different potentials to generate an initial suggested area, ensuring that the monitoring range accurately covers the risk area.

[0009] Furthermore, step S3 specifically includes: S31: A lightweight deep learning model with integrated attention mechanism is used for bird target detection. Considering that birds account for a small proportion and have varied shapes in the power transmission line scene, the loss function of the model adopts an improved FocalLoss, which can more effectively alleviate the class imbalance problem by dynamically adjusting the weight coefficients of easy and difficult samples. S32: A multi-target tracking algorithm based on depth appearance features is used to track detected bird targets, assigning a unique ID to each target and forming a motion trajectory; considering the characteristics of birds' fast flight speed and flexible trajectory, the association cost of the tracking algorithm is set as a weighted sum of Mahalanobis distance and appearance cosine distance, and the weight allocation ratio is optimized by using bird motion speed statistics to improve tracking stability; S33: The condition for determining the wire-threading behavior is that the movement trajectory of a tracked target must trigger state transitions in a continuous frame sequence, namely entering the first monitoring area, leaving the first monitoring area, and entering the second monitoring area, and the first and second monitoring areas represent wire gaps with different potentials; at the same time, the angle between the target's movement direction and the wire direction is calculated, and an adaptive threshold is set in combination with the layout rules of UHV line wires. When the angle is greater than the threshold, it is determined as a valid wire-threading event, so as to avoid misjudging birds flying along the wire.

[0010] Furthermore, in step S4, the recorded video uses a circular buffer technology, dynamically adjusting the buffer size according to the average duration of the bird threading event, and accurately saving the video before and after the event is triggered; the images in the alarm information are compressed using WebP format, the metadata is serialized using Protocol Buffers or MessagePack, and transmitted via the MQTT protocol, incorporating anti-interference optimization strategies for power communication scenarios during transmission.

[0011] Further, in step S5, the closed-loop optimization specifically involves: deploying a more accurate teacher model on the backend platform to review and label the alarm data uploaded from the frontend, forming a difficult sample library for bird-crossing scenarios (including complex scenario samples such as backlighting, flocking birds, and distant small targets); periodically using the difficult sample library, fine-tuning and optimizing the lightweight deep learning model at the frontend through improved knowledge distillation technology, focusing on transferring the teacher model's knowledge of bird morphology and flight trajectory recognition during the distillation process; and distributing the optimized model firmware to the frontend device via a wireless network to complete the model update and achieve continuous evolution of the system's recognition capabilities.

[0012] This application also provides a bird-threading intelligent monitoring system for implementing the aforementioned bird-threading intelligent monitoring method. The system adopts a cloud-edge-device collaborative architecture, including: a front-end acquisition and intelligent processing unit deployed at the tower site for acquiring video streams and performing localized intelligent analysis, behavior determination, and event triggering; a communication transmission unit for uploading alarm information to the back-end when an event is triggered, and receiving configuration instructions and model updates from the back-end; and a back-end platform and management unit for receiving, storing, and analyzing alarm data, and providing a human-computer interaction interface and system model optimization functions.

[0013] Furthermore, the front-end acquisition and intelligent processing unit includes: a high-definition wide-angle sensing module with ultra-high-definition resolution, wide dynamic range and infrared night vision function, which optimizes photosensitive parameters for the peak time of bird activity at dawn and dusk; and an edge intelligent analysis module, whose hardware carrier is a heterogeneous computing SoC integrating CPU and NPU, which has built-in runtime video analysis engine, multi-target tracker and behavior logic analyzer for running lightweight deep learning models, and optimizes the allocation of computing resources for bird threading recognition tasks.

[0014] Furthermore, the backend platform and management unit include: an event aggregation and access layer, which uses an MQTT Broker cluster to receive front-end data and supports concurrent access from a massive number of front-end devices; a data processing and storage layer, which uses a time-series database to store metadata, object storage to store media files, and uses Elasticsearch to build a metadata index, supporting fast retrieval by multiple dimensions such as bird species, crossing time, and route segment; an intelligent analysis and model management platform, used to run teacher models for review, manage difficult sample libraries, perform knowledge distillation training, and manage model versions and distribution; and a business application and visualization layer, which provides global situation display, alarm handling center, and statistical analysis report functions. The reports can display key data such as bird crossing frequency and high-incidence time periods in different areas.

[0015] Furthermore, the system also includes a power supply electronics unit, which is a solar power supply system, including solar panels, a power controller with MPPT function, and a deep-cycle battery. The power supply strategy is optimized for outdoor tower deployment scenarios to ensure continuous power supply under extreme weather conditions and provide stable power to the entire front-end equipment.

[0016] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0017] The beneficial effects of this invention are as follows: Targeting the specific risk scenario of birds crossing UHV transmission lines, this invention uses high-definition cameras deployed on the towers to collect video streams. The monitoring area is intelligently delineated and corrected based on the conductor distribution and potential characteristics within the image, solving the problem of poor adaptability of general monitoring area delineation. A lightweight deep learning model with an integrated attention mechanism is employed, and the loss function is optimized based on bird target characteristics to achieve high-precision real-time detection. Combined with multi-target tracking technology based on deep appearance features, the association cost weights are optimized for bird flight characteristics to form a stable trajectory. By analyzing the continuous state transitions of the trajectory between multiple monitoring areas with different potentials, and combining the conductor's direction angle to determine the crossing behavior, the false alarm rate is effectively reduced. After an event is triggered, video is automatically recorded, images are captured, and alarm information is uploaded with optimized compression, ensuring efficient and reliable data transmission. The backend platform, based on a difficult sample library of bird crossing scenarios and knowledge distillation technology, achieves incremental learning and closed-loop optimization of the model, improving the system's adaptability to complex scenarios. The system adopts a cloud-edge-device collaborative architecture, featuring low power consumption, high accuracy, and self-evolution capabilities. It is specifically adapted to the field deployment environment of UHV transmission lines and the need for bird crossing behavior identification, effectively solving the problems of high false alarm rate, limited resources, and poor scenario adaptability of existing technologies. It achieves accurate early warning and intelligent management of bird crossing behavior on UHV transmission lines. This invention combines bird crossing scenario-specific optimization strategies (such as dynamic parameter adjustment, a dedicated difficult sample library, and anti-interference transmission of power communication), featuring low power consumption, high accuracy, and self-evolution capabilities, adapting to the field tower deployment environment, and effectively realizing intelligent management and pre-emptive prevention of bird crossing behavior on UHV transmission lines. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only drawings of some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings. In addition, the drawings in the following description can be regarded as schematic diagrams and are not intended to limit the actual size of the product, the actual flow of the method, the actual timing of the signals, etc. involved in the embodiments of the present invention.

[0019] Figure 1 This is a block diagram of the overall structure of the system of the present invention; Figure 2 This is a flowchart of the method of the present invention; Figure 3 This is a schematic diagram of the virtual monitoring area delineation of the present invention (the area within the frame in the diagram is the delineated monitoring area). Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] like Figures 1 to 3 As shown, in a first aspect, an intelligent monitoring method for birds threading wires, provided by an embodiment of the present invention, is applied to ultra-high voltage transmission lines and includes the following steps: S1: Continuous acquisition and preprocessing of wide-angle video stream: Sensor selection: High-definition network cameras with wide dynamic range are preferred to cope with adverse lighting conditions such as backlight and strong light. The light sensitivity and shutter speed parameters of the camera are optimized for the peak bird activity periods at dawn and dusk.

[0022] Pre-processing line: 1) Spatial filtering: Gaussian filtering is first applied to the acquired raw video stream to reduce noise and minimize the interference of image noise on subsequent detection.

[0023] 2) Temporal Sampling: To balance computational load and real-time performance, the system employs an adaptive frame rate analysis strategy. By default, detection is performed at a lower frame rate (e.g., 5fps). When a moving target is detected entering the preset "wake-up zone" (covering the safety warning range around the wire), the system automatically switches to a high frame rate mode (e.g., 15fps) for precise tracking and analysis, ensuring the capture of the bird's instantaneous movement as it quickly traverses the wire.

[0024] S2: Intelligent delineation and perspective correction of monitoring area This step aims to establish a more accurate mapping relationship between image space and real physical space, adapting to the distribution and potential characteristics of UHV line conductors.

[0025] Camera calibration: During the deployment phase, it is recommended to use methods such as Zhang Zhengyou's calibration to calibrate the intrinsic and extrinsic parameters of the camera, obtain the distortion coefficients of the lens, and perform image distortion correction to provide a geometric basis for subsequent perspective transformation.

[0026] Adaptive Monitoring Area Generation: 1) Principle: Conductors typically appear as straight lines in images. Combining this with the characteristics of parallel arrangement and fixed spacing of UHV line conductors, the system automatically identifies conductors in video footage using Hough transform or LSD segment detectors; 2) Algorithm: Based on the detected conductor segments, the system automatically calculates their intersections or parallel areas. Combined with safety distance thresholds between conductors at different potentials, it generates an initial suggested monitoring area covering the air gaps between conductors. Maintenance personnel can then fine-tune this, greatly simplifying the deployment process.

[0027] Inverse application of perspective transformation: In subsequent steps, the trajectory points of the bird are calculated within the corrected regular region. x ′, y After ′), it can be obtained through the inverse matrix of the perspective matrix. M -1 The coordinates are then back-projected back to the original image coordinates to overlay intuitive tracking boxes and trajectory lines onto the original video, facilitating visualization, debugging, and result verification. The perspective transformation matrix is ​​involved. M In the solution process, physical constraints such as conductor spacing and tower structure are incorporated to improve the matching degree between the corrected area and the actual dangerous gap.

[0028] S3: Object Detection and Behavior Determination Based on Lightweight Deep Learning and Multi-Object Tracking S3.1 Optimized Lightweight Bird Target Detection Model Model Structure Innovation: Based on the standard YOLOv5s, targeted optimizations were made to address the characteristics of birds in power transmission line scenarios, which have a small proportion and varied forms: 1) Enhanced shallow features: To improve the detection of small targets, the feature maps of the shallower layers (containing more details and location information) in the backbone network are more fully integrated with the deep features through a lightweight variant of the path aggregation network, thereby enhancing the ability to identify small birds at a distance; 2) Integrated attention mechanism: An efficient channel attention module is embedded in each layer of the feature pyramid. This module has a very small computational cost but a significant effect, allowing the model to adaptively focus on the channels where bird features are more prominent.

[0029] Loss function optimization: An improved CIoU Loss is used to replace the original IoU Loss as the bounding box regression loss. In combination with the characteristics of the variable morphology of bird targets, the weight allocation of the aspect ratio consistency parameter is optimized so that it can more accurately consider the overlapping area, center point distance and aspect ratio at the same time.

[0030] S3.2 Trajectory Analysis Based on Improved Multi-Target Tracking Data association: An improved Deep SORT is adopted as a multi-target tracking framework. Taking into account the characteristics of birds' high flight speed and flexible trajectory, the motion model and association cost calculation method are optimized. Its core is the fusion association of Mahalanobis distance and appearance features.

[0031] 1) Motion Model: Kalman filtering is used to predict the target's position in the next frame. Combining this with the uniform linear motion characteristics of birds in flight, the state vector and prediction equation are optimized. The state vector is [ x , y , γ , h , x ˙, y ˙, γ ˙,h ˙], where (x,y) is the center point, γ is the aspect ratio, h is the height, and the dotted variables are their respective velocities. The prediction and update process follows the optimized Kalman filter equation, improving the prediction accuracy for birds' rapid directional changes of flight.

[0032] 2) Mahalanobis distance: measures the motion consistency between the predicted position and the measured detection box. Combined with the statistical distribution of bird movement speed, it optimizes the initial value and update strategy of the covariance matrix.

[0033] 3) Appearance features: A lightweight ReID network pre-trained on a small-scale bird dataset is used to extract a 128-dimensional feature vector for each detection box and calculate the minimum cosine distance between two appearance vectors to enhance the ability to distinguish the appearance differences of different bird individuals.

[0034] 4) Joint Association: The final association cost is the weighted sum of Mahalanobis distance and appearance distance. Combining the bird movement characteristics in the UHV line scenario, the weight coefficients are optimized through offline experiments to improve the association accuracy in complex scenarios.

[0035] Threading behavior logic determination: Two states are defined for each monitoring area: entry and exit. A complete "threading event" is strictly defined as follows: a bird target ID being stably tracked, whose trajectory triggers a state transition sequentially in a continuous frame sequence: "entering area A" -> "leaving area A" -> "entering area B" -> "leaving area B", where areas A and B represent dangerous gaps at different potentials. Simultaneously, the average motion direction vector d of the trajectory is calculated, ensuring that its angle θ with the guide wire direction (vector w) satisfies | θ |> θ threshold (For example, 60°), meaning that the direction of movement is mainly lateral crossing, rather than longitudinal following.

[0036] S4: Event-triggered, adaptive recording, and efficient transmission Intelligent recording strategy: Utilizing a circular buffer, the buffer size is dynamically adjusted based on the average duration of bird threading events (derived from extensive statistical data): B size =T pre ×FPS, where T pre This is the pre-recording duration.

[0037] Alarm information compression and serialization: To save bandwidth, key captured images are stored in WebP format, which is smaller in size than JPEG at similar image quality. Alarm metadata (time, location, trajectory points, etc.) is encoded using binary serialization formats such as Protocol Buffers or MessagePack, which are smaller in size and faster to parse than JSON.

[0038] S5: Data Storage, Management, and Closed-Loop Learning System Enhanced backend analysis: Videos uploaded to the backend are subjected to secondary analysis and verification using a larger and more accurate detection model (such as YOLOv8x). This model is specifically trained for bird-threading scenarios to ensure the accuracy of events ultimately stored in the database, and its judgment results serve as the gold standard.

[0039] Incremental learning and knowledge distillation: 1) The labeled data (correct / false positives) from operations and maintenance personnel form a difficult sample library; 2) This sample library is used periodically to perform knowledge distillation on the student model (lightweight front-end model) using the teacher model in the back-end; 3) The distilled and optimized new model firmware is then distributed to the front-end device via network security, completing a system-level intelligent evolution.

[0040] Secondly, to implement the aforementioned intelligent monitoring method, this invention also provides an intelligent monitoring system for birds threading wires. This system adopts a cloud-edge-device collaborative architecture and includes the following units: 1. Front-end data acquisition and intelligent processing unit: 1.1 High-definition wide-angle sensor module Core components: A webcam with ultra-high-definition resolution (such as 4K) and wide-angle distortion correction lens.

[0041] Key features: 1) Wide dynamic range: ensures clear and detailed images can be captured even in scenes with drastic changes in lighting, such as strong light and backlight; 2) Infrared night vision capability: equipped with a high-power infrared fill light to achieve 24-hour uninterrupted monitoring; 3) Industrial-grade protection: has an IP67 or higher protection rating, is resistant to high and low temperatures, dustproof, moistureproof, and adaptable to harsh outdoor environments.

[0042] 1.2 Edge Intelligent Analysis Module Hardware carrier: Integrated inside the camera or deployed in an adjacent waterproof and shockproof front-end processing box. The core is a system-on-a-chip with powerful AI inference capabilities. This chip typically adopts a heterogeneous computing architecture of CPU and NPU, where the NPU is dedicated to accelerating the inference calculation of deep learning models.

[0043] Software and Algorithm Core: 1) Lightweight Model Runtime: Supports and efficiently runs a deeply optimized lightweight bird detection model (such as YOLOv5s-ECA), and integrates improved mechanisms such as CBAM and Focal Loss; 2) Video Analysis Engine: Responsible for performing video stream decoding, preprocessing (Gaussian filtering), adaptive frame rate sampling, and perspective transformation calculation; 3) Multi-Target Tracker: Built-in tracking engine based on Deep SORT algorithm, maintaining the motion trajectories of multiple bird targets in real time; 4) Behavior Logic Analyzer: Based on preset thread-crossing behavior judgment rules (state machine), performs real-time analysis of tracking trajectories and ultimately decides whether to trigger alarm events; 5) Data Caching and Management: Built-in circular buffer for pre-recorded video, and temporarily stores event-related images, video clips, and metadata.

[0044] 2. Communication transmission unit: Core component: Industrial-grade wireless communication module.

[0045] Transmission Mode: Event-Driven Transmission: Under normal circumstances, only heartbeat packet connections are maintained. Once the front-end intelligent analysis module determines that a wiring event has occurred, it immediately uploads alarm information, key images, and short video clips asynchronously and concurrently via 4G / 5G network or wireless bridge.

[0046] Data transmission optimization: 1) Load compression: WebP format is used for efficient image compression, and ProtocolBuffers is used for binary serialization of alarm metadata, greatly reducing the amount of data transmitted; 2) Adaptive bitrate: The bitrate and resolution of video upload are dynamically adjusted according to the network signal strength (RSSI) to ensure that critical information can still be reported even when network conditions are poor; 3) Communication protocol: The lightweight MQTT protocol is used as the main communication protocol. It is based on the publish / subscribe model and has the advantages of low power consumption and low bandwidth consumption, making it very suitable for IoT scenarios.

[0047] 3. Backend Platform and Management Unit: 3.1 Event Aggregation and Access Layer Components: MQTT Broker cluster and API gateway; Function: Receive connections and data from all front-end devices with high concurrency, and perform initial verification and forwarding.

[0048] 3.2 Data Processing and Storage Layer Event data storage: A time-series database is used to efficiently store timestamped alarm metadata; object storage or a distributed file system is used to store associated image and video files.

[0049] Metadata Indexing: Using an Elasticsearch search engine cluster, full-text indexes are built for alarm information, supporting millisecond-level fast retrieval and aggregation analysis by multiple dimensions such as time, geographic location, tower ID, target quantity, and confidence level.

[0050] 3.3 Intelligent Analysis and Model Management Platform Teacher model verification: Deploy a larger and more accurate teacher detection model (such as YOLOv8x) to perform secondary analysis on the videos uploaded from the front end, generate more authoritative labels, and use them to evaluate the performance of the front-end student model and as ground truth for incremental learning.

[0051] Incremental Learning and Knowledge Distillation Engine: 1) Hard Sample Library: Stores false alarms and missed alarms annotated by maintenance personnel on the platform; 2) Model Training Pipeline: Regularly triggers training tasks, utilizes the hard sample library, and uses knowledge distillation technology to enable the lightweight front-end model to learn the knowledge of the teacher model; 3) Model Version Management and Distribution: Manages different versions of model firmware and distributes model updates in batches to front-end devices that have passed back-end verification via secure OTA.

[0052] 3.4 Business Application and Visualization Layer Global Situation Dashboard: Based on an electronic map, it displays the distribution, health status, and latest alarm events of all online devices in real time.

[0053] Alarm Handling Center: Provides functions such as listing alarm events, viewing details (including original video, track overlay video and metadata), confirmation, and false alarm marking.

[0054] Statistical analysis reports: Automatically generate reports such as bird activity frequency statistics, high-risk area rankings, and equipment online rates to provide data support for operation and maintenance decisions.

[0055] 4. Power supply electronic unit: Solar power supply system: 1) Solar panels: Monocrystalline silicon panels with appropriate power are calculated and selected according to local sunshine conditions; 2) Power controller: Equipped with MPPT function to maximize solar energy conversion efficiency; 3) Battery pack: High-capacity deep-cycle lithium iron phosphate batteries are used to ensure continuous operation of the system during continuous cloudy and rainy weather; 4) Intelligent power consumption management: The power controller communicates with the front-end processing unit to monitor system power consumption and automatically enters low-power sleep mode when the battery power is low, prioritizing the protection of critical communication functions.

[0056] Reference Figure 1 and Figure 2A smart bird tracking system has been deployed. First, maintenance personnel securely installed the front-end data acquisition and intelligent processing unit onto the crossarm of an ultra-high-voltage transmission tower. This unit's high-definition wide-angle sensor module features 4K resolution and WDR functionality. The lens has been precisely calibrated to ensure complete coverage of the conductor and key air gaps within the span, and the light sensitivity parameters have been optimized for bird activity periods.

[0057] After the system is powered on, the power supply unit (in this example, a solar-powered system) begins to supply power stably. Maintenance personnel remotely log into the device via the backend platform. In the platform's interface, the built-in automatic wire identification tool (based on Hough transform / LSD line segment detection) is first used to initially locate the wires in the image. Then, manual fine-tuning is performed to delineate two virtual monitoring areas corresponding to wires at different potentials. The platform then calculates and stores the perspective transformation matrix M that maps these two irregular areas to standard rectangles, and its inverse matrix M. -1 This is used for subsequent trajectory analysis and result visualization.

[0058] After deployment, the system entered routine monitoring. The camera captures data at 5fps, and the video stream, after Gaussian filtering preprocessing, is inferred in real-time only within two defined, perspective-corrected rectangular areas by a lightweight bird detection model (YOLOv5s-ECA) accelerated by a built-in NPU. This model integrates the ECA attention mechanism and uses FocalLoss to address class imbalance, demonstrating excellent detection capabilities for small, distant birds.

[0059] Suppose a large bird (such as an Oriental White Stork) flies close to a power transmission line. When it enters the frame, the system detects the target at a frame rate of 5 fps. Once the target enters the wake-up area, the system automatically increases the analysis frame rate to 15 fps. The Deep SORT multi-target tracker immediately assigns it a unique ID (e.g., ID=47) and predicts its position in the next frame using Kalman filtering. Combining appearance features with Mahalanobis distance, it performs stable tracking, forming a continuous motion trajectory T47.

[0060] Trajectory T47 was fed into the behavior logic analyzer in real time. The analyzer found that the sequence of motion trajectory points of target ID=47, in the perspective-corrected coordinate space, continuously triggered state transitions of entering Zone_A -> leaving Zone_A -> entering Zone_B. Simultaneously, the angle between its motion direction vector and the guide wire direction was calculated to be greater than 70 degrees, satisfying the condition of "lateral crossing". Based on this, the system determined that a high-confidence bird crossing event had occurred. Upon triggering an event, the system immediately executes a response: the circular buffer in the edge intelligent analysis module merges the video clips from 8 seconds before the event (pre-recorded) and 2 seconds after the event, saving them as a 10-second 1080P high-definition video; simultaneously, a high-definition WebP format image is captured the instant the bird is in the center of the monitoring area; finally, an alarm message containing metadata such as tower number, time, GPS coordinates, target ID, trigger area, and confidence level is generated. This message is serialized using ProtocolBuffers and efficiently uploaded to the backend platform and management unit via the 4G / 5G MQTT protocol.

[0061] Platform staff receive a real-time alert pop-up and can immediately click to view details, including images, videos, and a visual playback overlaid with bird tracks and the monitored area. Staff can then mark the event as a valid alert or a false alarm.

[0062] This annotation action triggered the system's closed-loop learning mechanism. The event data was stored in the hard sample library. The platform periodically initiates incremental learning and knowledge distillation processes: using data from the sample library, the powerful teacher model (YOLOv8x) guides the lightweight student model (front-end model) for retraining, optimizing its weights. After training, the new model firmware is deployed to the front-end device via a secure OTA channel, thereby enabling the entire system to continuously evolve its recognition accuracy in real-world environments, becoming increasingly intelligent with use.

[0063] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart monitoring method for birds threading wires, applied to ultra-high voltage transmission lines, characterized in that, Includes the following steps: S1: Video stream acquisition and preprocessing: Deploy cameras on ultra-high voltage transmission towers to continuously acquire video streams covering the conductor area of ​​the span, and preprocess the video streams; S2: Intelligent delineation and geometric correction of monitoring area: Delineate one or more virtual monitoring areas in the video frame that cover the dangerous air gaps between conductors with different potentials, and perform geometric correction on the area to eliminate perspective distortion; S3: Track analysis-based wire-threading behavior recognition: A lightweight deep learning model is used to analyze the corrected virtual monitoring area in real time to detect bird targets and form a motion trajectory through multi-target tracking technology; based on the continuous state transitions of the motion trajectory between multiple virtual monitoring areas, the bird's wire-threading behavior is determined. S4: Event Triggering and Data Upload: Once the threading behavior is detected, an alarm is triggered, and video is recorded and images are captured. The alarm information containing event metadata is then uploaded to the backend platform. S5: Data Management and Closed-Loop Optimization: The alarm data is stored, managed and analyzed on the backend platform, and the lightweight deep learning model is incrementally learned and optimized based on manually labeled data.

2. The intelligent monitoring and filming method for birds using threaded wires according to claim 1, characterized in that, In step S2, the geometric correction is a perspective transformation. By solving the perspective transformation matrix M that adapts to the distribution characteristics of UHV conductors, the irregular monitoring area in the image is mapped into a regular rectangular processing area. The solution process of the matrix M incorporates physical parameter constraints such as conductor spacing and tower structure. The delineation of the virtual monitoring area is based on the Hough transform or LSD line segment detector automatically identifying the conductors, combined with the safety distance threshold of conductor gaps at different potentials to generate an initial suggested area.

3. The intelligent monitoring method for birds using threading techniques according to claim 1, characterized in that: Step S3 specifically includes: S31: A lightweight deep learning model with integrated attention mechanism is used for bird target detection. The loss function of the model adopts an improved Focal Loss, which alleviates the class imbalance problem by dynamically adjusting the weight coefficients of easy and difficult samples. S32: A multi-target tracking algorithm based on deep appearance features is used to track detected bird targets, assigning a unique ID to each target and forming a movement trajectory; the association cost of the tracking algorithm is a weighted sum of Mahalanobis distance and appearance cosine distance, and the weights are optimized and allocated through bird movement speed statistics. S33: The condition for determining the wire-threading behavior is that the movement trajectory of a tracked target must trigger state transitions in a continuous frame sequence, namely entering the first monitoring area, leaving the first monitoring area, and entering the second monitoring area, and the first and second monitoring areas represent wire gaps with different potentials; at the same time, the angle between the target's movement direction and the wire direction is calculated, and an adaptive threshold is set based on the arrangement law of UHV line wires. When the angle is greater than the threshold, it is determined to be a valid wire-threading event.

4. The intelligent monitoring and filming method for birds using threaded wires according to claim 1, characterized in that, In step S4, the recorded video uses a circular buffer technology, and the buffer size is dynamically adjusted according to the average duration of the bird threading event; the images in the alarm information are compressed in WebP format, the metadata is serialized using Protocol Buffers or MessagePack, and transmitted through the MQTT protocol incorporating power communication anti-interference strategies.

5. The intelligent monitoring and filming method for birds using threaded wires according to claim 1, characterized in that, In step S5, the closed-loop optimization specifically involves: deploying a more accurate teacher model on the backend platform to review and label the alarm data uploaded from the frontend, forming a dedicated difficult sample library for birds flying against the light, flocks of birds converging, and long-distance small target scenarios; periodically using the difficult sample library, fine-tuning and optimizing the lightweight deep learning model at the frontend through improved knowledge distillation technology, with the distillation process focusing on transferring the teacher model's knowledge of bird morphology and flight trajectory recognition; and then updating the optimized model firmware by downloading it to the frontend device via a wireless network.

6. A bird threading intelligent monitoring system, used to implement the bird threading intelligent monitoring method according to any one of claims 1-5, characterized in that, The system adopts a cloud-edge-device collaborative architecture, including: a front-end acquisition and intelligent processing unit, deployed at the tower site, used to acquire video streams and perform localized intelligent analysis, behavior judgment, and event triggering; a communication transmission unit, used to upload alarm information to the back-end when an event is triggered, and to receive configuration instructions and model updates from the back-end; a back-end platform and management unit, used to receive, store, and analyze alarm data, and to provide a human-machine interface and system model optimization functions; and a power supply unit, which provides stable power to the entire front-end equipment.

7. The intelligent bird-threading monitoring system according to claim 6, characterized in that: The front-end acquisition and intelligent processing unit includes: a high-definition wide-angle sensing module with ultra-high-definition resolution, wide dynamic range and infrared night vision function, which optimizes photosensitive parameters and shutter speed for peak bird activity at dawn and dusk; and an edge intelligent analysis module, whose hardware carrier is a heterogeneous computing SoC integrating CPU and NPU, which has a built-in runtime video analysis engine, multi-target tracker and behavior logic analyzer for running lightweight deep learning models, and optimizes the allocation of computing resources for bird thread-and-wire recognition tasks.

8. The intelligent bird-threading monitoring system according to claim 6, characterized in that, The backend platform and management unit include: an event aggregation and access layer, which uses an MQTT Broker cluster to receive front-end data and supports concurrent access from a massive number of front-end devices; a data processing and storage layer, which uses a time-series database to store metadata, object storage to store media files, and Elasticsearch to build a metadata index, supporting fast retrieval by bird species, crossing time period, and route segment; an intelligent analysis and model management platform, used to run teacher models for review, manage a dedicated difficult sample library for bird crossing, perform knowledge distillation training, and manage and distribute model versions; and a business application and visualization layer, which provides statistical analysis reports on global situation display, alarm handling center, bird activity frequency statistics, and high-risk area ranking.

9. The intelligent bird-threading monitoring system according to claim 6, characterized in that, The communication transmission unit adopts an industrial-grade wireless communication module, and the transmission mode is event-driven transmission. Data transmission optimization includes: using WebP format to compress images, Protocol Buffers to serialize metadata, dynamically adjusting the video upload bitrate and resolution according to the network signal strength, and using the MQTT protocol that incorporates power communication anti-interference strategies. The power supply unit is a solar power system, including solar panels, a power controller with MPPT function and a deep-cycle battery. It optimizes the power supply strategy for outdoor tower deployment scenarios, has intelligent power consumption management function, and automatically enters a low-power sleep mode when the battery power is low, while prioritizing the protection of critical communication functions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images