Event camera based real-time aerial tracking of power lines

Abstract

The application provides a power transmission line aerial real-time tracking method and system based on an event camera, relates to the technical field of unmanned aerial vehicle visual perception and power inspection, and comprises the following steps: generating a linear candidate based on the time information and spatial neighborhood relationship of a plurality of events in an asynchronous event stream output by the event camera; performing joint geometric statistical analysis on the spatial coordinates and time stamps of the events in the linear candidate to determine power transmission line parameters; taking the linear candidate as a linear target and maintaining a tracking state comprising an active state and a hold state based on event density or event age, updating the power transmission line parameters in the active state, and reactivating the target in the hold state when a preset recovery condition is met; and outputting the power transmission line parameters based on the tracking state for use by flight control. The application realizes high-frequency, low-delay and robust real-time tracking of the power transmission line in a high-speed dynamic scene.

Description

Technical Field

[0001] This application relates to the fields of UAV visual perception and power line inspection technology, specifically to a method and system for real-time aerial tracking of power transmission lines based on event cameras. Background Technology

[0002] Currently, in the field of UAV inspection of power transmission lines, visual perception typically employs traditional frame cameras or LiDAR as sensors. However, traditional frame cameras, based on a fixed frame rate, are prone to severe motion blur in high-speed UAV flight scenarios, and there are temporal blind spots between adjacent frames, making it difficult to capture the instantaneous motion state of the power transmission line. Simultaneously, the large amount of redundant background data increases transmission bandwidth and computational burden, and is prone to overexposure or underexposure during drastic changes in lighting. While LiDAR solutions offer high accuracy, they suffer from issues such as large equipment weight, high power consumption, and complex data processing, making them unsuitable for long-endurance inspection tasks. Furthermore, existing line feature detection and tracking algorithms largely rely on regular inter-frame time interval assumptions or complete grayscale images, failing to directly process the asynchronous, sparse event streams output by event cameras. Simply applying traditional algorithms would negate the high temporal resolution advantage of event cameras, and the tracking target is easily lost when it is temporarily occluded or when event density fluctuates. Therefore, there is an urgent need for a real-time power transmission line tracking technology that can fully utilize the asynchronous characteristics of event cameras and achieve low latency and robustness in high-speed dynamic scenarios. Summary of the Invention

[0003] To address the problems of insufficient temporal resolution and severe motion blur of traditional visual sensors in high-speed inspection scenarios, as well as the poor tracking stability caused by the inability of existing algorithms to adapt to asynchronous event streams, this application proposes a real-time aerial tracking method and system for power transmission lines based on event cameras, which achieves high-frequency, low-latency, and robust real-time tracking of power transmission conductors and ground wires.

[0004] To achieve the above objectives, this application adopts the following technical solution: A real-time aerial tracking method for power transmission lines based on an event camera includes: generating linear candidates by performing directed chain growth on multiple events based on the temporal information and spatial neighborhood relationships of multiple events in an asynchronous event stream output by the event camera; performing joint geometric statistical analysis on the spatial coordinates and timestamps of the events in the linear candidates to determine the power transmission line parameters; using the linear candidates as linear targets and maintaining the tracking state of the linear targets based on event density or event age, the tracking state including an active state and a held state; updating the power transmission line parameters in the active state; and reactivating the linear targets in the held state to the active state when a preset recovery condition is met; and outputting the power transmission line parameters for flight control use.

[0005] As a preferred method, generating linear candidates by performing directed chain growth on multiple events includes: using an event time surface to record the time information of the nearest events with different polarities corresponding to each pixel position; searching for candidate events that meet the time connection conditions in the neighborhood based on the seed event along a preset direction, and connecting the candidate events that meet the time connection conditions to form an event chain; and generating linear candidates based on the event chain when the event chain meets a preset length condition.

[0006] Preferably, the directed chain growth of multiple events to generate linear candidates further includes: after selecting a seed event, determining whether a preset dense region condition is met based on the gradient characteristics of the local region where the seed event is located; if not, skipping the current seed event; if met, initiating the directed chain growth process; during the directed chain growth process, determining the priority of the current search direction from a preset direction priority sequence based on the spatial direction of the previous candidate event added to the event chain relative to the current search position, and searching for candidate events that meet the temporal connectivity condition in the neighborhood according to the priority; when no candidate event that meets the temporal connectivity condition is found in the first search range, expanding to the second search range to continue searching; terminating the directed chain growth when the length of the event chain reaches a preset upper limit, no candidate event that meets the temporal connectivity condition is found, or the image boundary is reached; performing two-dimensional geometric analysis on the spatial coordinates of events in the event chain that meets the preset length condition to generate a linear cluster as a linear candidate.

[0007] As a preferred embodiment, the joint geometric statistical analysis includes: constructing a joint sample set by combining the spatial coordinates and timestamps of events in the linear candidates, and calculating the distribution statistics based on the joint sample set; performing feature decomposition on the distribution statistics to extract the spatiotemporal normal vector and line direction of the linear candidates; and analyzing the position, direction, and velocity in the transmission line parameters based on the spatiotemporal normal vector and line direction.

[0008] Preferably, the position, direction, and velocity of the transmission line parameters are analyzed based on the spatiotemporal normal vector and the line direction, including: determining the eigenvector corresponding to the smallest eigenvalue in the eigenvalue decomposition as the spatiotemporal normal vector, and determining the line direction based on the projection direction of the spatiotemporal normal vector in the image plane; determining the length of the transmission line based on the distribution statistics of events along the line direction in the joint sample set, and determining the width of the transmission line based on the perpendicular distance statistics from the events to the line direction; determining the apparent motion velocity based on the correlation between the spatial dimension and the time dimension in the spatiotemporal normal vector; and performing time compensation on the midpoint of the linear target based on the apparent motion velocity to eliminate the positional deviation caused by the event time delay.

[0009] Preferably, maintaining the tracking status of linear targets based on event density or event age includes: when the event density of the linear target is lower than a preset density condition or the latest event age exceeds a preset age condition, switching the linear target from an active state to a hold state; in the hold state, sampling the event density along the direction of the transmission line parameters, and when the sampling result meets a preset recovery condition, reactivating the linear target from the hold state to an active state; when the duration or number of times the linear target is in the hold state exceeds a preset hold condition, deleting the linear target.

[0010] Preferably, the tracking state also includes an initialization state. Maintaining the tracking state of the linear target based on event density or event age further includes: in the initialization state, projecting the events of the linear target onto the direction of the transmission line parameters and performing a connectivity length check, removing abnormal events that do not belong to the main connected part, and switching the linear target from the initialization state to the active state when the connectivity length check meets the preset connectivity conditions; in the active state, periodically updating the transmission line parameters and performing time compensation on the midpoint of the linear target; in the hold state, sampling the event-dense region along the direction of the transmission line parameters using a linear rasterization method; when the first linear target and the second linear target meet the preset merging conditions, merging the first linear target and the second linear target and recalculating the transmission line parameters, the preset merging conditions include the direction angle condition, the distance condition from the midpoint to the inferred line condition, and the midpoint spacing condition.

[0011] Preferably, before selecting seed events, the method further includes preprocessing the asynchronous event stream. The preprocessing includes: dynamically adjusting the sampling density of adaptive time sampling based on the comparison result between the current event rate and the target event rate; filtering out first time interval events that do not meet the preset refractory period condition based on the time interval of the closest events of the same polarity at the same pixel location, and filtering out second time interval events that do not meet the preset refractory period condition based on the time interval of the closest events of opposite polarity; and filtering out background noise events based on the number of events of the same polarity in the neighborhood of the current event.

[0012] Furthermore, this application also provides a real-time aerial tracking system for power lines based on an event camera, comprising: an event camera for outputting an asynchronous event stream; a memory for storing computer program instructions; and a processor coupled to the event camera and the memory for executing the computer program instructions to perform the following operations: generating linear candidates by performing directed chain growth on multiple events based on the temporal information and spatial neighborhood relationships of multiple events in the asynchronous event stream; performing joint geometric statistical analysis on the spatial coordinates and timestamps of the events in the linear candidates to determine the power line parameters; using the linear candidates as linear targets and maintaining the tracking state of the linear targets based on event density or event age, the tracking state including an active state and a held state; updating the power line parameters in the active state and reactivating the linear targets in the held state to the active state when a preset recovery condition is met; outputting the power line parameters; and a flight control interface coupled to the processor for outputting the power line parameters or flight control information generated based on the power line parameters.

[0013] Preferably, the system is applied to a drone, which includes an airframe, a power unit, and a flight controller. An event camera is mounted on the airframe. The processor is also used to generate attitude control commands, speed control commands, or heading control commands based on the power line parameters. The flight controller is used to control the power unit according to the attitude control commands, speed control commands, or heading control commands, so that the power line remains within a preset tracking relationship.

[0014] Beneficial effects: The real-time aerial tracking method and system for power transmission lines based on event cameras provided in this application constructs linear candidates by directly utilizing the temporal information and spatial neighborhood relationships of events in the asynchronous event stream to perform directed chain growth. This avoids the traditional path of accumulating events into frame images for processing, thus preserving the microsecond-level temporal resolution of the event camera and significantly reducing the computational load of redundant data. By performing joint geometric statistical analysis on the spatial coordinates and timestamps of events in the linear candidates, multidimensional power transmission line parameters, including position, direction, and velocity, can be directly parsed from the asynchronous data, eliminating the dependence on fixed frame rates and inter-frame optical flow, and achieving high-frequency, low-latency motion estimation. At the same time, by introducing a tracking state maintenance mechanism that includes active and hold states, the target is not immediately discarded when event density fluctuates or the target is briefly occluded, but instead enters a hold state and is reactivated when conditions are met. This effectively solves the tracking loss problem caused by data sparsity in high-speed dynamic scenarios, ensuring the continuity and stability of flight control.

[0015] This application utilizes an event time surface to record the time information of the most recent events and combines it with a preset directional priority sequence to perform directed chain growth, transforming the search process of linear structures into a time-dependent directional connection process. Compared with full graph traversal or exhaustive nearest neighbor search, this significantly reduces the time complexity of the algorithm. In addition, with the dense region gating mechanism, low-texture invalid regions are pre-removed before chain growth, further reducing the consumption of invalid computing resources, enabling the system to run in real time on an embedded platform.

[0016] This application constructs a joint sample set of spatial coordinates and timestamps and performs eigenvalue decomposition. It directly derives the apparent motion velocity by utilizing the spatiotemporal normal vector corresponding to the minimum eigenvalue and the correlation between its components, without the need for additional optical flow calculations or Kalman filter predictions, thus simplifying the motion parameter estimation process. Combined with a midpoint time compensation mechanism, the spatial position of the linear target is backtracked and corrected using the resolved velocity vector, eliminating the spatial error introduced by the time delay between event triggering and processing, and improving tracking accuracy.

[0017] This application manages the lifecycle of linear targets using a three-state finite state machine. In particular, by using linear rasterization to sample the event density along the line direction in the hold state, the existence assumption of the target can be maintained when the target is obscured by tree branches or the event generation rate drops sharply. Tracking can be quickly resumed through a reactivation mechanism, avoiding frequent creation and destruction of targets. At the same time, multiple merging conditions based on direction, distance and spacing ensure the correct association of multiple broken line segments, enhancing the integrity of the transmission line topology and tracking robustness in complex scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart of an embodiment of the real-time aerial tracking method for power transmission lines based on an event camera. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0021] Example 1 like Figure 1 As shown, this embodiment provides a real-time aerial tracking method for power transmission lines based on an event camera. This method utilizes the high temporal resolution of the event camera to directly parse the spatial and motion parameters of the power transmission line from the asynchronous data stream, thereby meeting the real-time requirements of UAV flight control. The method includes the following steps: Step S1 involves performing directed chain growth on multiple events in the asynchronous event stream output by the event camera, based on their temporal information and spatial neighborhood relationships, to generate linear candidates. Specifically, the core of this step is to abandon the traditional visual paradigm of accumulating events into grayscale images before processing, and instead operate directly on the raw data level of the asynchronous event stream. Each event output by the event camera carries a precise microsecond-level timestamp; accumulating these into frames would inevitably lose this crucial temporal dimension information and introduce motion blur. The directed chain growth in this embodiment is essentially a directional aggregation mechanism based on spatiotemporal correlation. It utilizes the temporal and spatial continuity of events to search and connect along potential line extension directions, thereby filtering and organizing discrete noisy events into geometrically meaningful linear structures. This processing method not only fully preserves the sensor's high-frequency dynamic response capability but also avoids computational redundancy caused by full-image traversal through directional search, providing a clean and temporally rich structured data foundation for subsequent processing. It should be understood that the specific implementation of directed chain growth here can be neighborhood search based on lookup tables, gradient-based path tracing, or other algorithms that can reflect the spatiotemporal directional connectivity characteristics. As long as they utilize the temporal information of events and spatial neighborhood relationships to construct linear candidates, they all fall within the protection scope of this application.

[0022] Step S2 involves performing joint geometrical statistical analysis on the spatial coordinates and timestamps of events in the linear candidate array to determine the transmission line parameters. Specifically, this joint geometrical statistical analysis differs from the traditional step-by-step processing method of fitting a two-dimensional line and then calculating the difference. In this embodiment, each event in the linear candidate array is considered a sample point in three-dimensional spatiotemporal space, with its coordinates consisting of pixel position (x, y) and timestamp. Together, they constitute a cohesive whole. By performing a comprehensive statistical analysis of the distribution characteristics of these three-dimensional sample points, the position, direction, and apparent velocity of the transmission lines can be directly calculated from the geometric structure of the data. The advantage of this approach is that it treats motion information as an inherent property of the data structure itself, rather than a change between two frames, thus completely eliminating the dependence on a fixed frame rate or uniform time interval. Even when the high-speed maneuvering of the drone causes drastic fluctuations in the event generation rate, this analysis method can still provide accurate parameter estimates based on the current local spatiotemporal distribution, ensuring the continuity and robustness of motion perception.

[0023] Step S3: Linear candidates are used as linear targets, and the tracking state of the linear targets is maintained based on event density or event age. The tracking state includes an active state and a held state. In the active state, the transmission line parameters are updated, and when a preset recovery condition is met, the linear targets in the held state are reactivated to the active state. Specifically, since the event camera only triggers events when the scene brightness changes, the density of effective events may drop instantaneously or even be interrupted when the transmission line is briefly obscured by tree branches, the background texture is monotonous, or the lighting conditions change abruptly. If the existence of the target is determined solely based on the data existence at the current moment, it is very easy to lose tracking. The held state introduced in this embodiment is essentially a target hypothesis maintenance mechanism. When the event density is detected to be lower than expected or the latest event is too old, the system does not immediately delete the target, but switches it to the held state. In this state, the system still retains the predicted trajectory and historical features of the target and continues to perform verification sampling on the predicted path. Once the sampling results confirm that the target has reappeared or the data quality has recovered, the target is immediately reactivated. This state management mechanism gives the system the ability to remember data in the case of sparse or missing data, effectively solving the inherent intermittency problem of asynchronous event streams and ensuring the smoothness and stability of flight control commands.

[0024] Step S4: Output the power line parameters for flight control. Specifically, the power line parameters obtained after the aforementioned processing include high-precision relative position, attitude angle, and relative velocity information, which are encapsulated into a real-time data stream conforming to the flight control system interface specification. Since the entire processing chain is completed under event-driven conditions, without waiting for fixed exposure periods or frame synchronization signals, the output parameters have extremely low end-to-end latency, which can meet the closed-loop control requirements of UAVs performing close-range autonomous following or fine-grained inspection of power lines in complex environments.

[0025] Example 2 Before selecting seed events, the method also includes preprocessing the asynchronous event stream. The preprocessing includes: dynamically adjusting the sampling density of adaptive time sampling based on the comparison result between the current event rate and the target event rate; filtering out first time interval events that do not meet the preset refractory period condition based on the time interval of the closest events of the same polarity at the same pixel location, and filtering out second time interval events that do not meet the preset refractory period condition based on the time interval of the closest events of opposite polarity; and filtering out background noise events based on the number of events of the same polarity in the neighborhood of the current event.

[0026] Specifically, the preprocessing steps described above aim to address the coexistence of high noise and high dynamic range in the raw data of event cameras. Regarding adaptive temporal sampling, this embodiment employs a three-segment weighted moving average strategy to evaluate the current event rate. This involves dividing the historical time window into three sub-intervals: recent, medium, and long term, assigning non-uniform weight coefficients to each for weighted averaging. The recent sub-interval has the highest weight, while the long term sub-interval has the lowest. This non-uniform weight allocation mechanism makes the system more sensitive to recent traffic spikes while retaining a smooth perception of long-term trends, thus enabling real-time adjustment of the sampling rate based on the dynamic changes in the actual scene. For example, when the event rate surges due to high-speed drone maneuvers, the system automatically reduces the sampling density to maintain stable processing load; while during hovering observations, the sampling rate is increased to ensure detail capture. Using only a simple arithmetic average would be insufficient to balance response speed and stability. Regarding dual filtering, refractory period filtering is implemented by maintaining a dual-channel activation event surface matrix, which records the timestamps of the most recent events of both positive and negative polarities for each pixel location. For newly arriving events, if the time interval between them and the nearest event of the same polarity is less than a first threshold, or the time interval between them and the nearest event of opposite polarity is less than a second threshold, they are identified as sensor thermal noise or reset artifacts and filtered out. Experiments show that if the first threshold is set too high, it can lead to the false deletion of real edge events, causing line breaks; if it is set too low, it cannot effectively suppress high-frequency noise. Neighborhood density filtering further verifies the effectiveness of events in the spatiotemporal dimension. Specifically, a spatial window is constructed with the current event as the center, and the number of events of the same polarity within this window that are within a set time difference from the current event is counted. If the number is less than the set number, it is considered isolated background noise. This mechanism utilizes the physical characteristic that transmission line events inevitably exhibit clustered distribution in spatiotemporal space, effectively eliminating random shot noise.

[0027] After preprocessing, the process involves using an event time surface (SAE) to record the most recent event time information of different polarities for each pixel location. Specifically, the event time surface (SAE) is a highly efficient temporal memory data structure where each element stores the timestamp of the most recent triggered event for the corresponding pixel. The SAE is not only used for the aforementioned filtering operations but also serves as the temporal reference for subsequent chain growth. Unlike traditional image caching, SAE only retains the latest temporal state, eliminating the need to store complete historical frame sequences, thus significantly reducing memory usage and access latency.

[0028] Next, the process involves searching for candidate events that meet the temporal connection condition within the neighborhood of the seed event along a preset direction, and then chaining these candidate events together to form an event chain. Specifically, the temporal connection condition is the core constraint that distinguishes an effective line structure from a cluster of random noise. In this embodiment, a candidate event is considered to meet the temporal connection condition only when the timestamp difference between the candidate event and the current chain tail event is less than a preset maximum time interval. This threshold setting comprehensively considers the speed of the transmission line's movement and the response frequency of the event camera, ensuring that the chain growth extends along the actual trajectory of the object's movement, rather than merely being spatially adjacent. It should be understood that although this embodiment uses a fixed time threshold, in other embodiments, this threshold can be adaptively adjusted based on the local event density or estimated speed of movement to adapt to different flight conditions.

[0029] Furthermore, to improve the efficiency and accuracy of chain growth, after selecting a seed event, the gradient characteristics of the local region where the seed event is located are used to determine whether a preset dense region condition is met. If not, the current seed event is skipped; otherwise, the directed chain growth process is initiated. Specifically, this step is called dense region gating. The system calculates the average Sobel gradient magnitude of the time surface within the pixel region surrounding the seed point. Only when this average value is greater than a preset gradient threshold is the region considered to have significant potential for linear structures, and chain growth is allowed to begin. Since most areas in the power transmission line inspection scenario consist of background sky or vegetation textures, lacking effective linear features, the gating mechanism can filter out a large number of invalid seed points in advance, avoiding wasting computational resources in meaningless areas.

[0030] During the directed chain growth process, based on the spatial orientation of the previous candidate event added to the event chain relative to the current search position, the priority of the current search direction is determined from a preset direction priority sequence, and candidate events that satisfy the temporal connection condition are searched in the neighborhood according to their priority. Specifically, this embodiment uses a static lookup table (chain_search_x) to achieve fast indexing of direction priorities. The lookup table predefines the most likely next search direction sequence under different relative positions, so that the algorithm does not need to perform complex vector operations or full neighborhood traversal at runtime, but only needs to look up the table to obtain the optimal search path. When no candidate event that satisfies the temporal connection condition is found in the first search range, the system automatically expands to the second search range to continue searching. This hierarchical search strategy not only ensures fast tracking on straight line segments, but also gives the algorithm robustness to slight bends, breaks, or jitters in the line.

[0031] The directed chain growth terminates when any of the following conditions are met: the length of the event chain reaches a preset upper limit; no candidate events satisfying the temporal connectivity condition are found within the current and extended search range; or the search reaches the image boundary. Finally, two-dimensional geometric analysis is performed on the spatial coordinates of events in the event chain that meets the preset length condition, generating linear clusters as linear candidates. Specifically, the two-dimensional geometric analysis typically employs Principal Component Analysis (PCA) to calculate the covariance matrix of the event coordinates and perform eigenvalue decomposition, extracting the eigenvector corresponding to the smallest eigenvalue as the line's normal vector, thereby determining the line's direction and center position. Only when the event chain is sufficiently long and exhibits a clear linear distribution characteristic will it be confirmed as a valid linear cluster, further excluding short pseudo-line segments formed by noise. It should be understood that although this embodiment uses PCA as an example, other geometric methods capable of achieving two-dimensional linear fitting or clustering analysis, such as least squares method and RANSAC line fitting, as long as they extract linear features based on the spatial distribution of the event chain, fall within the scope of protection of this application.

[0032] Example 3 In this embodiment, the three-dimensional spatiotemporal geometric analysis and parameter estimation, as well as state machine tracking and output, are described in detail. Joint geometric statistical analysis includes: constructing a joint sample set from the spatial coordinates and timestamps of events in the linear candidate, and calculating distribution statistics based on the joint sample set; performing feature decomposition on the distribution statistics to extract the spatiotemporal normal vector and line direction of the linear candidate; and resolving the position, direction, and velocity in the transmission line parameters based on the spatiotemporal normal vector and line direction. Specifically, this step is crucial for elevating the linear candidate from a two-dimensional image plane to a three-dimensional spatiotemporal domain for processing. The system treats each event in the linear candidate as a sample point in three-dimensional space. ,in For pixel coordinates, The timestamps are in the microsecond range, and T represents the transpose matrix. A 3×3 covariance matrix is ​​calculated based on these sample points. ,in For the number of events, The mean of the samples is used. Self-adjoint eigenvalue decomposition is performed on this covariance matrix to obtain three eigenvalues ​​arranged in ascending order and their corresponding eigenvectors. This processing method utilizes the inherent temporal dimension information of event camera data, directly transforming the planar structure formed by the transmission line in the spatiotemporal domain into computable geometric parameters. This avoids the step-by-step processing mode of traditional methods, which first performs spatial line fitting and then estimates the velocity through inter-frame difference, thus eliminating the dependence on a fixed frame rate and significantly improving the real-time performance and accuracy of motion parameter estimation.

[0033] Furthermore, the position, direction, and velocity of the transmission line parameters are analyzed based on the spatiotemporal normal vector and line direction, including: determining the eigenvector corresponding to the minimum eigenvalue in the eigenvalue decomposition as the spatiotemporal normal vector, and determining the line direction based on the projection direction of the spatiotemporal normal vector in the image plane; determining the length of the transmission line based on the distribution statistics of events along the line direction in the joint sample set, and determining the width of the transmission line based on the perpendicular distance statistics from events to the line direction; determining the apparent motion velocity based on the correlation between the spatial and temporal dimensions in the spatiotemporal normal vector; and performing time compensation on the midpoint of the linear target based on the apparent motion velocity to eliminate the positional deviation caused by event time delay. Specifically, the eigenvector corresponding to the minimum eigenvalue... This represents the direction in which the event is thinnestly distributed in the spatiotemporal domain, i.e., the three-dimensional spatiotemporal normal vector of the transmission line, where nx represents the x-pixel dimension component of the image, ny represents the y-pixel dimension component of the image, and nz represents time. Dimensional components. The projection of the line direction onto the image plane is called a dimension component. The length of the transmission line The variance of the event distribution along the line direction can be used. It is derived that, for example This is based on the assumption that events are uniformly distributed along the line; width This is determined by calculating the average perpendicular distance from the event to the fitted line and multiplying it by a coefficient. More crucially, the apparent velocity is analyzed, its physical essence being the slope of the spacetime plane. The system constructs the spacetime plane parameter vector through normal vector components. Then the apparent velocity , Represents the spacetime plane components, This represents the normal intensity of the transmission line on the image plane. This formula directly maps spatiotemporal geometric features to motion velocity in pixels per second, without requiring any optical flow iteration or Kalman filter prediction. Furthermore, considering the slight time delay between event triggering and processor reading, the system uses the resolved velocity vector to perform time compensation on the midpoint of the linear target; the compensation formula is as follows: ,in and These are the average values ​​of the event coordinates and time, respectively. This compensation mechanism backtracks and corrects the spatial position of the line to the current processing moment, effectively eliminating spatial errors caused by asynchronous transmission and processing pipelines, and ensuring the accuracy of flight control commands.

[0034] After obtaining accurate line parameters, the system also needs to address data fluctuations in complex environments. Maintaining the tracking status of linear targets based on event density or event age includes: when the event density of the linear target is lower than a preset density condition or the latest event age exceeds a preset age condition, the linear target is switched from an active state to a held state; in the held state, the event density is sampled along the direction of the transmission line parameters, and when the sampling result meets a preset recovery condition, the linear target is reactivated from the held state to an active state; when the duration or number of times the linear target is in the held state exceeds a preset held condition, the linear target is deleted. Specifically, the data generation of the event camera depends on changes in scene brightness. When the transmission line is briefly obscured by tree branches, the background texture is monotonous, or the lighting changes abruptly, the effective event density will drop instantaneously. If the existence of the target is judged solely based on the current data existence, it is very easy to cause tracking loss. The held state (HIBERNATE) introduced in this embodiment is essentially a target hypothesis maintenance mechanism. For example, when the detected new event density is below 0.15 ev / (px²·ms) or the latest event age exceeds 25 ms, the system does not immediately delete the target but enters a hold state. Here, 0.15 ev / (px²·ms) represents an average of 0.15 light intensity change events per square pixel per millisecond. In this state, the system retains the target's predicted trajectory and continuously performs verification sampling along the predicted path. Once sampling confirms the target's reappearance, it is immediately reactivated. Only when the hold duration exceeds a set duration or the number of hold events exceeds a set number is the target determined to have truly disappeared and deleted.

[0035] To further enhance tracking robustness and topological integrity, the tracking state also includes an initialization state. Maintaining the tracking state of linear targets based on event density or event age further includes: In the initialization state, projecting the events of the linear target onto the direction of the transmission line parameters and performing a connectivity length check, removing abnormal events that do not belong to the main connected part; when the connectivity length check meets the preset connectivity conditions, switching the linear target from the initialization state to the active state; In the active state, periodically updating the transmission line parameters and performing time compensation on the midpoint of the linear target; In the hold state, sampling the event-dense region along the direction of the transmission line parameters using linear rasterization; When the first linear target and the second linear target meet the preset merging conditions, merging the first linear target and the second linear target and recalculating the transmission line parameters. The preset merging conditions include the direction angle condition, the distance condition from the midpoint to the inferred line condition, and the midpoint spacing condition. Specifically, newly created line candidates first enter the initialization state (INIT). During this stage, the system projects all events onto the line direction and performs histogram statistics to find the longest connected component. Only when the connected component length is greater than 200 pixels is the system upgraded to the active state and assigned a permanent ID. This effectively filters out short pseudo-line segments accidentally formed by noise. In the active state, the system periodically updates the 3D PCA parameters every 15ms and performs the aforementioned time compensation. In the hold state, the sampling strategy is specified as scanning along the line direction using the Bresenham linear rasterization algorithm. This method better matches the actual direction of the line than a simple rectangular window search, improving the success rate of re-detection. Furthermore, for cases where transmission lines may be segmented, the system defines strict three-line merging conditions: for example, the angle between the two line directions is less than 10 degrees, the distance from the midpoint of any line to the inferred line of the other line is less than 40 pixels, and the distance between the midpoints of the two lines is less than 0.8 times the average line length. Merging is only performed when all three conditions are met, and the 3D PCA parameters are recalculated. This multi-constraint mechanism avoids incorrectly merging background interference lines onto the transmission line, ensuring the correctness of the transmission line topology and the stability of the tracking results in complex inspection scenarios.

[0036] Example 4 This embodiment provides a real-time aerial tracking system for power transmission lines based on an event camera. This system materializes the method flow described in the previous embodiments into a specific hardware architecture and control loop. The system includes an event camera, a memory, a processor, and a flight control interface. The event camera outputs an asynchronous event stream, possessing microsecond-level time resolution and high dynamic range characteristics, capable of adapting to the demands of drastic changes in lighting and high-speed movement in inspection scenarios. The memory stores computer program instructions, which include algorithmic logic for implementing functions such as adaptive sampling, directed chain growth, spatiotemporal geometric analysis, and state machine tracking. The processor, coupled with the event camera and memory, executes computer program instructions to perform the following operations: Based on the temporal information and spatial neighborhood relationships of multiple events in an asynchronous event stream, it performs directed chain growth on multiple events to generate linear candidates; it performs joint geometric statistical analysis on the spatial coordinates and timestamps of events in the linear candidates to determine transmission line parameters; it treats the linear candidates as linear targets and maintains the tracking state of the linear targets based on event density or event age, including active and held states; it updates the transmission line parameters in the active state and reactivates linear targets in the held state to the active state when preset recovery conditions are met; and it outputs the transmission line parameters. The flight control interface, coupled with the processor, outputs the transmission line parameters or flight control information generated based on the transmission line parameters.

[0037] Specifically, at the hardware data interaction level, to ensure the real-time performance of system processing, data exchange between the processor and the event camera, as well as between various functional modules within the processor, preferably employs the POSIX shared memory mechanism to achieve zero-copy transmission. Traditional data transmission methods often involve multiple memory copies between kernel mode and user mode, which introduces significant latency in high-frequency event stream processing scenarios. Through the shared memory mechanism, the raw event data collected by the event camera can be directly read by the perception algorithm thread on the processor, and the processed line parameters can also be directly accessed by the control algorithm thread, thereby minimizing end-to-end latency. It should be understood that although this embodiment uses POSIX shared memory as an example, in other implementations, DMA direct memory access, circular buffers, or other efficient data transmission mechanisms can be used depending on the characteristics of the hardware platform. Any data flow that achieves low latency falls within the scope of this application.

[0038] Furthermore, this system is applied to unmanned aerial vehicles (UAVs), which include an airframe, power system, and flight controller, with an event camera mounted on the airframe. In practical applications, the processor not only runs the perception algorithm but also transforms the perception results into control decisions. Specifically, the processor generates attitude control commands, speed control commands, or heading control commands based on the power line parameters. This transformation process is not a simple numerical pass-through but rather maps geometric deviations in the visual space to control quantities in the airframe's dynamic space based on a preset control law. For example, when the power line deviates from the preset tracking center in the image plane, the processor generates a lateral speed command or roll angle command based on the horizontal deviation to correct the UAV's lateral position; when the extension direction of the power line forms an angle with the longitudinal axis of the airframe, a yaw rate command is generated to adjust the nose direction; when the resolved apparent motion velocity indicates a change in the relative distance between the UAV and the power line, a longitudinal speed command or throttle command is generated to maintain a safe distance. The flight controller controls the power system according to the attitude control commands, speed control commands, or heading control commands to keep the power line within the preset tracking relationship. The preset tracking relationship here can be to keep the power line in the center of the field of view, maintain a fixed lateral offset for easy side inspection, or maintain a constant following distance. The power components typically include actuators such as motors, ESCs, and propellers. The flight controller achieves precise control of the UAV's attitude and trajectory by adjusting the rotational speed of each power component.

[0039] Based on actual UAV inspection scenarios, the system provided in this embodiment demonstrates excellent robustness under various complex conditions. During high-speed flight inspections, thanks to the event camera's lack of motion blur and the processor's direct processing capability for asynchronous event streams, the system can continuously and stably output high-precision line parameters, avoiding the feature loss problem caused by excessive exposure time in traditional frame cameras. In moments of drastic lighting changes due to entering or exiting tunnels or cloud cover, the event camera's high dynamic range ensures the normal triggering of edge events. Combined with the adaptive sampling mechanism of the preprocessing module, the system will not interrupt tracking due to overexposure or underexposure. More importantly, when power lines are briefly obstructed by obstacles such as tree branches or tower materials, the state machine tracking mechanism can switch the linear target to a hold state, maintaining the target hypothesis using a historical motion model, and quickly reactivating it after the obstruction ends. This avoids sudden changes in flight control commands or target loss, ensuring the safety and continuity of the UAV's autonomous inspection mission. This system-level design, encompassing sensor selection, algorithm architecture, and control closed-loop, makes the technical solution presented in this application not merely a data processing method, but a fully engineerable intelligent sensing and control solution for unmanned aerial vehicles.

[0040] Example 5 This embodiment provides a specific application example based on the system provided in Embodiment 4, illustrating the complete workflow of the power transmission line tracking method in UAV power transmission line inspection tasks. The system collects event streams through an event camera, processes them through a processor, and outputs power transmission line parameters for flight control.

[0041] In this application example, the drone takes off according to a pre-planned inspection route and flies towards the target power line. During the flight, the event camera continuously outputs an asynchronous event stream. When the drone is far from the power line, the power line has not yet entered the field of view of the event camera, or is only vaguely visible in the distance; at this time, the event stream is mainly background events. As the drone gradually approaches, the relative motion between the power line and the event camera causes changes in the brightness of the line edge, triggering a large number of events with obvious directional extension characteristics.

[0042] After the preprocessing module performs adaptive sampling and dual filtering on the event stream, the directed chain growth mechanism concatenates events with spatiotemporal continuity into an event chain along the extension direction of the transmission line. When the event chain meets a preset length condition, linear candidates are generated. The newly generated linear candidates first enter an initialization state. The system projects their events onto the direction of the transmission line parameters and performs a connectivity length check, removing abnormal events that do not belong to the main connected portion. When the connectivity length check meets the preset connectivity condition, the linear candidate switches from the initialization state to the active state, completing the tracking and establishment of the transmission line. Afterward, the processor continuously outputs stable transmission line parameters.

[0043] After entering the line-tracking flight phase, the system updates the power line parameters every 15 milliseconds and performs time compensation for the midpoint of the linear target to eliminate positional deviations caused by event time delays. The processor generates attitude control, velocity control, or heading control commands based on the power line parameters. The flight controller then controls the propulsion components according to these commands to keep the power line within a preset tracking relationship. In this application example, the preset tracking relationship is to keep the power line in the center of the event camera's field of view. When the UAV's heading deviates from the direction of the power line's extension, the processor generates heading control commands to correct the UAV's orientation; when the power line deviates from the center of the field of view in the image plane, a lateral velocity command is generated to correct the UAV's lateral position, thus ensuring the UAV continues to fly along the power line.

[0044] When the drone flies to a section of the power line that is obscured by obstacles such as tree branches and tower materials, the number of events triggered by the power line decreases sharply, and the event density of the linear target falls below the preset density condition. The system switches the linear target from an active state to a hold state, retaining the historical motion information of the power line and sampling the event density along the predicted direction of the power line parameters. Since the obstruction is usually temporary, after the drone passes through the obstructed area, the power line reappears in the field of view of the event camera. The sampling results meet the preset recovery conditions, and the system reactivates the linear target from the hold state to the active state, seamlessly continuing to track the power line. During this process, the power line parameters output by the processor do not change abruptly, the flight control commands remain continuous, and the drone's flight attitude is not disturbed by the temporary obstruction.

[0045] Furthermore, when a power transmission line is segmented into multiple linear targets due to obstruction, if the directional angles between the linear targets, the distance from the midpoint to the inferred straight line, and the midpoint spacing meet the preset merging conditions, the system will merge the corresponding linear targets and recalculate the power transmission line parameters, thereby maintaining the topological integrity of the power transmission line tracking results. After completing the inspection of the current span, the UAV flies to the next span according to the planned route and repeats the above tracking process.

[0046] In this application example, the system maintains continuous and stable tracking of the power transmission line throughout the entire process from target acquisition and line-tracking flight to occlusion recovery. It can output continuous and stable power transmission line parameters in scenarios such as high-speed flight, sudden changes in lighting, and brief occlusion, supporting the UAV to autonomously complete the power transmission line inspection task.

[0047] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes, modifications, substitutions, or variations made by those skilled in the art within the scope of the technology disclosed in this application, based on the core concept of real-time transmission line tracking using the asynchronous characteristics of event cameras, without departing from the principles of this application—such as using different spatiotemporal geometric analysis algorithms to replace feature decomposition, using other efficient data structures to replace the event time surface, adjusting the state machine transition threshold, or changing the data transmission mechanism between hardware—as long as they utilize the temporal information and spatial neighborhood relationship of events to construct linear candidates and maintain the tracking state to achieve low-latency robust tracking, should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for real-time aerial tracking of power transmission lines based on event cameras, characterized in that, include: Based on the temporal information and spatial neighborhood relationships of multiple events in the asynchronous event stream output by the event camera, a directed chain growth is performed on the multiple events to generate linear candidates; Perform joint geometric statistical analysis on the spatial coordinates and timestamps of events in the linear candidate to determine the transmission line parameters; The linear candidates are used as linear targets, and the tracking state of the linear targets is maintained based on event density or event age. The tracking state includes an active state and a hold state. The transmission line parameters are updated in the active state, and the linear targets in the hold state are reactivated to the active state when a preset recovery condition is met. The parameters of the power transmission line are output for use in flight control.

2. The method for real-time aerial tracking of power transmission lines based on event cameras according to claim 1, characterized in that, The step of performing directed chain growth on the multiple events to generate linear candidates includes: The event time surface is used to record the time information of the most recent events of different polarities corresponding to each pixel position; Based on the seed event, search for candidate events that meet the time connection conditions in the neighborhood along a preset direction, and then connect the candidate events that meet the time connection conditions to form an event chain. When the event chain meets the preset length condition, the linear candidate is generated based on the event chain.

3. The method for real-time aerial tracking of power transmission lines based on event cameras according to claim 2, characterized in that, The step of performing directed chain growth on the multiple events to generate linear candidates further includes: After selecting a seed event, the gradient characteristics of the local region where the seed event is located are used to determine whether the preset dense region condition is met. If the condition is not met, the current seed event is skipped. If the condition is met, the directed chain growth process is started. During the directed chain growth process, based on the spatial direction of the previous candidate event added to the event chain relative to the current search position, the priority of the current search direction is determined from the preset direction priority sequence, and candidate events that satisfy the time connection condition are searched in the neighborhood according to the priority. If no candidate event satisfying the time connection condition is found within the first search range, the search is expanded to the second search range to continue the search. The directed chain growth terminates when the length of the event chain reaches a preset upper limit, no candidate event satisfying the time connection condition is found, or the image boundary is reached. Two-dimensional geometric analysis is performed on the spatial coordinates of events in the event chain that meet the preset length condition to generate a linear cluster as the linear candidate.

4. The method for real-time aerial tracking of power transmission lines based on event cameras according to claim 1, characterized in that, The joint geometric statistical analysis includes: The spatial coordinates and timestamps of the events in the linear candidate are used to construct a joint sample set, and the distribution statistics are calculated based on the joint sample set. Perform feature decomposition on the distribution statistics to extract the spatiotemporal normal vector and line direction of the linear candidate; The position, direction, and velocity of the transmission line parameters are analyzed based on the spatiotemporal normal vector and the line direction.

5. The method for real-time aerial tracking of power transmission lines based on an event camera according to claim 4, characterized in that, The process of analyzing the position, direction, and velocity of the transmission line parameters based on the spatiotemporal normal vector and the line direction includes: The eigenvector corresponding to the minimum eigenvalue in the eigenvalue decomposition is determined as the spatiotemporal normal vector, and the line direction is determined based on the projection direction of the spatiotemporal normal vector in the image plane; The length of the transmission line is determined based on the distribution statistics of events along the line direction in the joint sample set, and the width of the transmission line is determined based on the statistics of the perpendicular distance of events to the line direction. The apparent motion velocity is determined based on the correlation between the spatial dimension and the time dimension in the spatiotemporal normal vector. The midpoint of the linear target is compensated for based on the apparent motion velocity to eliminate positional deviations caused by event time delays.

6. The method for real-time aerial tracking of power transmission lines based on event cameras according to claim 1, characterized in that, Maintaining the tracking state of the linear target based on event density or event age includes: When the event density of the linear target is lower than a preset density condition or the latest event age exceeds a preset age condition, the linear target is switched from the active state to the hold state. In the held state, the event density is sampled along the direction of the transmission line parameters. When the sampling result meets the preset recovery condition, the linear target is reactivated from the held state to the active state. When the duration or number of times the linear target is in the holding state exceeds a preset holding condition, the linear target is deleted.

7. The method for real-time aerial tracking of power transmission lines based on an event camera according to claim 6, characterized in that, The tracking state also includes an initialization state, and the maintenance of the tracking state of the linear target based on event density or event age also includes: In the initialization state, the events of the linear target are projected onto the direction of the transmission line parameters and a connectivity length check is performed. Abnormal events that do not belong to the main connectivity part are removed. When the connectivity length check meets the preset connectivity conditions, the linear target is switched from the initialization state to the active state. In the active state, the transmission line parameters are periodically updated and time compensation is performed on the midpoint of the linear target; In the held state, the event-dense region is sampled along the direction of the transmission line parameters using linear rasterization; When the first linear target and the second linear target meet the preset merging conditions, the first linear target and the second linear target are merged and the transmission line parameters are recalculated. The preset merging conditions include the directional angle condition, the distance condition from the midpoint to the inferred straight line condition, and the midpoint spacing condition.

8. The method for real-time aerial tracking of power transmission lines based on event cameras according to claim 2, characterized in that, Before selecting a seed event, the method further includes performing preprocessing on the asynchronous event stream, the preprocessing including: The sampling density of adaptive time sampling is dynamically adjusted based on the comparison between the current event rate and the target event rate. Based on the same polarity of the most recent event time interval at the same pixel location, first time interval events that do not meet the preset refractory period condition are filtered out, and based on the opposite polarity of the most recent event time interval, second time interval events that do not meet the preset refractory period condition are filtered out. Background noise events are filtered out based on the number of events of the same polarity in the neighborhood of the current event.

9. A real-time aerial tracking system for power transmission lines based on an event camera, characterized in that, include: An event camera is used to output an asynchronous event stream; Memory is used to store computer program instructions; A processor, coupled to the event camera and the memory, is configured to execute the computer program instructions to perform the following operations: Based on the temporal information and spatial neighborhood relationship of multiple events in the asynchronous event stream, directed chain growth is performed on the multiple events to generate linear candidates; Perform joint geometric statistical analysis on the spatial coordinates and timestamps of events in the linear candidate to determine the transmission line parameters; The linear candidates are used as linear targets, and the tracking state of the linear targets is maintained based on event density or event age. The tracking state includes an active state and a hold state. The transmission line parameters are updated in the active state, and the linear targets in the hold state are reactivated to the active state when a preset recovery condition is met. Output the parameters of the transmission line; A flight control interface, coupled to the processor, is used to output the power line parameters or flight control information generated based on the power line parameters.

10. The real-time aerial tracking system for power transmission lines based on an event camera according to claim 9, characterized in that, The system is applied to a drone, which includes an airframe, a power unit, and a flight controller. The event camera is mounted on the airframe. The processor is also used to generate attitude control commands, speed control commands, or heading control commands based on the parameters of the power transmission line. The flight controller is used to control the power unit according to the attitude control commands, speed control commands, or heading control commands, so that the power transmission line remains within a preset tracking relationship.

Images