A power line extraction method and system based on time-series LiDAR point cloud data

Abstract

The application relates to the technical field of power line monitoring, and particularly relates to a power line extraction method and system based on time-series LiDAR point cloud data. The method acquires point cloud data of the power line, and performs data preprocessing and data enhancement on the collected point cloud data. A feature extraction module of a deep learning network model is adopted, and through a multi-level local and global feature extraction network thereof, the geometric features of the power line can be extracted from each frame of point cloud. In combination with time-series information and the output of the deep learning model, a semantic segmentation method is used to extract the spatial region of the power line. Through establishment of a power line candidate region, dynamic changes of the time-series point cloud are analyzed, and the position and form of the power line are located. A geometric reconstruction method is used to finely reconstruct the extracted power line, and missing parts of the power line are repaired. Through the deep learning method of the time-series LiDAR point cloud data, the power line extraction is realized, and accurate capture of dynamic changes of the power line is achieved.

Description

Technical Field

[0001] This invention relates to the field of power line monitoring technology, specifically to a method and system for extracting power lines based on time-series LiDAR point cloud data. Background Technology

[0002] The rapid evolution of modern society, driven by industrialization and urbanization, relies heavily on efficient power grids. Power line networks comprise transmission infrastructure at the national, regional, and distribution levels, traversing diverse terrains including plains, mountains, and hills. However, transmission lines are chronically exposed to harsh weather conditions such as strong winds and storms, posing significant risks to their safety, management, and maintenance. Regular inspections, including transmission line cleaning checks, help prevent faults or discharges that could potentially ignite fire hazards. Therefore, ensuring the continuous and reliable operation of power transmission systems through effective monitoring, inspection, and maintenance is crucial.

[0003] Traditional methods for inspecting power transmission lines, such as manual photography using digital cameras and telescopic inspections by ground personnel, have become labor-intensive and inefficient due to the increasing length of transmission lines. To address the challenges posed by large labor forces and harsh terrain conditions, thermal imaging sensors and cameras have been mounted on helicopter platforms. While this method improves monitoring effectiveness, it introduces challenges in processing the collected data and does not always achieve optimal spatial positioning accuracy. In recent years, LiDAR (LiDAR for Laser Detection and Ranging) has emerged as a powerful alternative technology in the field of power transmission line corridor inspection. Unlike thermal imaging sensors, LiDAR is unaffected by fluctuations in ambient temperature. Furthermore, LiDAR's ability to generate detailed scene representations allows for accurate reconstruction of the real environment by collecting precise point cloud data.

[0004] For example, invention patent CN114167203A discloses a method for rapid extraction of electric field lines based on laser point clouds. This method, through determining perpendicular lines and multi-stage fitting, can be categorized as a tracking method. Invention patent CN109829199B discloses a method for rapid layered extraction of electric field lines based on LiDAR point clouds. This method utilizes straight and parabolic electric field line models for model growth to obtain a complete electric field line point cloud; the arrangement of the electric field lines is obtained based on the number of layers and the number of electric field lines in each layer. However, the above-mentioned temporal point cloud electric field line extraction methods have the following shortcomings:

[0005] 1. Sparsity of time-series data and low frame rate:

[0006] Existing LiDAR sensors typically acquire point cloud data at low frame rates, such as 10-20 Hz. Therefore, the sparsity of time-series data affects the accuracy and stability of power line extraction. Especially when faced with rapid dynamic changes in power lines, existing methods often fail to effectively capture these changes, resulting in low accuracy of the extraction results.

[0007] 2. The impact of obstruction and complex environment:

[0008] Power line extraction faces significant challenges in complex environments, especially the impact of occlusions on LiDAR point cloud data. These occlusions can lead to missing or misidentified point cloud data, thus affecting the extraction results. Currently, existing algorithms are still not robust enough in handling power lines in occluded and complex environments.

[0009] 3. Noise and Irregular Data:

[0010] LiDAR point cloud data inevitably contains noise and outliers, especially in edge, distant, or complex terrain areas. The presence of noise affects the accurate extraction of power lines, and existing noise removal methods may not be sufficient for practical applications, particularly in complex power line scenarios, where noise removal remains a challenge.

[0011] 4. Computational complexity and real-time performance issues:

[0012] Temporal power line extraction typically requires processing large amounts of point cloud data, especially with deep learning methods, where computational resource consumption is often very high. For real-time power line monitoring tasks, existing technologies have high computational complexity and may not meet the demands of real-time processing. In applications on UAVs or other mobile platforms, excessive computation time and poor real-time performance are among the main bottlenecks faced by current methods. Summary of the Invention

[0013] Purpose of the invention: In order to overcome the shortcomings of the prior art, the present invention provides a method for extracting power lines based on time-series LiDAR point cloud data, and the present invention also provides a system for extracting power lines based on time-series LiDAR point cloud data.

[0014] Technical solution: According to a first aspect of the present invention, a method for extracting power lines based on time-series LiDAR point cloud data is provided, characterized in that the method includes:

[0015] Acquire point cloud data of power lines, and perform data preprocessing and data augmentation on the acquired point cloud data;

[0016] The feature extraction module employs a deep learning network model, which, through its multi-layered local and global feature extraction network, is able to extract the geometric features of power lines from each frame of point cloud.

[0017] The LSTM model captures the dynamic changes of power lines at different time points. The feature vector of the point cloud in each frame is used as the input of the LSTM. The LSTM models the changes of power lines between consecutive frames and learns the temporal characteristics of power lines.

[0018] By combining temporal information and the output of a deep learning model, a semantic segmentation method is used to extract the spatial regions of power lines; and by establishing candidate regions for power lines, the dynamic changes of temporal point clouds are analyzed to locate the position and shape of power lines.

[0019] The extracted power lines are finely reconstructed using geometric reconstruction methods to repair missing parts of the power lines.

[0020] The extracted power lines are smoothed and the results are output.

[0021] Furthermore, including:

[0022] The data preprocessing includes:

[0023] Temporal data alignment: Since LiDAR point cloud data is acquired frame by frame, point cloud frame alignment is performed first to ensure spatial consistency between each frame of data. Here, the ICP algorithm is used to register data at different time steps; Noise removal and filtering: Ground point segmentation method is used to remove irrelevant ground points, and filtering algorithm is used to downsample the point cloud to reduce the amount of computation; Temporal information extraction: Temporal features are extracted from the point cloud data.

[0024] The data enhancements include:

[0025] Random cropping and rotation: To enhance the robustness of the model, the point cloud data is subjected to enhanced operations of rotation, scaling and cropping to simulate power line data at different angles and scales;

[0026] Data synthesis: Generate synthetic datasets using multi-time data to simulate the impact of different environmental and weather changes on power line extraction.

[0027] Furthermore, including:

[0028] The method of capturing the dynamic changes of power lines at different points in time using an LSTM model includes:

[0029] Point-level fusion and frame-level context fusion are performed on the preprocessed temporal information to generate candidate regions for power lines. The point-level fusion involves weighting the features of the neighboring points and the features of the current point together or using attention convergence for a point in the current frame and its neighboring points in the previous and next frames. The frame-level context fusion involves feature pooling for each frame as a whole to obtain a frame summary. The frame summary is then input into an LSTM network model or a Transformer model to model the overall changes of power lines in the time dimension. Finally, the temporal context is fed back to the point features.

[0030] Furthermore, including:

[0031] The process of establishing candidate regions for power lines, analyzing the dynamic changes of time-series point clouds, and locating the position and shape of power lines includes:

[0032] Each power line in the candidate power line region is accurately extracted. During the extraction process, an LSTM network model is used to add time-series information at the feature level and introduce consistency, smoothness and topology constraints to further improve the accuracy and stability of power line extraction.

[0033] Furthermore, including:

[0034] The feature extraction module employing a deep learning network model, through its multi-layered local and global feature extraction network, is able to extract the geometric features of power lines from each frame of point cloud, including:

[0035] The Lidar-LSTMNet model is used to extract point features and global features of different scales. The Lidar-LSTMNet model is based on an encode-decode deep learning architecture, and the encoding part adopts the same encoding part as PointNet++. By abstracting multiple sets and downsampling at multiple levels, point features and global features of different scales are obtained.

[0036] Furthermore, including:

[0037] The semantic segmentation method extracts the spatial region of the power line, including:

[0038] Semantic segmentation of temporal LiDAR point cloud data is performed based on the PointNet++ network model to distinguish power line regions from background regions. The semantic segmentation head outputs the probability value of each point belonging to the power line or the background. After thresholding, a set of candidate power line points is obtained. Connectivity analysis and skeleton extraction are performed on these candidate points to generate candidate regions for power lines.

[0039] Furthermore, including:

[0040] The consistency constraint ensures that the prediction results for power lines remain consistent between consecutive frames, preventing the same point from being incorrectly identified as a power line or background at different times, especially for temporally continuous power line targets, and is expressed as:

[0041] Suppose that in frame t and frame (t-1), for point p... i The predicted probability of it being a power line is and Consistency constraints are expressed as:

[0042]

[0043] in, It is the probability that point i in frame t belongs to the power line category. It is the probability of the power line at the same point in the previous frame. By calculating the absolute error, the constraint model is made to maintain consistency over time.

[0044] Furthermore, including:

[0045] The smoothing constraint ensures a smooth transition of the power line over time, guaranteeing that the boundaries and positions of the power line do not exhibit illogical fluctuations. It is defined as the difference in the predicted probability of the power line between adjacent frames, expressed as:

[0046]

[0047] in, It is the probability that point i in frame t belongs to the power line category. It is the point p in frame t+1. i The probability of electric field lines.

[0048] Furthermore, including:

[0049] The topological constraints are used to constrain the temporal connectivity of power lines, reducing false detections caused by power line disconnections or incorrect connections. This is achieved by calculating the connectivity of power line regions in each frame, as shown below:

[0050]

[0051] Among them, |C t ∩C t-1 | represents the intersection of the power line regions in frame t and frame t-1, |C t ∪C t-1 | represents the union of the power line regions in frame t and frame t-1, which is used to encourage the connectivity of power lines in time and avoid unnecessary breaks; the overall loss function is a combination of consistency, smoothness, topology constraints and segmentation loss.

[0052] On the other hand, the present invention also provides a method and system for extracting power lines based on time-series LiDAR point cloud data, the system comprising:

[0053] The data preprocessing module is used to acquire point cloud data of power lines and perform data preprocessing and data enhancement on the acquired point cloud data.

[0054] The feature extraction module, which employs a deep learning network model, is capable of extracting the geometric features of power lines from each frame of point cloud through its multi-layered local and global feature extraction network.

[0055] The temporal feature learning module is used to capture the dynamic changes of power lines at different time points through the LSTM model. The feature vector of each frame of point cloud is used as the input of the LSTM. The LSTM models the changes of power lines between consecutive frames and learns the temporal characteristics of power lines.

[0056] The power line candidate region construction module is used to extract the spatial regions of power lines by combining temporal information and the output of deep learning models and using semantic segmentation methods; and by establishing power line candidate regions, the dynamic changes of temporal point clouds are analyzed to locate the position and shape of power lines.

[0057] The reconstruction module is used to perform fine reconstruction of the extracted power lines using geometric reconstruction methods, and to repair the missing parts of the power lines.

[0058] The results output module is used to smooth the extracted power lines and output the results.

[0059] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0060] This application uses a deep learning method based on temporal LiDAR point cloud data to extract power lines, achieving accurate capture of dynamic changes in power lines. Compared with traditional static point cloud methods, this invention has the following advantages: (1) Improved accuracy: By modeling with temporal data, the dynamic changes of power lines can be effectively captured, reducing errors caused by environmental changes, thereby improving the accuracy of power line extraction. (2) Enhanced robustness: Temporal point cloud data can capture more background information and environmental changes, enhancing the robustness of the model, especially in complex environments where it performs more stably. (3) Noise resistance: Through various means such as context constraints, droop model constraints, and temporal optimization, the interference of noise on power lines is minimized, enabling stable and effective extraction of power lines. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the power line extraction method based on time-series LiDAR point cloud data as described in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the power line extraction system based on time-series LiDAR point cloud data according to an embodiment of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1: This example demonstrates the innovative task of extracting power lines using deep learning methods based on temporal LiDAR point cloud data. It primarily relies on temporal information to improve the accuracy and robustness of power line extraction. Temporal LiDAR data offers significant advantages in power line extraction because it captures dynamic changes in the scene, increasing the accuracy of power line feature recognition. For example... Figure 1 The above includes the following steps:

[0065] Step S1: Data Acquisition and Time Series Data Generation

[0066] Data acquisition involves using LiDAR sensors to collect point cloud data of the area where the power lines are located, obtaining a 3D point cloud around the power lines. This point cloud data includes timestamps to identify data collected at different times.

[0067] Time-series data generation involves generating a series of time-series point cloud datasets based on the collected point cloud data, arranged in chronological order. Each frame of data represents the spatial location of a power line at a specific moment and is associated with the point clouds of preceding and following frames via timestamps, forming a time-series data stream.

[0068] Step S2: Temporal point cloud data preprocessing

[0069] Noise removal involves applying filtering algorithms, such as the Voxel Grid filter, to downsample the original point cloud, reduce the density of the point cloud, and remove unnecessary noise points and outliers.

[0070] Ground point separation involves removing ground points using a ground point separation algorithm, retaining only the point cloud information of power lines and their surrounding environment, focusing on the task of extracting power lines.

[0071] Specifically, the preprocessing in this embodiment includes: Since LiDAR point cloud data is acquired frame by frame, point cloud frame alignment is first required to ensure spatial consistency between each frame of data. This can be achieved using ICP, the Iterative Nearest Point algorithm, to register data from different time steps. In addition, this embodiment also includes:

[0072] Noise Removal and Filtering: Ground point segmentation methods are used to remove irrelevant ground points, while filtering algorithms, such as the Voxel Grid filter, are used to downsample the point cloud to reduce computational load; Temporal Information Extraction: Temporal features are extracted from the point cloud data, for example, by using timestamps to mark each frame of data and record the dynamic changes of power lines.

[0073] The working principle of the Voxel Grid filter in this embodiment includes:

[0074] Defining voxel sizes: First, the voxel sizes need to be defined, which determines the spatial size of each voxel. The choice of voxel size depends on the specific application requirements and the density of the point cloud.

[0075] Spatial partitioning: The entire point cloud space is divided into regular grids according to defined voxel sizes. Each voxel is a cube with a side length equal to the set voxel size.

[0076] Point cloud assignment: Each point is assigned to its corresponding voxel. If a point is located at the boundary between two voxels, it is usually assigned to one of the voxels, depending on the implementation method. For example, it can be specified that it is assigned to the smaller voxel.

[0077] Voxel merging: For each voxel, calculate the average position of all points within that voxel, or other statistical information such as minimum, maximum, median, etc., and use this position as the representative point of that voxel.

[0078] Generate a new point cloud: The representative points of all voxels constitute a new point cloud, which is sparser than the original point cloud but retains the key features of the original shape.

[0079] This embodiment also provides data augmentation functionality, including: primarily random cropping and rotation: to enhance the robustness of the model, point cloud data is subjected to augmentation operations such as rotation, scaling, and cropping to simulate power line data at different angles and scales. And data synthesis: using multi-time-lapse data to generate synthetic datasets, mimicking the impact of different environmental and weather changes on power line extraction.

[0080] Step S3: Feature Extraction and Deep Learning Processing

[0081] This embodiment specifically includes the following step: improved PointNet++ feature extraction. The feature extraction module, primarily the `encode` module, of the improved PointNet++ network model is used for point cloud feature extraction. Through its multi-layered local and global feature extraction network, it can extract the geometric features of electric field lines from each frame of the point cloud. The model extracts electric field line features from the point cloud based on information at different scales, such as linearity, curvature, and angles.

[0082] Specifically, in this embodiment, the multi-scale feature extraction using Lidar-LSTMNet is based on an encode-decode deep learning architecture. The encoding part employs the same encoding mechanism as PointNet++, abstracting multiple sets and performing multi-level downsampling to obtain point features and global features of different scales. These features serve as the input to the Long Short-Term Memory (LSTM) network. The LSTM network structure is primarily used to model the dynamic changes of electric field lines over time. Through the nesting of these network structures, the patterns of electric field line changes over time are learned.

[0083] In this embodiment, the encoder part of PointNet++ is its core architecture, mainly responsible for downsampling point cloud data and local feature aggregation through multi-level feature extraction. Its specific implementation mechanism is as follows:

[0084] Hierarchical downsampling structure: The Encoder consists of multiple set abstraction layers stacked together, each layer containing three key modules: sampling, grouping, and PointNet feature extraction.

[0085] By recursively applying this structure, the network can gradually expand its receptive field, transitioning from local geometric features to global semantic features.

[0086] Temporal modeling utilizes a Long Short-Term Memory (LSTM) network to process temporal point cloud data. The LSTM model captures the dynamic changes of power lines at different time points. The feature vector of each frame's point cloud serves as the input to the LSTM, which models the temporal characteristics of the power lines based on their variation patterns between consecutive frames.

[0087] In this embodiment, the specific dynamic change capture mainly includes fusing temporal context information, and mainly adopts point-level fusion strategy and frame-level context fusion strategy. The frame-level context fusion strategy is applied to each frame of point cloud data.

[0088] In this process, point-level fusion finds neighboring points in the preceding and following frames for a given point in the current frame, aligning the current point with its neighbors. The features of these neighbors are then weighted and averaged or aggregated with attention based on the features of the current point. This allows the model to determine whether a point is a power line not only by relying on a single frame but also by leveraging similar points in preceding and following frames.

[0089] Frame-level context fusion performs feature pooling on each frame to obtain a frame summary. These summaries are then input into an LSTM or Transformer model to model the overall changes of power lines over time, and the temporal context is then fed back to the point features.

[0090] In this embodiment, LSTM (Long Short-Term Memory) is a special type of recurrent neural network (RNN) designed specifically to address the vanishing / exploding gradient problem in traditional RNNs when processing long sequences of data. Its core functionality utilizes gating mechanisms (forget gate, input gate, output gate) and memory cell states to effectively capture long-term dependencies. It employs point-level fusion and frame-level fusion, using point-level fusion to capture details and frame-level context to ensure global continuity.

[0091] The Transformer model is a deep learning architecture based on a self-attention mechanism, proposed by the Google Brain team in 2017. Its core innovation lies in completely abandoning the sequence processing method of traditional RNNs and significantly improving the efficiency of tasks such as natural language processing (NLP) through parallel computing and global dependency modeling capabilities.

[0092] Step S4: Power Line Extraction and Location Prediction

[0093] Power line extraction involves combining temporal information and the output of a deep learning model, using semantic segmentation to extract the spatial regions of the power lines. Then, the model analyzes the dynamic changes in the temporal point cloud to locate the position and shape of the power lines.

[0094] In this embodiment, the generation of candidate power line regions involves semantic segmentation of temporal LiDAR point cloud data based on the PointNet++ network model, distinguishing power line regions from background regions. The time-varying patterns of power lines extracted in the above steps are incorporated into the existing semantic segmentation. That is, by introducing temporal context information, feature information from previous and subsequent frames is added to each frame of the point cloud, enhancing the accuracy of power line extraction.

[0095] Semantic segmentation of power lines uses a semantic segmentation head to output the probability that each point belongs to "power line" or "background". After thresholding, a set of candidate power line points is obtained. Connectivity analysis and skeleton extraction are performed on these candidate points to generate candidate regions for power lines. Because the features have been fused with information from previous and subsequent frames, the candidate regions are more complete and stable, reducing "broken lines" and "false detections".

[0096] In this embodiment, by establishing candidate regions for power lines, each power line is accurately extracted. By using the known parallel relationships and number of strands of power lines, reasonable inferences are made to determine that the power line at a certain location meets this constraint. The power line is then tracked and finally extracted by the constraints of the power lines before and after this constraint, the contextual time information, and the catenary model regression.

[0097] Among these methods, temporal smoothing techniques are introduced to eliminate fluctuations in power line detection caused by noise and erroneous data, ensuring the stability of the extraction results. Temporal information is used to optimize power line extraction, avoiding erroneous extractions due to the instability of single-frame data. LSTM networks can model the temporal variations of power lines, thereby extracting power lines more accurately within a temporal context.

[0098] In the temporal constraint section, in addition to incorporating temporal information at the feature level, the accuracy and stability of power line extraction can be further improved by introducing constraints such as consistency, smoothness, and topology. The following is a detailed explanation of each constraint and its corresponding formula.

[0099] (1) Consistency constraint: It ensures that the prediction results of power lines are consistent between consecutive frames, avoiding the same point being incorrectly identified as "power line" or "background" at different times, especially in power line targets that are continuous in time.

[0100] Suppose that in frame t and frame (t-1), for point p... i The predicted probability of it being a power line is and Consistency constraints are expressed as:

[0101]

[0102] in, It is the probability that point i in frame t belongs to the power line category. It is the probability of the power line at the same point in the previous frame. By calculating the absolute error, the constraint model is made to maintain consistency over time.

[0103] This loss function ensures that the class prediction of power line points remains stable between consecutive frames, preventing sudden false positives or false negatives. The consistency constraint is particularly suitable for continuous and stable targets such as power lines, preventing erroneous predictions caused by noise or occlusion in a single frame.

[0104] (2) Smoothing constraint:

[0105] The smoothing constraint ensures a smooth transition of the power line over time, guaranteeing that the boundaries and positions of the power line do not exhibit illogical fluctuations. It is defined as the difference in the predicted probability of the power line between adjacent frames, expressed as:

[0106]

[0107] in, It is the probability that point i in frame t belongs to the power line category. It is the point p in frame t+1. i The probability of electric field lines.

[0108] This loss function encourages the model to make smooth transitions between adjacent frames, avoiding instability at the power line boundaries due to changes in lighting or rapid movement.

[0109] Smoothing constraints are very important, especially when the position and shape of the power lines between adjacent frames change little, but the model may produce large prediction biases due to noise or other factors.

[0110] (3) Topological constraints:

[0111] Topological constraints are used to constrain the temporal connectivity of power lines, reducing false detections caused by power line cuts or incorrect connections. They are implemented by calculating the connectivity of power line regions in each frame, as follows:

[0112]

[0113] Among them, |C t ∩C t-1 | represents the intersection of the power line regions in frame t and frame t-1, |C t ∪C t-1 | represents the union of the power line regions in frame t and frame t-1, used to encourage temporal connectivity of power lines and avoid unnecessary breaks; topological constraints help ensure that changes in the shape and position of power lines over time are continuous. If the connectivity of power lines is lost between adjacent frames (i.e., the false or false detection area is too large), this constraint guides the model to correct this. This constraint is for the overall topology of the power lines and is suitable for preventing unreasonable breaks in power lines over time.

[0114] By combining consistency, smoothness, topological constraints, and segmentation loss, a complete temporal constraint loss function can be obtained:

[0115] L = L seg +λ1L consistency +λ2L smooth +λ3L topology ;

[0116] Where λ1, λ2, and λ3 are weight hyperparameters used to balance the contributions of various losses, L seg It is the standard segmentation loss, used to train the model to recognize power lines and the background; L consistency Ensure the consistency of power lines over time. smooth This makes the changes in power lines over time smoother, avoiding excessive fluctuations. topology Strengthen the connectivity of power lines to ensure their stability over time.

[0117] These constraints not only improve the stability and robustness of the model, but also make power line extraction more accurate and coherent. By optimizing these losses, the model can fully utilize contextual information when processing time-series data, effectively reducing false positives and false negatives.

[0118] Geometric reconstruction of power lines is performed using geometric reconstruction methods, such as the catenary model, to finely reconstruct the extracted power lines, repair missing parts of the power lines, and ensure the integrity and accuracy of the extraction results.

[0119] Step 5: Post-processing and result output

[0120] Post-processing mainly involves smoothing and denoising the data results. The extracted power lines are smoothed to eliminate noise caused by sensor errors or environmental variations. Bilateral filtering or B-spline curve smoothing methods can be used to optimize the extraction results.

[0121] The final extracted power line data is output in standard formats, such as LAS and PLY, for subsequent applications. The spatial coordinates and geometric information of the power lines can be used for tasks such as power line monitoring, inspection, and maintenance.

[0122] Specifically, smoothing algorithms, such as the Bilateral Filter, are applied to smooth the power line point cloud, removing noise caused by sensor errors and environmental influences. If the point cloud is sparse in certain areas, interpolation methods can be used to reconstruct these missing parts, ensuring the integrity of the power line model. The extracted power line point cloud is then converted into commonly used formats, such as LAS, PLY, and PCD, for subsequent applications and analysis.

[0123] In this embodiment, B-spline curve smoothing is a technique for path optimization through mathematical approximation. Its core lies in constructing piecewise polynomial curves using control points and node vectors, which has the characteristics of local controllability and adjustable continuity.

[0124] This embodiment can achieve the following advantages:

[0125] Handling the sparsity of time-series data: Current LiDAR sensors have low sampling frequencies, resulting in relatively sparse time-series data. This patent addresses the data sparsity problem to some extent by extracting multi-scale information from limited time-series data and extracting contextual information based on LSTM.

[0126] Modeling and Adaptability of Dynamic Changes: In practical applications, power lines may be affected by external factors (such as wind speed, temperature changes, etc.), leading to dynamic changes. This embodiment can effectively model these dynamic changes and use an LSTM network to capture long-term dependencies of temporal variations.

[0127] Strong noise resistance: In this embodiment, noise interference to power lines is minimized through various means such as temporal context association, droop model constraints, and temporal optimization, and power lines can be extracted stably and effectively.

[0128] Example 2: This invention also provides a power line extraction system based on time-series LiDAR point cloud data, such as... Figure 2 As shown, the system includes:

[0129] The data preprocessing module is used to acquire point cloud data of power lines and perform data preprocessing and data enhancement on the acquired point cloud data.

[0130] The feature extraction module, which employs a deep learning network model, is capable of extracting the geometric features of power lines from each frame of point cloud through its multi-layered local and global feature extraction network.

[0131] The temporal feature learning module is used to capture the dynamic changes of power lines at different time points through the LSTM model. The feature vector of each frame of point cloud is used as the input of the LSTM. The LSTM models the changes of power lines between consecutive frames and learns the temporal characteristics of power lines.

[0132] The power line candidate region construction module is used to extract the spatial regions of power lines by combining temporal information and the output of deep learning models and using semantic segmentation methods; and by establishing power line candidate regions, the dynamic changes of temporal point clouds are analyzed to locate the position and shape of power lines.

[0133] The reconstruction module is used to perform fine reconstruction of the extracted power lines using geometric reconstruction methods, and to repair the missing parts of the power lines.

[0134] The results output module is used to smooth the extracted power lines and output the results.

[0135] The other technical features of the power line extraction system based on time-series LiDAR point cloud data described in this embodiment are similar to those of the power line extraction method based on time-series LiDAR point cloud data, and will not be repeated here.

[0136] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0137] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0138] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0140] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0141] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0142] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0143] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for extracting power lines based on time-series LiDAR point cloud data, characterized in that, The method includes: Acquire point cloud data of power lines, and perform data preprocessing and data augmentation on the acquired point cloud data; The feature extraction module employs a deep learning network model, which uses its multi-layered local and global feature extraction network to extract the geometric features of power lines from each frame of point cloud. The LSTM model captures the dynamic changes of power lines at different time points. The feature vector of the point cloud in each frame is used as the input of the LSTM model. The LSTM model models the changes of power lines between consecutive frames and learns the temporal characteristics of power lines. By combining temporal information and the output of a deep learning model, a semantic segmentation method is used to extract the spatial regions of power lines; and by establishing candidate regions for power lines, the dynamic changes of temporal point clouds are analyzed to locate the position and shape of power lines. The extracted power lines are finely reconstructed using geometric reconstruction methods to repair missing parts of the power lines. The extracted power lines are smoothed and the results are output. The method of capturing the dynamic changes of power lines at different points in time using an LSTM model includes: Point-level fusion and frame-level context fusion are performed on the preprocessed temporal information to generate candidate regions for power lines. The point-level fusion is to take a point in the current frame and its neighboring points in the previous and next frames, and then weight the features of the neighboring points and the features of the current point or perform attention convergence. The frame-level context fusion is to perform feature pooling on the entire frame to obtain a frame summary. The frame summary is input into an LSTM model or a Transformer model to model the overall changes of power lines in the time dimension, and then the temporal context is fed back to the point features. The process of establishing candidate regions for power lines, analyzing the dynamic changes of time-series point clouds, and locating the position and shape of power lines includes: Each power line in the candidate power line region is accurately extracted. During the extraction process, an LSTM network model is used to add temporal information at the feature level, as well as introduce consistency, smoothness and topology constraints to further improve the accuracy and stability of power line extraction. The topological constraints are used to constrain the temporal connectivity of power lines, reducing false detections caused by power line disconnections or incorrect connections. This is achieved by calculating the connectivity of power line regions in each frame, as shown below: ; in, Indicates the first Frame and the The intersection of the power line regions of the frames. Indicates the first Frame and the The union of power line regions in a frame is used to encourage the connectivity of power lines over time and avoid unnecessary breaks.

2. The method for extracting power lines based on time-series LiDAR point cloud data according to claim 1, characterized in that, The data preprocessing includes: Time-series data alignment: Since LiDAR point cloud data is acquired frame by frame, point cloud frame alignment is performed first to ensure spatial consistency between each frame of data. Here, the ICP algorithm is used to register data at different time steps. Noise Removal and Filtering: Irrelevant ground points are removed using ground point segmentation methods, while filtering algorithms are used to downsample the point cloud to reduce computational load; Temporal Information Extraction: Temporal features are extracted from the point cloud data; The data enhancements include: Random cropping and rotation: To enhance the robustness of the model, the point cloud data is subjected to enhanced operations of rotation, scaling and cropping to simulate power line data at different angles and scales; Data synthesis: Generate synthetic datasets using multi-time data to simulate the impact of different environmental and weather changes on power line extraction.

3. The method for extracting power lines based on time-series LiDAR point cloud data according to claim 1, characterized in that, The feature extraction module employing a deep learning network model, through its multi-layered local and global feature extraction network, is able to extract the geometric features of power lines from each frame of point cloud, including: The Lidar-LSTMNet model is used to extract point features and global features of different scales. The Lidar-LSTMNet model is based on an encode-decode deep learning architecture, and the encoding part adopts the same encoding part as PointNet++. By abstracting multiple sets and downsampling at multiple levels, point features and global features of different scales are obtained.

4. The method for extracting power lines based on time-series LiDAR point cloud data according to claim 1, characterized in that, The semantic segmentation method extracts the spatial region of the power line, including: Semantic segmentation of temporal LiDAR point cloud data is performed based on the PointNet++ network model to distinguish power line regions from background regions. The semantic segmentation head outputs the probability value of each point belonging to the power line or the background. After thresholding, a set of candidate power line points is obtained. Connectivity analysis and skeleton extraction are performed on these candidate points to generate candidate regions for power lines.

5. The method for extracting power lines based on time-series LiDAR point cloud data according to claim 1, characterized in that, The consistency constraint ensures that the prediction results for power lines remain consistent between consecutive frames, preventing the same point from being incorrectly identified as a power line or background at different times, as expressed as: Assuming in the first Frame and the In a frame, for a point The predicted probability of it being a power line is and The consistency constraint is expressed as: ; in, It is the first Frame point The probability of belonging to the power line category. It is the probability of the power line at the same point in the previous frame. By calculating the absolute error, the constraint model is made to maintain consistency over time.

6. The method for extracting power lines based on time-series LiDAR point cloud data according to claim 5, characterized in that, The smoothing constraint ensures a smooth transition of the power line over time, guaranteeing that the boundaries and positions of the power line do not exhibit illogical fluctuations. It is defined as the difference in the predicted probability of the power line between adjacent frames, expressed as: ; in, It is the first Frame point The probability of belonging to the power line category. It is the first Frame point The probability of electric field lines.

7. A power line extraction system based on time-series LiDAR point cloud data, characterized in that, The system includes: The data preprocessing module is used to acquire point cloud data of power lines and perform data preprocessing and data enhancement on the acquired point cloud data. The feature extraction module is used to extract the geometric features of the electric power lines from each frame of point cloud through its multi-layered local and global feature extraction network. The temporal feature learning module is used to capture the dynamic changes of power lines at different time points through the LSTM model. The feature vector of each frame of point cloud is used as the input of the LSTM model. The LSTM model models the changes of power lines between consecutive frames and learns the temporal characteristics of power lines. The method of capturing the dynamic changes of power lines at different points in time using an LSTM model includes: Point-level fusion and frame-level context fusion are performed on the preprocessed temporal information to generate candidate regions for power lines. The point-level fusion is to take a point in the current frame and its neighboring points in the previous and next frames, and then weight the features of the neighboring points and the features of the current point or perform attention convergence. The frame-level context fusion is to perform feature pooling on the entire frame to obtain a frame summary. The frame summary is input into an LSTM model or a Transformer model to model the overall changes of power lines in the time dimension, and then the temporal context is fed back to the point features. The power line candidate region construction module is used to extract the spatial regions of power lines by combining temporal information and the output of deep learning models and using semantic segmentation methods; and by establishing power line candidate regions, the dynamic changes of temporal point clouds are analyzed to locate the position and shape of power lines. The process of establishing candidate regions for power lines, analyzing the dynamic changes of time-series point clouds, and locating the position and shape of power lines includes: Each power line in the candidate power line region is accurately extracted. During the extraction process, an LSTM network model is used to add temporal information at the feature level, as well as introduce consistency, smoothness and topology constraints to further improve the accuracy and stability of power line extraction. The topological constraints are used to constrain the temporal connectivity of power lines, reducing false detections caused by power line disconnections or incorrect connections. This is achieved by calculating the connectivity of power line regions in each frame, as shown below: ; in, Indicates the first Frame and the The intersection of the power line regions of the frames. Indicates the first Frame and the The union of the power line regions of the frame is used to encourage the connectivity of power lines in time and avoid unnecessary breaks; The reconstruction module is used to perform fine reconstruction of the extracted power lines using geometric reconstruction methods, and to repair the missing parts of the power lines. The results output module is used to smooth the extracted power lines and output the results.

Images