Method for detecting tree obstacles in power transmission line channel based on computer vision

Abstract

The invention provides a method for detecting tree obstacles in a power transmission line channel based on computer vision, and relates to the technical field of power system safety monitoring, and the method comprises the steps: carrying out the parallel feature extraction and time-space correlation of multispectral image data and laser point cloud data, and solving a recognition conflict according to a preset arbitration rule, generating a three-dimensional semantic segmentation result uniformly containing a tree target and a traverse target; calculating the current shortest distance between the three-dimensional model of each tree in the final three-dimensional scene model and the safety corridor, and predicting the time point when each tree invades the corridor in the future based on historical time sequence data; and according to the current shortest distance and the future intrusion time point, generating graded early warning information with different disposal priorities. According to the invention, deep credible fusion of multi-source information can be realized, a self-optimization capability is provided, and intelligent tree obstacle detection and early warning of dynamic risk deduction can be carried out, so that the accuracy, high efficiency and foresight of operation and maintenance of a power transmission line are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power system safety monitoring, in particular to a tree barrier detection method in a power transmission line channel based on computer vision. BACKGROUND

[0002] The growth of trees in the power transmission line channel is one of the main hidden dangers threatening the safe and stable operation of the power grid, which can easily cause discharge tripping, short circuit ablation and other accidents. The traditional manual inspection method has inherent defects such as low efficiency, high risk and many blind spots.

[0003] With the development of remote sensing technology, the automatic inspection method based on unmanned aerial vehicle platform has gradually become the mainstream. The existing technical solutions mainly fall into three categories: the first category is a pure vision method based on visible light images, which identifies trees through a deep learning model and estimates the tree line distance with monocular vision or digital surface model. This method is significantly affected by light, shadow and seasonal changes, has large distance estimation error, and has insufficient reliability in complex scenes such as mountainous areas and forest areas. The second category is a geometric analysis method based on laser point cloud, which can obtain accurate three-dimensional information, but the point cloud data lacks semantics, making it extremely difficult to automatically segment the conductor and trees, especially to distinguish dense vegetation from the conductor, which still requires a lot of manual interaction and is low in efficiency. The third category is a simple serial method that tries to fuse images and point clouds, such as identifying targets in images and then mapping to point cloud to calculate height. However, this "open loop" fusion method cannot correct the front-end recognition errors, and does not fully utilize the three-dimensional geometric information to verify and optimize the two-dimensional recognition results. Essentially, it is a simple superposition at the data level rather than a deep coupling at the information level, and its precision and robustness are limited.

[0004] In addition, existing tree barrier warning models generally use static safety distance thresholds and do not consider the dynamic changes of conductor sag with temperature, the amplitude of tree wind deflection, and the growth trend of trees. This leads to rigid warning results, which are prone to false positives or false negatives under extreme conditions such as high temperature and strong wind, and cannot achieve the transition from "after-treatment" to "prior accurate warning".

[0005] Therefore, a tree barrier detection method in a power transmission line channel based on computer vision is proposed. SUMMARY

[0006] The present application provides a tree barrier detection method in a power transmission line channel based on computer vision, which can realize deep and reliable fusion of multi-source information, has self-optimization capability, and can perform dynamic risk deduction for intelligent tree barrier detection and warning, thereby improving the accuracy, efficiency and forward-looking nature of power transmission line operation and maintenance.

[0007] The present application provides a tree barrier detection method in a power transmission line channel based on computer vision, comprising:

[0008] acquire multi-spectral image data and laser point cloud data of a power transmission channel;

[0009] perform parallel feature extraction and spatio-temporal correlation on the multi-spectral image data and the laser point cloud data, and solve recognition conflicts according to a preset arbitration rule to generate a three-dimensional semantic segmentation result containing tree targets and conductor targets;

[0010] guided by the three-dimensional semantic segmentation result, apply physical constraints to the conductor point cloud for three-dimensional reconstruction, and perform single-tree segmentation on the tree point cloud based on spatial and spectral features to obtain an initial three-dimensional scene model;

[0011] back-project the single-tree segmentation result in the initial three-dimensional scene model into the multi-spectral image data, perform contour consistency verification, and generate a verification result;

[0012] based on the verification result, iteratively adjust parameters used in the single-tree segmentation process, and repeat the guided segmentation step and the verification feedback step until the contour consistency meets a preset condition, and output a final three-dimensional scene model;

[0013] based on the conductor three-dimensional model in the final three-dimensional scene model, construct a spatio-temporal dynamic safety corridor in combination with electrical safety distance, wind-induced swing envelope, and thermal-induced sag dynamics;

[0014] calculate the current shortest distance between each tree three-dimensional model in the final three-dimensional scene model and the safety corridor, and predict the time point at which each tree will intrude into the corridor based on historical time series data;

[0015] generate hierarchical warning information with different handling priorities according to the current shortest distance and the future intrusion time point.

[0016] Optionally, the preset arbitration rule includes:

[0017] when the recognition result of the same area is identified as a first category in the multi-spectral image data and a second category in the laser point cloud data:

[0018] if the first category or the second category involves vegetation, the recognition result of the laser point cloud data is preferred;

[0019] if the first category and the second category involve different power line components, the recognition result of the multi-spectral image data is preferred, and its morphological features are verified against a preset component geometric template.

[0020] Optionally, the physical constraints applied to the conductor point cloud for three-dimensional reconstruction include at least one of:

[0021] Continuity constraint of the conductor in three-dimensional space;

[0022] Conductor catenary or approximate linear model constraint;

[0023] Based on the line voltage level and topological structure simulation of the electric field distribution constraint, used to distinguish different phase conductors in multi-loop line scene.

[0024] Optionally, the spatial and spectral feature coupling of the tree point cloud for single tree segmentation includes:

[0025] Taking the tree point cloud in the three-dimensional semantic segmentation result as the initial seed point;

[0026] Based on the region growing algorithm, the growth criterion contains both the Euclidean distance threshold in three-dimensional space and the color space similarity threshold mapped from the multispectral image data;

[0027] When the growth stops, each connected point cloud cluster formed is identified as an independent tree.

[0028] Optionally, based on the verification result, the parameters used in the single tree segmentation process of the guided segmentation module are iteratively adjusted, including:

[0029] According to the quantization error generated by the contour consistency verification, the distance threshold and / or color similarity threshold used in the single tree segmentation process are dynamically adjusted; wherein, when the quantization error increases, the threshold is tightened to perform over-segmentation; when the quantization error decreases, the threshold is relaxed to perform under-segmentation correction.

[0030] Optionally, the construction of the spatio-temporal dynamic safety corridor includes:

[0031] Taking the normal direction of each point on the conductor three-dimensional model as the reference, a dynamic offset vector is superimposed to form the corridor boundary;

[0032] The dynamic offset vector is obtained by vector composition of the basic electrical safety distance, the maximum wind deflection amplitude calculated according to the historical wind speed and the conductor parameters, and the sag change value calculated according to the real-time environmental temperature and the conductor thermal expansion coefficient.

[0033] Optionally, the prediction of the time point when each tree invades the corridor in the future based on historical time series data includes:

[0034] Obtain the three-dimensional model of the same tree at at least two historical time points, and fit to obtain the growth rate of tree height or crown volume;

[0035] Identify the tree species of the tree, and obtain the typical seasonal growth coefficient of the tree species;

[0036] In combination with the growth rate and the seasonal growth coefficient, a time series prediction model is established to deduce the evolution of the tree three-dimensional model in a specified future period of time and calculate the time point at which the shortest distance between the tree three-dimensional model and the spatiotemporal dynamic safety corridor boundary becomes zero.

[0037] Optionally, the hierarchical early warning information comprises:

[0038] The first-level early warning corresponds to the current shortest distance being less than zero, indicating that intrusion has occurred.

[0039] The second-level early warning corresponds to the current shortest distance being greater than zero but less than a first threshold value, and the future intrusion time point being predicted to be within the next planned inspection cycle.

[0040] The third-level early warning corresponds to the current shortest distance being greater than the first threshold value but less than a second threshold value, or the future intrusion time point being predicted to be after the next planned inspection cycle but within a preset time limit.

[0041] The fourth-level early warning corresponds to the current shortest distance being greater than the second threshold value and the future intrusion time point exceeding the preset time limit, indicating that only regular monitoring is required.

[0042] The present specification provides a computer vision-based tree barrier detection device for a power transmission line corridor, comprising:

[0043] A data acquisition module is configured to acquire synchronous multispectral image data and laser point cloud data of the power transmission corridor.

[0044] A fusion perception module is configured to perform parallel feature extraction and spatiotemporal correlation on the multispectral image data and the laser point cloud data, and generate a unified three-dimensional semantic segmentation result containing tree targets and conductor targets according to a preset arbitration rule to resolve recognition conflicts.

[0045] A guided segmentation module is configured to use the three-dimensional semantic segmentation result as a guide to perform three-dimensional reconstruction on conductor point clouds by applying physical constraints, and perform single-tree segmentation on tree point clouds by coupling spatial and spectral features, to obtain an initial three-dimensional scene model.

[0046] A verification module is configured to back-project single-tree segmentation results in the initial three-dimensional scene model to the multispectral image data to perform contour consistency verification and generate a verification result.

[0047] An iteration control module is configured to iteratively adjust parameters used in the single-tree segmentation process of the guided segmentation module based on the verification result, and control the guided segmentation module and the verification module to repeatedly execute until the contour consistency meets a preset condition, and output a final three-dimensional scene model.

[0048] a dynamic corridor construction module, configured to construct a time-space dynamic safety corridor based on a conductor three-dimensional model in the final three-dimensional scene model, in combination with electrical safety distance, wind-induced swing envelope and thermal-induced sag dynamics;

[0049] a calculation prediction module, configured to calculate a current shortest distance of each tree three-dimensional model in the final three-dimensional scene model to the safety corridor, and predict a time point of future invasion of each tree to the corridor based on historical time series data;

[0050] a warning generation module, configured to generate hierarchical warning information with different handling priorities according to the current shortest distance and the future invasion time point.

[0051] Optionally, the preset arbitration rule comprises:

[0052] when the identification result of the same region is identified as a first category in the multispectral image data and as a second category in the laser point cloud data:

[0053] if the first category or the second category involves vegetation, the identification result of the laser point cloud data is preferentially adopted;

[0054] if the first category and the second category involve different power line components, the identification result of the multispectral image data is preferentially adopted, and it is verified whether a shape feature thereof conforms to a preset component geometric template.

[0055] Optionally, the three-dimensional reconstruction by applying physical constraints to the conductor point cloud comprises at least one of the following:

[0056] continuity constraint of the conductor in three-dimensional space;

[0057] catenary or approximate linear model constraint of the conductor;

[0058] electric field distribution constraint based on line voltage level and topological structure simulation, for distinguishing conductors of different phases in a multi-circuit line scene.

[0059] Optionally, the guided segmentation module comprises:

[0060] taking a tree point cloud in the three-dimensional semantic segmentation result as an initial seed point;

[0061] based on a region growing algorithm, a growth criterion of which comprises a Euclidean distance threshold in three-dimensional space and a color space similarity threshold mapped from the multispectral image data;

[0062] when the growth is stopped, each connected point cloud cluster formed is identified as an independent tree.

[0063] Optionally, the iteration control module comprises:

[0064] Based on the quantization error generated by contour consistency verification, the distance threshold and / or color similarity threshold used in the single-tree segmentation process are dynamically adjusted; wherein, when the quantization error increases, the threshold is tightened to perform over-segmentation; when the quantization error decreases, the threshold is relaxed to perform under-segmentation correction.

[0065] Optionally, the dynamic corridor construction module includes:

[0066] Using the normal direction of each point on the three-dimensional model of the conductor as a reference, a dynamic offset vector is superimposed to form the corridor boundary;

[0067] The dynamic offset vector is obtained by vector synthesis of the basic electrical safety distance, the maximum wind deflection amplitude calculated based on historical wind speed and conductor parameters, and the sag change value calculated based on real-time ambient temperature and conductor thermal expansion coefficient.

[0068] Optionally, the calculation and prediction module includes:

[0069] Obtain 3D models of the same tree at at least two historical time points, and fit them to obtain the growth rate of its tree height or canopy volume;

[0070] Identify the tree species and obtain the typical seasonal growth coefficient of the tree species;

[0071] By combining the growth rate and the seasonal growth coefficient, a time series prediction model is established to predict the evolution of the tree's three-dimensional model within a specified future time period, and to calculate the time point at which the shortest distance between the tree and the boundary of the spatiotemporal dynamic safety corridor becomes zero.

[0072] Optionally, the early warning generation module includes:

[0073] The first level of warning corresponds to the current shortest distance being less than zero, indicating that an intrusion has occurred;

[0074] The second level of warning corresponds to the current shortest distance being greater than zero but less than the first threshold, and the predicted future intrusion time point being within the next planned inspection cycle;

[0075] The third level of warning corresponds to the current shortest distance being greater than the first threshold but less than the second threshold, or the predicted future intrusion time being after the next planned inspection cycle but within a preset number of years.

[0076] The fourth level of warning corresponds to the current shortest distance being greater than the second threshold and the future intrusion time exceeding the preset number of years, indicating that only routine monitoring is required.

[0077] This specification also provides an electronic device, wherein the electronic device includes:

[0078] A processor; and a memory storing computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0079] This specification also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement any of the methods described above.

[0080] This invention employs a parallel feature extraction-arbitration fusion mechanism and establishes a closed-loop process of guided segmentation-backprojection verification-iterative optimization. This allows for deep and dynamic mutual verification and enhancement of two-dimensional texture information and three-dimensional geometric information, fundamentally overcoming the shortcomings of high false detection rates and poor anti-interference capabilities associated with single data sources or simple fusion methods. From data input to early warning output, the entire process requires no manual intervention. In particular, the dynamic parameter adjustment strategy based on quantization error enables the system to possess adaptive optimization capabilities, automatically adjusting to the optimal processing state for different scenarios, greatly enhancing the method's universality and practicality. Breaking through the limitations of static thresholds, by integrating dynamic factors from multiple physical fields such as electrical safety, wind-induced sway, and thermally induced sag, a spatiotemporal dynamic safety corridor is constructed. This ensures that risk assessment results accurately reflect the actual risk status of line operation, significantly improving the scientific rigor and accuracy of early warnings. By introducing historical time-series data and growth prediction models, not only can current risks be assessed, but the future intrusion risk time of trees can also be proactively predicted, providing valuable early warning leads for maintenance departments and realizing the transformation and upgrading of transmission line maintenance from passive response to proactive defense. The final output of tiered early warning information transforms abstract detection data into handling suggestions of different urgency levels, and links them to inspection cycles and planned lifespans. This can directly guide the prioritization and resource allocation of on-site operation and maintenance work, greatly improving management efficiency. Attached Figure Description

[0081] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 A schematic diagram illustrating the principle of a computer vision-based tree obstacle detection method in a power transmission line corridor, as provided in the embodiments of this specification.

[0083] Figure 2 A schematic diagram of a tree obstacle detection device based on computer vision in a power transmission line corridor, provided as an embodiment of this specification;

[0084] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0085] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification. Detailed Implementation

[0086] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0087] The following is in conjunction with the appendix Figures 1-4 Exemplary embodiments of the invention will be described more fully here. However, exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the figures denote the same or similar elements, components, or parts, and therefore repeated descriptions of them are omitted.

[0088] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0089] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.

[0090] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0091] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0092] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0093] Figure 1 This is a schematic diagram illustrating the principle of a computer vision-based tree obstacle detection method in a power transmission line corridor, as provided in an embodiment of this specification. The method may include:

[0094] S110: Acquire multispectral image data and laser point cloud data of the power transmission channel;

[0095] In the specific implementation of this manual, the tree obstruction inspection of a 220kV single-circuit transmission line is taken as an example. A DJI Matrice 350 RTK drone is used, equipped with a Zenmuse P1 multispectral camera and a Zenmuse L2 lidar. The drone follows a planned terrain-following flight path along the transmission line to collect high-overlapping oblique images and high-density laser point clouds.

[0096] S120: Perform parallel feature extraction and spatiotemporal correlation on the multispectral image data and the laser point cloud data, resolve identification conflicts according to preset arbitration rules, and generate a unified three-dimensional semantic segmentation result that includes tree targets and guide wire targets.

[0097] In the specific implementation of this specification, the YOLOv8 instance segmentation model based on the PyTorch framework (which has been fine-tuned with power scene data) is used to identify the image and output pixel-level masks for trees, wires, etc.; the point cloud is coarsely segmented based on the RandLA-Net point cloud semantic segmentation model (which has also been pre-trained).

[0098] For an area that is identified as a "house" by the image model but as "vegetation" by the point cloud model, the point cloud results are adopted according to the rules, and the area is marked as "trees" to generate the initial 3D semantic segmentation results.

[0099] Optionally, the preset arbitration rules include:

[0100] When the identification result for the same region is identified as category 1 in the multispectral image data and category 2 in the laser point cloud data:

[0101] If the first or second category involves vegetation, the identification results of the laser point cloud data shall be adopted first.

[0102] If the first category and the second category involve different power line components, the identification results of the multispectral image data shall be adopted first, and their morphological features shall be verified to conform to the preset component geometric template.

[0103] In the specific implementation of this specification, a clear software arbitration logic is established during the fusion perception stage. When the image recognition module identifies a target as an "insulator," while the point cloud segmentation module identifies it as "background (low vegetation)," the system checks the target's morphology in the image: it has a distinct disk-shaped structure and metallic texture, conforming to the geometric template library features of an "insulator." Simultaneously, although its 3D point cloud is sparse, its spatial location is near the tower structure, conforming to the topological distribution of line components. Therefore, the system adopts the image recognition result according to the rules and corrects its 3D category to "power line component (insulator)," avoiding misclassification as vegetation.

[0104] S130: Guided by the three-dimensional semantic segmentation results, apply physical constraints to the wire point cloud for three-dimensional reconstruction, and simultaneously perform single-tree segmentation of the tree point cloud using spatial and spectral feature coupling to obtain an initial three-dimensional scene model.

[0105] In the specific implementation of this specification, point clouds classified as "wires" are fitted using the RANSAC algorithm combined with catenary model constraints to reconstruct 3D wires. Point clouds classified as "trees" are used with each cluster center as a seed, and an initial distance threshold of 0.5 meters and a color similarity threshold of 15 (in HSV space) are set to perform region growing, complete single-tree segmentation, and obtain the initial scene model.

[0106] Optionally, the physical constraints applied to the conductor point cloud for 3D reconstruction include at least one of the following:

[0107] Continuity constraints of the conductor in three-dimensional space;

[0108] Catenary or approximate linear model constraints;

[0109] Electric field distribution constraints based on line voltage level and topology simulation are used to distinguish conductors of different phases in multi-circuit line scenarios.

[0110] In the specific implementation of this specification, when reconstructing the conductor point cloud for fine-grained reconstruction, in addition to using the least squares method to fit a straight line, an additional physical constraint algorithm is applied. For a double-circuit line on the same tower, the program simulates a simple electrostatic field model to calculate the potential gradient direction at each point in space. When segmenting the point cloud, points belonging to the same phase conductor are required to not only satisfy the linear constraint, but their spatial distribution trend should also be approximately perpendicular to the simulated potential gradient direction around that phase conductor. This constraint effectively assists the algorithm in distinguishing conductor point clouds that are spatially close but belong to different phases.

[0111] Optionally, the single-tree segmentation of the tree point cloud by spatial and spectral feature coupling includes:

[0112] The tree point cloud in the three-dimensional semantic segmentation result is used as the initial seed point;

[0113] Based on the region growing algorithm, its growing criteria simultaneously include the Euclidean distance threshold in three-dimensional space and the color space similarity threshold mapped from the multispectral image data.

[0114] Once growth ceases, each connected point cluster formed will be identified as an independent tree.

[0115] S140: Backproject the single-tree segmentation results in the initial 3D scene model to the multispectral image data, perform contour consistency verification, and generate verification results;

[0116] S150: Based on the verification results, the parameters used in the single-tree segmentation process are iteratively adjusted, and the guided segmentation step and verification feedback step are repeated until the contour consistency meets the preset conditions, and the final three-dimensional scene model is output.

[0117] In the specific implementation of this specification, the point cloud of a tree in this model is back-projected onto all visible images. The calculated IoU (Intersection over Union) between its 2D projected contour and the actual tree edge in the image is only 0.65 (the preset condition is ≥0.8). The system determines that the verification fails, automatically tightens the distance threshold to 0.3 meters, and re-segments and verifies the individual tree in that area. After two iterations, the IoU improves to 0.82, meeting the condition, and the final 3D scene model is output.

[0118] Optionally, based on the verification results, the parameters used in the single-tree segmentation process in the guided segmentation module are iteratively adjusted, including:

[0119] Based on the quantization error generated by contour consistency verification, the distance threshold and / or color similarity threshold used in the single-tree segmentation process are dynamically adjusted; wherein, when the quantization error increases, the threshold is tightened to perform over-segmentation; when the quantization error decreases, the threshold is relaxed to perform under-segmentation correction.

[0120] In the specific implementation of this specification, contour consistency verification generates a quantization error. The system maintains a parameter adjustment lookup table: if the current error E_current is greater than the previous error E_previous, it indicates a decrease in segmentation quality, possibly due to oversegmentation (dividing a tree into multiple trees). In this case, the system increases the distance threshold by a preset step size (e.g., 0.1 meters) to make growth conditions more lenient and encourage point cloud merging. Conversely, if the error decreases, the color similarity threshold is decreased by a step size to make growth more sensitive to color, correcting potential undersegmentation (merging multiple trees) caused by similar colors.

[0121] S160: Based on the 3D model of the conductor in the final 3D scene model, and combined with electrical safety distance, wind-induced sway envelope and thermal sag dynamics, a spatiotemporal dynamic safety corridor is constructed.

[0122] In the specific implementation of this specification, parameters such as the voltage level of the line (220kV), the maximum wind speed record provided by the local meteorological station (28m / s), the conductor type (LGJ-400 / 35), and the current temperature (30℃) are used to calculate wind deflection and dynamic sag, constructing a dynamic safety corridor that changes over time within this span. The system reads the height data of the tree from the previous two inspection records (12.1m, 12.9m), fits its growth rate, and predicts that the treetop will reach the boundary of the dynamic safety corridor in 5 months.

[0123] Optionally, the construction of the spatiotemporal dynamic security corridor includes:

[0124] Using the normal direction of each point on the three-dimensional model of the conductor as a reference, a dynamic offset vector is superimposed to form the corridor boundary;

[0125] The dynamic offset vector is obtained by vector synthesis of the basic electrical safety distance, the maximum wind deflection amplitude calculated based on historical wind speed and conductor parameters, and the sag change value calculated based on real-time ambient temperature and conductor thermal expansion coefficient.

[0126] In the specific implementation of this specification, the formula for calculating the dynamic offset vector D_total is: D_total = D_electric + D_wind + D_sag.

[0127] D_electric: According to the "DL / T741 Overhead Transmission Line Operation Regulations", the minimum vertical distance between a 220kV line and a tree is 4.5 meters.

[0128] D_wind: Calculated according to the formula D_wind=L*sin(θ), where L is the horizontal distance from the calculation point to the tower suspension point, and θ is the maximum wind deflection angle calculated from the design wind speed and conductor parameters.

[0129] D_sag: Based on the current temperature t and conductor characteristics, the sag increment relative to the standard temperature (20℃) is obtained by referring to the sag characteristic curve.

[0130] By superimposing D_total along the normals of each point on the traverse, the dynamic safety corridor boundary surface at that moment is formed. The system updates D_sag every certain period of time (e.g., every hour) based on the real-time temperature, realizing the "dynamic" change of the corridor.

[0131] S170: Calculate the current shortest distance between each tree 3D model in the final 3D scene model and the safety corridor, and predict the time point when each tree will invade the corridor in the future based on historical time series data;

[0132] Optionally, the prediction of the future intrusion time point of each tree into the corridor based on historical time-series data includes:

[0133] Obtain 3D models of the same tree at at least two historical time points, and fit them to obtain the growth rate of its tree height or canopy volume;

[0134] Identify the tree species and obtain the typical seasonal growth coefficient of the tree species;

[0135] By combining the growth rate and the seasonal growth coefficient, a time series prediction model is established to predict the evolution of the tree's three-dimensional model within a specified future time period, and to calculate the time point at which the shortest distance between the tree and the boundary of the spatiotemporal dynamic safety corridor becomes zero.

[0136] S180: Based on the current shortest distance and the future intrusion time, generate graded early warning information with different handling priorities.

[0137] Optionally, the tiered early warning information includes:

[0138] The first level of warning corresponds to the current shortest distance being less than zero, indicating that an intrusion has occurred;

[0139] The second level of warning corresponds to the current shortest distance being greater than zero but less than the first threshold, and the predicted future intrusion time point being within the next planned inspection cycle;

[0140] The third level of warning corresponds to the current shortest distance being greater than the first threshold but less than the second threshold, or the predicted future intrusion time being after the next planned inspection cycle but within a preset number of years.

[0141] The fourth level of warning corresponds to the current shortest distance being greater than the second threshold and the future intrusion time exceeding the preset number of years, indicating that only routine monitoring is required.

[0142] In the specific implementation of this specification, after the warning information is generated, it is displayed on the GIS map of the management platform using different icons and colors:

[0143] Red lightning bolt icon: indicates intrusion (distance < 0), triggers an audible alarm, automatically generates an emergency response work order, and notifies the team leader via SMS.

[0144] Orange exclamation mark icon: indicates high risk (intrusion within 0-2 meters and predicted within 3 months), the work order will be added to the "Weekly Plan" list.

[0145] Yellow triangle icon: indicates medium risk (distance within 2-4 meters, or predicted intrusion within 3-12 months). The work order will be added to the "Monthly Plan" list, and you will be prompted to review it carefully during the next inspection.

[0146] Blue dot icon: indicates low risk (distance > 4 meters and intrusion predicted for more than 1 year), recorded only in the electronic log and included in the scope of regular patrols.

[0147] This tiered approach allows limited operational resources to be precisely allocated to the highest priority potential hazards.

[0148] This invention employs a parallel feature extraction-arbitration fusion mechanism and establishes a closed-loop process of guided segmentation-backprojection verification-iterative optimization. This allows for deep and dynamic mutual verification and enhancement of two-dimensional texture information and three-dimensional geometric information, fundamentally overcoming the shortcomings of high false detection rates and poor anti-interference capabilities associated with single data sources or simple fusion methods. From data input to early warning output, the entire process requires no manual intervention. In particular, the dynamic parameter adjustment strategy based on quantization error enables the system to possess adaptive optimization capabilities, automatically adjusting to the optimal processing state for different scenarios, greatly enhancing the method's universality and practicality. Breaking through the limitations of static thresholds, by integrating dynamic factors from multiple physical fields such as electrical safety, wind-induced sway, and thermally induced sag, a spatiotemporal dynamic safety corridor is constructed. This ensures that risk assessment results accurately reflect the actual risk status of line operation, significantly improving the scientific rigor and accuracy of early warnings. By introducing historical time-series data and growth prediction models, not only can current risks be assessed, but the future intrusion risk time of trees can also be proactively predicted, providing valuable early warning leads for maintenance departments and realizing the transformation and upgrading of transmission line maintenance from passive response to proactive defense. The final output of tiered early warning information transforms abstract detection data into handling suggestions of different urgency levels, and links them to inspection cycles and planned lifespans. This can directly guide the prioritization and resource allocation of on-site operation and maintenance work, greatly improving management efficiency.

[0149] Figure 2 This is a schematic diagram of a tree obstacle detection device based on computer vision provided in an embodiment of this specification. The device may include:

[0150] Data acquisition module 10 is used to acquire synchronous multispectral image data and laser point cloud data of the power transmission channel;

[0151] The fusion perception module 20 is used to perform parallel feature extraction and spatiotemporal correlation on the multispectral image data and the laser point cloud data, and resolve identification conflicts according to preset arbitration rules to generate a unified three-dimensional semantic segmentation result that includes tree targets and guide wire targets.

[0152] The guided segmentation module 30 is used to apply physical constraints to the wire point cloud for three-dimensional reconstruction based on the three-dimensional semantic segmentation result, and simultaneously perform single-tree segmentation of the tree point cloud by coupling spatial and spectral features to obtain an initial three-dimensional scene model.

[0153] The verification module 40 is used to backproject the single-tree segmentation results in the initial three-dimensional scene model to the multispectral image data, perform contour consistency verification, and generate verification results.

[0154] The iterative control module 50 is used to iteratively adjust the parameters used in the single-tree segmentation process in the guided segmentation module based on the verification results, and control the guided segmentation module and the verification module to be executed repeatedly until the contour consistency meets the preset conditions, and output the final three-dimensional scene model.

[0155] The dynamic corridor construction module 60 is used to construct a spatiotemporal dynamic safety corridor based on the 3D model of the conductor in the final 3D scene model, combined with electrical safety distance, wind-induced sway envelope and thermal sag dynamics.

[0156] The calculation and prediction module 70 is used to calculate the current shortest distance between each tree 3D model in the final 3D scene model and the safety corridor, and to predict the time point when each tree will invade the corridor in the future based on historical time series data.

[0157] The early warning generation module 80 is used to generate graded early warning information with different handling priorities based on the current shortest distance and the future intrusion time.

[0158] Optionally, the preset arbitration rules include:

[0159] When the identification result for the same region is identified as category 1 in the multispectral image data and category 2 in the laser point cloud data:

[0160] If the first or second category involves vegetation, the identification results of the laser point cloud data shall be adopted first.

[0161] If the first category and the second category involve different power line components, the identification results of the multispectral image data shall be adopted first, and their morphological features shall be verified to conform to the preset component geometric template.

[0162] Optionally, the physical constraints applied to the conductor point cloud for 3D reconstruction include at least one of the following:

[0163] Continuity constraints of the conductor in three-dimensional space;

[0164] Catenary or approximate linear model constraints;

[0165] Electric field distribution constraints based on line voltage level and topology simulation are used to distinguish conductors of different phases in multi-circuit line scenarios.

[0166] Optionally, the guiding segmentation module 30 includes:

[0167] The tree point cloud in the three-dimensional semantic segmentation result is used as the initial seed point;

[0168] Based on the region growing algorithm, its growing criteria simultaneously include the Euclidean distance threshold in three-dimensional space and the color space similarity threshold mapped from the multispectral image data.

[0169] Once growth ceases, each connected point cluster formed will be identified as an independent tree.

[0170] Optionally, the iteration control module 50 includes:

[0171] Based on the quantization error generated by contour consistency verification, the distance threshold and / or color similarity threshold used in the single-tree segmentation process are dynamically adjusted; wherein, when the quantization error increases, the threshold is tightened to perform over-segmentation; when the quantization error decreases, the threshold is relaxed to perform under-segmentation correction.

[0172] Optionally, the dynamic corridor construction module 60 includes:

[0173] Using the normal direction of each point on the three-dimensional model of the conductor as a reference, a dynamic offset vector is superimposed to form the corridor boundary;

[0174] The dynamic offset vector is obtained by vector synthesis of the basic electrical safety distance, the maximum wind deflection amplitude calculated based on historical wind speed and conductor parameters, and the sag change value calculated based on real-time ambient temperature and conductor thermal expansion coefficient.

[0175] Optionally, the calculation and prediction module 70 includes:

[0176] Obtain 3D models of the same tree at at least two historical time points, and fit them to obtain the growth rate of its tree height or canopy volume;

[0177] Identify the tree species and obtain the typical seasonal growth coefficient of the tree species;

[0178] By combining the growth rate and the seasonal growth coefficient, a time series prediction model is established to predict the evolution of the tree's three-dimensional model within a specified future time period, and to calculate the time point at which the shortest distance between the tree and the boundary of the spatiotemporal dynamic safety corridor becomes zero.

[0179] Optionally, the early warning generation module 80 includes:

[0180] The first level of warning corresponds to the current shortest distance being less than zero, indicating that an intrusion has occurred;

[0181] The second level of warning corresponds to the current shortest distance being greater than zero but less than the first threshold, and the predicted future intrusion time point being within the next planned inspection cycle;

[0182] The third level of warning corresponds to the current shortest distance being greater than the first threshold but less than the second threshold, or the predicted future intrusion time being after the next planned inspection cycle but within a preset number of years.

[0183] The fourth level of warning corresponds to the current shortest distance being greater than the second threshold and the future intrusion time exceeding the preset number of years, indicating that only routine monitoring is required.

[0184] The functions of the apparatus in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0185] Based on the same inventive concept, embodiments of this specification also provide an electronic device.

[0186] The following describes embodiments of the electronic device of the present invention, which can be considered as specific implementations of the methods and apparatus embodiments of the present invention described above. Details described in the embodiments of the electronic device of the present invention should be considered as supplements to the methods or apparatus embodiments described above; details not disclosed in the embodiments of the electronic device of the present invention can be implemented with reference to the methods or apparatus embodiments described above.

[0187] Figure 3 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 3 The electronic device 300 according to this embodiment of the present invention will be described. Figure 3 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0188] like Figure 3 As shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including storage unit 320 and processing unit 310), a display unit 340, etc.

[0189] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform, for example... Figure 1 The steps are shown.

[0190] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only memory unit (ROM) 3203.

[0191] The storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0192] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0193] Electronic device 300 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable viewers to interact with electronic device 300, and / or with any device that enables electronic device 300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. Network adapter 360 can communicate with other modules of electronic device 300 via bus 330. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the method described above according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the method described above, i.e.: as... Figure 1 The method shown.

[0195] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.

[0196] accomplish Figure 1 The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0197] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0198] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the audience's computing device, partially on the audience's device, as a standalone software package, partially on the audience's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the audience's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0199] In summary, the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0200] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0201] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0202] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting tree obstacles in power transmission line corridors based on computer vision, characterized in that, include: Acquire multispectral image data and laser point cloud data of the power transmission channel; Parallel feature extraction and spatiotemporal correlation are performed on the multispectral image data and the laser point cloud data, and recognition conflicts are resolved according to preset arbitration rules to generate a unified three-dimensional semantic segmentation result that includes tree targets and guide wire targets. Guided by the three-dimensional semantic segmentation results, physical constraints are applied to the wire point cloud for three-dimensional reconstruction, and at the same time, the tree point cloud is segmented into individual trees with spatial and spectral features coupled to obtain an initial three-dimensional scene model. The single-tree segmentation results in the initial 3D scene model are back-projected into the multispectral image data to verify the contour consistency and generate verification results. Based on the verification results, the parameters used in the single-tree segmentation process are iteratively adjusted, and the guided segmentation step and verification feedback step are repeated until the contour consistency meets the preset conditions, and the final three-dimensional scene model is output. Based on the 3D model of the conductor in the final 3D scene model, and combined with electrical safety distance, wind-induced sway envelope and thermal sag dynamics, a spatiotemporal dynamic safety corridor is constructed. Calculate the current shortest distance between each tree's 3D model in the final 3D scene model and the safety corridor, and predict the time point when each tree will invade the corridor in the future based on historical time series data; Based on the current shortest distance and the future intrusion time, hierarchical early warning information with different handling priorities is generated.

2. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, The preset arbitration rules include: When the identification result for the same region is identified as category 1 in the multispectral image data and category 2 in the laser point cloud data: If the first or second category involves vegetation, the identification results of the laser point cloud data shall be adopted first. If the first category and the second category involve different power line components, the identification results of the multispectral image data shall be adopted first, and their morphological features shall be verified to conform to the preset component geometric template.

3. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, The physical constraints applied to the conductor point cloud for 3D reconstruction include at least one of the following: Continuity constraints of the conductor in three-dimensional space; Catenary or approximate linear model constraints; Electric field distribution constraints based on line voltage level and topology simulation are used to distinguish conductors of different phases in multi-circuit line scenarios.

4. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, The single-tree segmentation of tree point clouds by coupling spatial and spectral features includes: The tree point cloud in the three-dimensional semantic segmentation result is used as the initial seed point; Based on the region growing algorithm, its growing criteria simultaneously include the Euclidean distance threshold in three-dimensional space and the color space similarity threshold mapped from the multispectral image data. Once growth ceases, each connected point cluster formed will be identified as an independent tree.

5. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, Based on the verification results, the parameters used in the single-tree segmentation process in the guided segmentation module are iteratively adjusted, including: Based on the quantization error generated by contour consistency verification, the distance threshold and / or color similarity threshold used in the single-tree segmentation process are dynamically adjusted; wherein, when the quantization error increases, the threshold is tightened to perform over-segmentation; when the quantization error decreases, the threshold is relaxed to perform under-segmentation correction.

6. The tree obstacle detection method in power transmission line corridors based on computer vision as described in claim 1, characterized in that, The construction of the spatiotemporal dynamic security corridor includes: Using the normal direction of each point on the three-dimensional model of the conductor as a reference, a dynamic offset vector is superimposed to form the corridor boundary; The dynamic offset vector is obtained by vector synthesis of the basic electrical safety distance, the maximum wind deflection amplitude calculated based on historical wind speed and conductor parameters, and the sag change value calculated based on real-time ambient temperature and conductor thermal expansion coefficient.

7. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, The prediction of the future intrusion time points of each tree into the corridor based on historical time-series data includes: Obtain 3D models of the same tree at at least two historical time points, and fit them to obtain the growth rate of its tree height or canopy volume; Identify the tree species and obtain the typical seasonal growth coefficient of the tree species; By combining the growth rate and the seasonal growth coefficient, a time series prediction model is established to predict the evolution of the tree's three-dimensional model within a specified future time period, and to calculate the time point at which the shortest distance between the tree and the boundary of the spatiotemporal dynamic safety corridor becomes zero.

8. The tree obstacle detection method based on computer vision in power transmission line corridors as described in claim 1, characterized in that, The tiered early warning information includes: The first level of warning corresponds to the current shortest distance being less than zero, indicating that an intrusion has occurred; The second level of warning corresponds to the current shortest distance being greater than zero but less than the first threshold, and the predicted future intrusion time point being within the next planned inspection cycle; The third level of warning corresponds to the current shortest distance being greater than the first threshold but less than the second threshold, or the predicted future intrusion time being after the next planned inspection cycle but within a preset number of years. The fourth level of warning corresponds to the current shortest distance being greater than the second threshold and the future intrusion time exceeding the preset number of years, indicating that only routine monitoring is required.

Images