A method and apparatus for monitoring tree growth for power lines

Abstract

The application provides a tree growth monitoring method and device for a power transmission line. The method inputs a monitoring image into a preset target segmentation model to obtain a target region image. The target segmentation model is trained by a plurality of tree sample images. The target region image includes at least tree region pixel points. By using a preset coordinate conversion matrix and the target region image, a tree point cloud space in corresponding point cloud data is determined. The corresponding point cloud data is synchronously collected with the monitoring image. Based on the tree point cloud space and a historical tree point cloud space, a corresponding tree difference space is determined. Based on the tree difference space and the tree point cloud space subjected to preset segmentation processing, a tree hazard of the power transmission line and a threat level of the tree hazard are determined.

Description

Technical Field

[0001] This application relates to the field of line protection technology, and in particular to a method and device for monitoring tree growth in power transmission lines. Background Technology

[0002] Transmission lines are a crucial component of the power grid. High-altitude transmission lines are often surrounded by forests or areas close to trees. Insufficient safety distance between trees and conductors can easily lead to electrical discharges from the lines to the trees, causing transmission line tripping and outages. During transmission line construction, trees that might affect the normal operation of the lines may be pruned, and regular manual spot checks or inspections are subsequently conducted.

[0003] Currently, manual inspections mainly rely on drones to detect potential tree hazards near power transmission lines. This method is wasteful of manpower, and airborne lidar equipment is expensive. Using drones equipped with lidar to detect tree obstructions is too costly and lacks real-time accuracy. Therefore, how to achieve low-cost, real-time monitoring of potential tree hazards near power transmission lines and prevent tree growth from causing line accidents has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method and device for monitoring tree growth near power transmission lines, which can monitor potential tree hazards near power transmission lines in a low-cost, real-time manner and prevent tree growth from causing line accidents.

[0005] On one hand, embodiments of this application provide a method for monitoring tree growth along power transmission lines, the method comprising:

[0006] The monitoring images are input into a preset target segmentation model to obtain the target area image. The target segmentation model is trained using several tree sample images. The target area image includes at least the pixels of the tree regions. Using a preset coordinate transformation matrix and the target area image, the tree point cloud space in the corresponding point cloud data is determined. The corresponding point cloud data is acquired synchronously with the monitoring images. Based on the tree point cloud space and historical tree point cloud spaces, the corresponding tree difference space is determined. Based on the tree difference space and tree point cloud space after preset segmentation processing, the tree hazards along the transmission line and their threat levels are determined.

[0007] In one implementation of this application, the corresponding tree point cloud data points for each tree region pixel are determined based on a coordinate transformation matrix. The coordinate transformation matrix is ​​a coordinate transformation relationship matrix obtained through the internal and external parameters of the image acquisition device. This matrix represents the transformation relationship between the two-dimensional coordinates of the monitoring image and the three-dimensional coordinates of the point cloud data. The tree point cloud data points and the tree region pixels correspond to the same object. The point cloud space corresponding to each tree point cloud data point is used as the tree point cloud space.

[0008] In one implementation of this application, a preset algorithm is used to compare the historical tree point cloud space with the tree point cloud space to determine the incremental point cloud space of the tree point cloud space, which is the tree difference space. The preset algorithm is either the kd-tree algorithm or the octree algorithm. The intersection of the point clouds of the incremental point cloud space and the historical tree point cloud space is an empty set.

[0009] In one implementation of this application, it is determined whether the spatial increment of the incremental point cloud space is greater than a corresponding first preset threshold. The spatial increment includes at least one or more of the following: volume, height, length, and width. If so, the incremental point cloud space is used as the tree difference space.

[0010] In one implementation of this application, a region growth seed information is generated using a preset region generation algorithm. The region growth seed information includes at least: seed distance and voxel distance. Based on the region growth seed information, the point cloud data is segmented to determine several voxel grids. Based on each voxel grid that meets preset conditions, several seed points are determined. The preset conditions are that the distance between each voxel grid is less than a second preset threshold and the volume of each voxel grid is less than a third preset threshold. Based on the feature similarity of various sub-points, the various sub-points are clustered to determine the point cloud data. Feature similarity includes spatial distance similarity, reflectivity similarity, and normal vector similarity.

[0011] In one implementation of this application, a safety monitoring space for the transmission line is generated based on the transmission voltage level of the transmission line. The safety monitoring space is a point cloud space located at a predetermined safe distance from the transmission line. The overlapping point cloud spaces between the tree point cloud space and the safety monitoring space are determined, and these overlapping point cloud spaces are divided into several point cloud subspaces according to the axial direction of the transmission line and a preset step size. Each point cloud subspace is a cuboid. Each point cloud subspace contains at least one tree point cloud data point. The coordinates of the highest point in each point cloud subspace are determined. The coordinates of the highest point are the coordinates of the tree point cloud data point in the point cloud subspace with the largest vertical distance to the ground. The shortest distance between the highest point coordinates and the corresponding conductor point cloud data point of the transmission line is calculated. Based on the difference between the shortest distance and the pre-segmented tree space, tree hazards are determined and corresponding threat levels are matched. Tree hazards refer to trees.

[0012] In one implementation of this application, the spatial increment corresponding to the tree difference space that has been pre-segmented is determined. The spatial increment is the spatial increment of a pre-defined time interval. A pre-defined threat level interval corresponding to the spatial increment and the shortest distance is matched. Given the matched threat level interval, the trees corresponding to the successfully matched tree difference space are identified as potential tree hazards. The threat level of the potential tree hazard is then determined based on the matching result.

[0013] In one implementation of this application, the target segmentation model is the Mask-RCNN model.

[0014] In one implementation of this application, the monitoring images come from an image acquisition device installed on a power transmission line tower. The point cloud data comes from a lidar system integrated with the image acquisition device.

[0015] On the other hand, embodiments of this application also provide a tree growth monitoring device for power transmission lines, the device comprising:

[0016] At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0017] The monitoring images are input into a preset target segmentation model to obtain the target area image. The target segmentation model is trained using several tree sample images. The target area image includes at least the pixels of the tree regions. Using a preset coordinate transformation matrix and the target area image, the tree point cloud space in the corresponding point cloud data is determined. The corresponding point cloud data is acquired synchronously with the monitoring images. Based on the tree point cloud space and historical tree point cloud spaces, the corresponding tree difference space is determined. Based on the tree difference space and tree point cloud space after preset segmentation processing, the tree hazards along the transmission line and their threat levels are determined.

[0018] This application, through the aforementioned scheme, can utilize segmented target area images to identify tree images within point cloud data, and further determine the degree of threat posed by trees to power transmission lines based on historical point cloud data. This achieves low-cost, real-time monitoring of potential tree hazards near power transmission lines, preventing line accidents caused by tree growth.

[0019] Furthermore, the technical solution of this application, by combining lidar with image acquisition equipment, can timely and fully utilize the advantages of dense image information and the precision of laser point cloud to monitor the spatial differences in trees caused by tree growth and the threat level of tree point cloud space to power transmission lines. At the same time, it solves the shortcoming of long update cycle of UAV point cloud data to a certain extent, and provides more timely, reliable and effective early warning information for the safety of power transmission line channels. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a schematic flowchart of a tree growth monitoring method for power transmission lines, as described in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the joint calibration process in a tree growth monitoring method for power transmission lines according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of coordinate transformation in a tree growth monitoring method for power transmission lines according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the fusion of point cloud data and monitoring images in a tree growth monitoring method for power transmission lines according to an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of a marked tree area in a tree growth monitoring method for power transmission lines according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram illustrating the clearance distance between marked trees and transmission lines in a tree growth monitoring method for power transmission lines according to an embodiment of this application.

[0027] Figure 7 This is another flowchart illustrating a tree growth monitoring method for power transmission lines, as described in this application.

[0028] Figure 8 This is a schematic diagram of a tree growth monitoring method for power transmission lines according to an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] Transmission lines cover vast areas, traverse complex terrains and harsh natural environments, particularly passing through extensive mountainous and forested regions. Trees are a significant contributor to external damage; insufficient safety distance between trees and conductors can easily lead to line-to-tree discharges, causing transmission line tripping and outages. Trees within transmission corridors are increasingly becoming a major problem affecting the safe operation of transmission lines.

[0031] Ensuring a safe distance between power transmission lines and trees, and preventing trees from affecting power transmission lines, has become an urgent technical problem to be solved.

[0032] Based on this, this application provides a method and device for monitoring tree growth near power transmission lines, which can monitor potential tree hazards near power transmission lines in a low-cost and real-time manner, and prevent tree growth from causing line accidents to power transmission lines.

[0033] The various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0034] This application provides a method for monitoring tree growth along power transmission lines, such as... Figure 1 As shown, the method may include steps S101-S105:

[0035] S101, the server determines the coordinate transformation matrix between the image acquisition device and the LiDAR based on the internal and external parameters of the image acquisition device, so as to perform joint calibration of the image acquisition device and the LiDAR.

[0036] The image acquisition device and lidar are integrated into a single unit and installed on the power transmission line tower to collect images and point clouds along the direction of the power transmission line. The image acquisition device can be a camera.

[0037] It should be noted that the server, as the execution subject of the tree growth monitoring method for power transmission lines, is only an example. The execution subject is not limited to the server. For example, a controller such as a microcontroller can be used. The execution subject can be the controller at the device end of the image acquisition device and the lidar. This application does not make any specific limitations in this regard.

[0038] The aforementioned joint calibration steps only need to be performed once within the data acquisition period, or, according to user settings, the joint calibration can be performed once at a fixed time period. This application does not impose specific limitations on the time interval and number of joint calibrations.

[0039] The specific steps of its joint calibration are as follows: Figure 2 As shown, the specific process is as follows:

[0040] First, calibration point cloud data of the power transmission line corridor scene is collected by lidar, and calibration scene images of the power transmission line corridor scene are collected simultaneously by image acquisition equipment.

[0041] Next, the image acquisition device was calibrated using Zhang Zhengyou's calibration method to obtain its internal parameters, including f. x f y c is the camera focal length parameter. x c y Camera optical center parameters.

[0042] Subsequently, the calibration point cloud data is read, and based on the calibration scene image, feature points in the image are selected. At the same time, the corresponding feature point positions in the calibration point cloud data are found, and multiple pairs of points are selected (at least 4 pairs of points are selected).

[0043] Subsequently, using the coordinates of the calibrated point cloud data as world coordinates, a camera pose estimation algorithm is employed to combine the 3D point cloud coordinates with the pixel coordinates of the calibrated scene image from the image acquisition device, thereby obtaining the external parameters (rotation and translation) of the point cloud and the camera.

[0044] Then, based on the obtained internal and external parameters, a joint calculation is performed to calculate and output the spatial coordinate transformation relationship of 3D-2D (point cloud-image).

[0045] The spatial coordinate transformation relationship is described as follows:

[0046] The origin coordinate system of the laser point cloud is used as the unified world coordinate system. Defined as X... W Y W Z W The unit is a unit of length. The coordinate system of the image acquisition device has the optical center as the origin of the camera coordinate system, and points parallel to the image (such as...). Figure 3 As shown, the x and y directions are the Xc and Yc axes, respectively. The Zc axis is parallel to the optical axis, and Xc, Yc, and Zc are perpendicular to each other. The unit is length. The image physical coordinate system has its origin at the intersection of the principal optical axis and the image plane. The x and y directions are as follows: Figure 3 As shown, the unit is a unit of length.

[0047] The image pixel coordinate system has the vertex of the image as the origin, and the u and v directions are parallel to the x and y directions. The unit is pixels.

[0048] The coordinate transformation matrix of this application is as follows:

[0049]

[0050] Where R is the rotation matrix (3 degrees of freedom), t is the translation matrix, and the two together form a 3×4 matrix, which is the external parameter matrix of the image acquisition device. xf y c x c y These are the internal parameters of the image acquisition device. The schematic diagram showing the fusion effect of the monitoring image and point cloud data after the above joint calibration is shown below. Figure 4 As shown.

[0051] S102, the server inputs the monitoring image into the preset target segmentation model to obtain the target area image.

[0052] The target segmentation model was trained using several tree sample images. The target region image must include at least the pixels representing the tree regions. The target segmentation model is a Mask-RCNN model.

[0053] In this embodiment, the monitoring images come from image acquisition devices installed on power transmission line towers. The tree sample images can be images of trees along the power transmission line corridor obtained by the user via the Internet, used to generate a sample image set, which is then used to train the Mask-RCNN model, enabling it to identify and segment tree regions in two-dimensional images.

[0054] The target region image contains at least one pixel representing a tree region, such as Figure 5 As shown, 501, 502, 503, and 504 are images of the target area.

[0055] In this embodiment of the application, the point cloud data needs to be preprocessed and clustered. Therefore, before determining the spatial distribution of tree point clouds in the corresponding point cloud data using a preset coordinate transformation matrix and a target region image, the process further includes:

[0056] First, the server generates region growth seed information using a preset region generation algorithm. This region growth seed information includes at least: seed distance and voxel distance.

[0057] In other words, the seeds are first arranged in a regular pattern to grow in the designated areas, with the seed distance and voxel distance specified.

[0058] Then, the server segments the point cloud data based on the regional growth seed information to determine several voxel grids.

[0059] Voxel connection segmentation of point cloud is performed using voxel distance.

[0060] Next, based on each voxel grid that meets the preset conditions, several seed points are determined. The preset conditions are that the distance between each voxel grid is less than a second preset threshold and the volume of each voxel grid is less than a third preset threshold.

[0061] In other words, seed points are obtained by searching for adjacent leaves in the voxel grid and merging excessively small voxel particles into the nearest large voxel particles. The second and third preset thresholds are set manually, and this application does not impose specific limitations on them.

[0062] Finally, the server clusters the various sub-points based on their feature similarity to determine the point cloud data. Feature similarity includes spatial distance similarity, reflectivity similarity, and normal vector similarity.

[0063] In other words, the expansion from the seed point involves a multi-feature metric control. These multi-features consist of spatial distance, reflectivity, and normal vectors within a feature space. Spatial information is used to extract shape features and normal information. An adjacency graph is used to determine which neighboring seed points are included in the normal calculation (feature similarity greater than a preset value), thereby clustering the 3D point cloud data into blocks and completing the clustering and segmentation process to obtain the clustered point cloud data.

[0064] S103, the server determines the spatial distribution of tree point clouds in the corresponding point cloud data by using a preset coordinate transformation matrix and the target area image.

[0065] The corresponding point cloud data was acquired synchronously with the monitoring images. This point cloud data originates from a LiDAR system integrated with the image acquisition equipment.

[0066] In this embodiment of the application, the server determines the spatial distribution of tree point clouds in the corresponding point cloud data by using a preset coordinate transformation matrix and a target area image, specifically including:

[0067] The server uses a coordinate transformation matrix to determine the corresponding tree point cloud data points for each tree region pixel. The coordinate transformation matrix is ​​a matrix derived from the internal and external parameters of the image acquisition device. It represents the transformation between the two-dimensional coordinates of the monitoring image and the three-dimensional coordinates of the point cloud data. Each tree point cloud data point and each tree region pixel corresponds to the same object (tree). The server uses the point cloud space corresponding to each tree point cloud data point as the tree point cloud space.

[0068] In other words, the server can Figure 5 In determining the area of ​​trees, in such Figure 4 In the point cloud space after fusing the point cloud with the image, identification is performed, and the corresponding point cloud space is used as the tree point cloud space to calculate the shortest distance between the tree point cloud data points and the power transmission line point cloud data points.

[0069] S104, the server determines the corresponding tree difference space based on the tree point cloud space and the historical tree point cloud space.

[0070] Specifically, the server determines the corresponding tree difference space based on the tree point cloud space and the historical tree point cloud space, including:

[0071] The server uses algorithms such as kd-tree and octree to compare the historical tree point cloud space with the existing tree point cloud space. It checks for matches and discrepancies in the point cloud data points to determine the incremental point cloud space, known as the tree difference space. The intersection of the incremental point cloud space and the historical tree point cloud space is an empty set.

[0072] In other words, using the aforementioned preset algorithm, a point cloud space that differs from the historical tree point cloud space can be obtained; that is, the point cloud space of the area where the trees have grown since the last collection. The complement of the historical tree point cloud space to the point cloud space of the tree point cloud space constitutes the incremental point cloud space, and the intersection of the incremental point cloud space and the historical tree point cloud space is the empty set. The historical tree point cloud space can be the tree point cloud space collected at preset time intervals, such as one day or one month.

[0073] In this embodiment of the application, the server determines the incremental point cloud space of the tree point cloud space as the tree difference space, specifically including:

[0074] First, the server determines whether the spatial increment of the incremental point cloud space is greater than a corresponding first preset threshold. The spatial increment includes at least one or more of the following: volume, height, length, and width. If the spatial increment of the incremental point cloud space is determined to be greater than the corresponding first preset threshold, the incremental point cloud space is used as the tree difference space.

[0075] The first preset threshold is set by the user, and this application does not impose specific limitations on it. The spatial increment of the incremental point cloud space can be the volume increment, height increment, length increment, or width increment calculated based on coordinates. For example, if the spatial increment of a tree is 1 meter and the first preset threshold is 0.5 meters, then this incremental point cloud space is regarded as the tree difference space.

[0076] The server can pre-determine the spatial volume of the grid according to the pre-set grid space division principle. Based on the grid with the pre-set spatial volume, the differential space of the trees is divided. The growth range is calculated by the specific spatial increment obtained from the division and the pre-set time interval. The growth range is the spatial increment divided by the time interval.

[0077] If the spatial increment is not greater than the corresponding first preset threshold, then it will be considered that the growth range of the tree corresponding to the incremental point cloud space does not meet the requirements for dividing tree differences, that is, its growth range is low.

[0078] S105, the server determines the tree hazards and threat levels of the power transmission line based on the tree difference space and tree point cloud space that have been pre-segmented and processed.

[0079] In this embodiment of the application, the server determines the potential tree hazards and threat levels of the power transmission line based on the tree difference space and tree point cloud space that have been pre-segmented and processed, specifically including:

[0080] First, the server generates a safety monitoring space for the power transmission line based on its voltage level. This safety monitoring space is a point cloud space located at a predetermined safe distance from the power transmission line.

[0081] In other words, the server pre-stores the power transmission voltage level of the power transmission line collected by the image acquisition device and the lidar. Each power transmission voltage level corresponds to a safety monitoring space with different set distances. For example, if the power transmission voltage level is 220V, the predetermined safety distance of its safety monitoring space is 5 meters. The point cloud space within 5 meters of the power transmission line constitutes the safety monitoring space.

[0082] Secondly, the server determines the overlapping point cloud space between the tree point cloud space and the security monitoring space, and divides the overlapping point cloud space into several point cloud subspaces according to the axial direction of the transmission line and a preset step size. Each point cloud subspace is a cuboid. Each point cloud subspace contains at least one tree point cloud data point.

[0083] The overlapping point cloud space can be understood as the intersection of the point cloud set corresponding to the tree point cloud space and the point cloud set of the security monitoring space. The point cloud set corresponding to the intersection is taken as the overlapping point cloud space. This overlapping point cloud space may or may not include incremental point cloud spaces. Each point cloud subspace is adjacent and contains tree point cloud data points. At least one side of each point cloud subspace has a preset step size and is parallel to the axial direction of the transmission line. The vertical side length of each point cloud subspace is the distance from the point cloud data point with the smallest vertical distance to the ground to the point cloud data point with the largest vertical distance to the ground in the overlapping point cloud space.

[0084] Next, the server determines the coordinates of the highest point in each point cloud subspace. The coordinates of the highest point are the coordinates of the tree point cloud data point with the largest vertical distance from the ground in the point cloud subspace.

[0085] The highest point coordinates can be understood as the highest (with the largest vertical distance from the ground) tree point cloud data point within each point cloud subspace along the cable direction of the power transmission line.

[0086] Then, the server calculates the shortest distance between the coordinates of the highest point and the corresponding conductor point cloud data points of the transmission line.

[0087] The formula for calculating the shortest distance can be the formula for the distance between two points or the Pythagorean theorem. This application does not impose specific limitations on the method of calculating the shortest distance.

[0088] Furthermore, this application can generate distance-marked images after calculating the shortest distance and send them to the corresponding user terminals. The user terminals can be personnel responsible for pruning trees or inspecting the safety of power transmission lines. Through the distance-marked images of each power transmission line, trees can be managed. Figure 6 As shown.

[0089] Finally, the server determines tree hazards and matches them with corresponding threat levels based on the spatial difference between the shortest distance and the pre-defined segmented trees. Tree hazards refer to trees.

[0090] Specifically, the server determines the spatial increment corresponding to the spatial differences between the trees that have been pre-segmented. The spatial increment is the spatial increment over a pre-defined time interval.

[0091] Next, the preset threat level ranges corresponding to the spatial increment and the shortest distance are matched.

[0092] The preset threat level range can be the range corresponding to the spatial increment and the shortest distance, respectively. For example, if the spatial increment is in the range [a, b] and the shortest distance is in the range [c, d], then the matched threat level range based on the ranges [a, b] and [c, d] is [e, f]. As another example, if the spatial increment is in the range [0, 0.5] meters and the shortest distance is in the range [4, 5] meters, then the matched threat level range will be [0, 0.5 / 30*X1], where X1 is the threat level value corresponding to the shortest distance range [4, 5] meters, and 0.5 / 30 refers to the threat level weight of the spatial increment within a preset time interval, such as 30 days. In other words, different threat level ranges can be determined based on different spatial increments and shortest distances.

[0093] Once the corresponding threat level ranges are matched, the trees corresponding to the successfully matched tree differences are identified as potential tree hazards. Based on the matching results, the threat level of these potential tree hazards is determined.

[0094] Upon matching a threat level range, the server can identify the trees corresponding to the tree difference space as potential threats. Based on pre-defined relationships, it will match the threat level of the range from a pre-set database and send an alert to the appropriate user terminal, such as a mobile phone or computer. The alert information may include SMS notifications or image notifications. Figure 6 The image shown is sent to the user's terminal.

[0095] This application, through the aforementioned scheme, can utilize segmented target area images to identify tree images within point cloud data. Based on historical point cloud data, it further determines the tree growth rate and the distance between the tree's highest point and the power line, thereby determining the degree of threat posed by the trees to the transmission line. This achieves low-cost, real-time monitoring of potential tree hazards near transmission lines, preventing line accidents caused by tree growth. The lidar used in this application is positioned on the transmission line, reducing monitoring costs. Furthermore, in determining potential tree hazards, the spatial difference between trees is segmented into subspaces. First, the highest point is determined, and then the shortest distance is calculated, reducing computational load and improving the efficiency of hazard identification and threat level assessment.

[0096] This application can promptly utilize the advantages of dense image information and the precision of laser point cloud to achieve accurate distance calculation from trees to power lines, with a relative distance measurement error of less than 5%. It can also compare and analyze the changes in tree growth based on historical point cloud data, realizing real-time and accurate monitoring of tree growth. To a certain extent, it solves the shortcoming of the long update cycle of UAV point cloud data, and provides more timely, reliable and effective early warning information for the safety of power transmission line channels.

[0097] In addition, in one embodiment of this application, the server can also separately determine the shortest distance between the highest point of the tree point cloud space and the power transmission line, and the shortest distance between the highest point of the tree difference space and the power transmission line, thereby obtaining the tree hazard and threat level. This application does not make specific limitations on this.

[0098] This application relates to a method for monitoring tree growth along power transmission lines, used to monitor the threat posed by trees to power transmission lines. The specific process is as follows: Figure 7 As shown below:

[0099] Using an integrated acquisition device, images and point clouds are periodically acquired (with configurable acquisition frequency). The acquired images are identified, and tree regions are segmented to obtain 2D tree regions. The point cloud is then segmented and clustered, and finally, the point cloud and tree regions are fused and projected to generate the transmission line's corridor protection zone (i.e., safety monitoring space), thus obtaining 3D tree regions. The 3D tree regions are further segmented into several grid spatial regions (point cloud subspaces) according to a set step size, and the highest point of the point cloud within each tree region is obtained. Simultaneously, point cloud comparison and analysis are performed to determine if point cloud changes exceed a certain range, i.e., whether the spatial increment exceeds a corresponding first preset threshold. If so, the highest point of the changed area is obtained. In three-dimensional space, the clearance distance between the highest point and the conductor is calculated, and the hazard threat level is output.

[0100] Figure 8 A schematic diagram of a tree growth monitoring device for power transmission lines provided in this application embodiment, the device comprising:

[0101] At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0102] The monitoring images are input into a preset target segmentation model to obtain the target area image. The target segmentation model is trained using several tree sample images. The target area image includes at least the pixels of the tree regions. Using a preset coordinate transformation matrix and the target area image, the tree point cloud space in the corresponding point cloud data is determined. The corresponding point cloud data is acquired synchronously with the monitoring images. Based on the tree point cloud space and historical tree point cloud spaces, the corresponding tree difference space is determined. Based on the tree difference space and tree point cloud space after preset segmentation processing, the tree hazards along the transmission line and their threat levels are determined.

[0103] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0104] The devices and methods provided in this application are one-to-one correspondences. Therefore, the devices also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices will not be repeated here.

[0105] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0106] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for monitoring tree growth for a power transmission line, characterized by, The method includes: The monitoring image is input into a preset target segmentation model to obtain a target region image; wherein, the target segmentation model is trained using several tree sample images; the target region image includes at least tree region pixels; and the target segmentation model is a Mask-RCNN model. The spatial distribution of tree point clouds in the corresponding point cloud data is determined by using a preset coordinate transformation matrix and the target area image; the corresponding point cloud data is acquired synchronously with the monitoring image; the coordinate transformation matrix is ​​a coordinate transformation relationship matrix obtained by using the internal and external parameters of the image acquisition device; the coordinate transformation relationship matrix is ​​a transformation relationship matrix between the two-dimensional coordinates of the monitoring image and the three-dimensional coordinates of the point cloud data; Based on the tree point cloud space and the historical tree point cloud space, a corresponding tree difference space is determined; specifically, this includes: comparing the historical tree point cloud space with the tree point cloud space to determine the incremental point cloud space of the tree point cloud space, which is the tree difference space; the point cloud intersection of the incremental point cloud space and the historical tree point cloud space is an empty set. Based on the tree difference space and tree point cloud space that have been pre-segmented, the tree hazards of the transmission line and the threat level of the tree hazards are determined; wherein, according to the pre-set grid space small area division principle, the spatial volume of the grid is preset, and the tree difference space is segmented according to the grid of the preset spatial volume.

2. The method according to claim 1, characterized in that, By using a preset coordinate transformation matrix and the target region image, the spatial distribution of tree point clouds in the corresponding point cloud data is determined, specifically including: Based on the coordinate transformation matrix, the corresponding tree point cloud data points for each tree region pixel are determined; the tree point cloud data points and the tree region pixels correspond to the same object; The point cloud space corresponding to each of the tree point cloud data points is taken as the tree point cloud space.

3. The method according to claim 1, characterized in that, Based on the tree point cloud space and the historical tree point cloud space, the corresponding tree difference space is determined, specifically including: The historical tree point cloud space is compared with the tree point cloud space using a preset algorithm to determine the incremental point cloud space of the tree point cloud space, which is the tree difference space; the preset algorithm is a kd-tree algorithm or an octree algorithm.

4. The method according to claim 3, characterized in that, Determining the incremental point cloud space of the tree point cloud space, which is the tree difference space, specifically includes: Determine whether the spatial increment of the incremental point cloud space is greater than a corresponding first preset threshold; the spatial increment includes at least one or more of the following: volume, height, length, and width; If so, the incremental point cloud space is used as the tree difference space.

5. The method according to claim 1, characterized in that, Before determining the tree point cloud space in the corresponding point cloud data using a preset coordinate transformation matrix and the target region image, the method further includes: A preset region generation algorithm is used to generate region growth seed information; wherein, the region growth seed information includes at least: seed distance and voxel distance; Based on the seed growth information in the region, the point cloud data is segmented to determine several voxel grids; Based on each voxel grid that meets the preset conditions, a number of seed points are determined; wherein, the preset conditions are that the distance between each voxel grid is less than a second preset threshold and the volume of each voxel grid is less than a third preset threshold. Based on the feature similarity of each seed point, the seed points are clustered to determine the point cloud data; the feature similarity includes spatial distance similarity, reflectivity similarity, and normal vector similarity.

6. The method according to claim 1, characterized in that, Based on the tree difference space and tree point cloud space that have been pre-segmented, the potential tree hazards along the transmission line and the threat level of these hazards are determined, specifically including: Based on the transmission voltage level of the transmission line, a safety monitoring space for the transmission line is generated; the safety monitoring space is a point cloud space at a predetermined safety distance from the transmission line. The overlapping point cloud space between the tree point cloud space and the security monitoring space is determined, and the overlapping point cloud space is divided into several point cloud subspaces according to the axial direction of the transmission line and a preset step size; each point cloud subspace is a cuboid; each point cloud subspace contains at least one tree point cloud data point. Determine the coordinates of the highest point in each of the point cloud subspaces; the coordinates of the highest point are the coordinates of the tree point cloud data point with the largest vertical distance from the ground in the point cloud subspace. Calculate the shortest distance between the coordinates of the highest point and the corresponding conductor point cloud data point of the transmission line; Based on the difference space between the shortest distance and the tree that has been pre-segmented, the tree hazard is determined and the corresponding threat level is matched; the tree hazard is a tree.

7. The method according to claim 6, characterized in that, Based on the difference space between the shortest distance and the pre-segmented tree data, the potential tree hazards and corresponding threat levels are determined, specifically including: Determine the spatial increment corresponding to the spatial difference of the trees that have been pre-segmented; the spatial increment is the spatial increment of a pre-defined time interval. Match the spatial increment with the preset threat level range corresponding to the shortest distance; If a matching threat level range is obtained, the tree corresponding to the successfully matched tree difference space is identified as the tree hazard; and the threat level of the tree hazard is determined based on the matching result.

8. The method according to claim 1, characterized in that, The monitoring images are from image acquisition devices installed on the power transmission line towers; The point cloud data comes from a lidar integrated with the image acquisition device.

9. A tree growth monitoring device for power transmission lines, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a tree growth monitoring method for power transmission lines as described in any one of claims 1-8.

Images