Vegetation risk hidden danger detection method and system based on sparse point cloud segmentation

Abstract

The invention relates to the technical field of power detection, and discloses a vegetation risk hidden danger detection method and system based on sparse point cloud segmentation, and the method comprises the steps: collecting image data of a monitoring region at different angles, generating sparse point cloud data, and carrying out the preprocessing of the sparse point cloud data; and performing point cloud segmentation on the sparse point cloud data through a neural network, dividing the monitoring region according to region types, analyzing a spatial relationship between vegetation and power equipment in a target region, performing risk assessment, and generating an early warning signal in combination with multi-modal data. According to the method, the sparse point cloud data is precisely segmented, the three-dimensional space characteristics of the tree are extracted, and the potential risk of the tree and the power transmission facility can be dynamically monitored in real time by precisely calculating the space relation between the tree and the power transmission facility; according to the multi-modal fusion data, multi-level early warning information is generated, so that the efficiency and precision of power transmission line inspection are greatly improved, and the pre-judgment and timely treatment of hidden dangers are realized.

Description

Technical Field

[0001] The present invention relates to the field of electric power detection technology, and in particular to a vegetation risk hidden danger detection method and system based on sparse point cloud segmentation. Background Art

[0002] With the rapid development of the power industry, especially the expansion of transmission lines, vegetation management and risk detection have become increasingly important in ensuring the safe operation of power facilities. If the vegetation growth in the transmission line corridor is not monitored and managed in a timely manner, it may have a serious impact on the power system and even cause power failures, fires and other safety accidents. In particular, if trees, shrubs and other vegetation along high-voltage transmission lines grow too high or too dense, it is easy to cause line obstruction, wire contact or arcing, which poses a huge hidden danger to power equipment and personal safety. Therefore, how to efficiently and accurately detect vegetation risk hazards in transmission line corridors has become a technical problem that needs to be solved urgently in the power industry.

[0003] Most existing vegetation monitoring methods rely on manual inspections or remote sensing data analysis. Although these methods can detect vegetation growth problems to a certain extent, they have obvious shortcomings in practical applications. Manual inspections are inefficient, costly, and difficult to provide timely warnings, especially in remote areas or areas with complex environments. Inspections are difficult. Although remote sensing technology can provide relatively comprehensive coverage, it has poor adaptability to environmental changes (such as light, meteorological conditions, etc.), and it is difficult to accurately determine the specific spatial relationship between vegetation and transmission lines, making it difficult to make accurate risk assessments.

[0004] With the rapid development of computer vision and deep learning technology, vegetation detection methods based on images and point cloud data have gradually become a research hotspot. Although traditional point cloud processing methods can provide relatively accurate three-dimensional data, their computational efficiency and accuracy are difficult to meet the needs of real-time monitoring and large-scale regional monitoring when faced with sparse point cloud data. In addition, most existing point cloud segmentation technologies are based on dense data, and the special structural characteristics of vegetation and the spatial relationship with transmission lines are often ignored during the processing process, resulting in large errors in the detection of vegetation risk hazards. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a vegetation risk hidden danger detection method based on sparse point cloud segmentation to solve the above problems.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a vegetation risk hazard detection method based on sparse point cloud segmentation, comprising: Collect image data from different angles in the monitoring area, generate sparse point cloud data, and preprocess the sparse point cloud data; Based on the preprocessed sparse point cloud data, perform point cloud segmentation on the sparse point cloud data through a neural network, divide the monitoring area according to the area type, and obtain the target area; Based on the target area, analyze the spatial relationship between the vegetation and power equipment in the target area, and conduct a risk assessment to obtain the first risk assessment result; Combine multi-modal data to optimize the first risk assessment result, evaluate the risk level according to the optimization result, and generate a warning signal.

[0007] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: collecting image data from different angles in the monitoring area and generating sparse point cloud data includes: Set the height and angle of image acquisition, and perform image acquisition of different angles and different vegetation types according to the preset acquisition frequency and acquisition quantity; Based on the collected images, compare the image pairs at different angles in the same scene through a disparity matching algorithm to obtain the disparity map in the images; Based on the disparity map, calculate the corresponding depth information to obtain the depth map, map each pixel point in the image to the corresponding position in the three-dimensional space, and generate sparse point cloud data.

[0008] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: preprocessing the sparse point cloud data includes: Perform denoising processing on the sparse point cloud data to detect and remove the outlier points in the point cloud data; Perform downsampling processing on the sparse point cloud data to obtain the optimized sparse point cloud data, and perform visualization processing to obtain the distribution characteristics of different vegetation types in the point cloud data.

[0009] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: performing point cloud segmentation on the sparse point cloud data through a neural network, dividing the monitoring area according to the area type, and obtaining the target area includes: Collect historical point cloud data of various different vegetation species, and perform primary annotation and secondary annotation on the historical point cloud data. The primary annotation is to annotate the corresponding area type in the point cloud data, and the secondary annotation is to annotate the corresponding structure type in the point cloud data; Build a point cloud segmentation model through a neural network algorithm, and input the annotated historical point cloud data into the point cloud segmentation model for model training; Input the preprocessed sparse point cloud data into a point cloud segmentation model for point cloud segmentation, identify the region types and vegetation types in the point cloud data, determine the target area, and mark the spatial positions of each type of vegetation within the target area.

[0010] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: analyze the spatial relationship between the vegetation and power equipment within the target area, and conduct a risk assessment, and the obtained first risk assessment result includes: Based on the segmented point cloud data, perform 3D modeling on the power facilities, calculate the minimum distance between each vegetation and the power equipment, and use it as the first index for risk assessment; Calculate the vertical distance between each vegetation and the power equipment, and use it as the second index for risk assessment; Predict the future growth and crown data of the vegetation, and use the prediction result as the third index for risk assessment; Combine the first index, the second index, and the third index to conduct a comprehensive risk assessment, obtain the first risk assessment result, and determine whether there is a risk of contact between the vegetation and the power equipment.

[0011] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: combine multi-modal data to optimize the first risk assessment result, including: The multi-modal data includes sparse point cloud data and image data. According to the spatial information of the vegetation, calculate the feature values of the vegetation in the sparse point cloud data and the image data respectively, and compare them with the preset standards to assign weights to the sparse point cloud data and the image data; Based on the assigned weights, fuse and merge the two types of data of the sparse point cloud data and the image data, and further calculate the spatial relationship between the vegetation and the power equipment.

[0012] As a preferred solution of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to the present invention, wherein: evaluate the risk level and generate a warning signal, including: If there is a risk of contact for the vegetation, generate a risk report and report it; Evaluate the risk level according to the optimization result, classify the risk into levels such as high risk, medium risk, and low risk, and take corresponding response measures.

[0013] In a second aspect, the present invention provides a vegetation risk and hidden danger detection system for sparse point cloud segmentation, including: An acquisition and processing module, configured to acquire image data from different angles of a monitoring area, generate sparse point cloud data, and preprocess the sparse point cloud data; A segmentation module, configured to perform point cloud segmentation on the preprocessed sparse point cloud data through a neural network, divide the monitoring area according to area types, and obtain target areas; An evaluation module, configured to analyze the spatial relationship between vegetation and power equipment within the target areas based on the target areas, and perform risk assessment to obtain a first risk assessment result; An output module, configured to optimize the first risk assessment result by combining multi-modal data, evaluate the risk level according to the optimization result, and generate a warning signal.

[0014] In a third aspect, the present invention provides a computer device, including: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the vegetation risk and hidden danger detection method for sparse point cloud segmentation are implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the vegetation risk and hidden danger detection method for sparse point cloud segmentation are implemented.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: By accurately segmenting the sparse point cloud data, the present invention can extract the three-dimensional spatial features of trees under the condition of sparse data, ensuring the efficiency and accuracy of data processing; By accurately calculating the spatial relationship between trees and transmission facilities, the potential risks between trees and power facilities can be monitored in real time and dynamically; According to the multi-modal fusion data and the risk assessment model, multi-level warning information can be generated in real time, greatly improving the efficiency and accuracy of transmission line inspection, and realizing the early prediction and timely treatment of potential hazards. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic diagram of the overall process of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the three-dimensional point cloud generation process based on binocular vision of the vegetation risk and hidden danger detection method for sparse point cloud segmentation according to an embodiment of the present invention; Figure 3 Schematic diagram of the SparseConvNet algorithm framework for the vegetation risk hidden danger detection method of sparse point cloud segmentation according to an embodiment of the present invention; Figure 4 Schematic diagram of the tree growth prediction model for the vegetation risk hidden danger detection method of sparse point cloud segmentation according to an embodiment of the present invention. Detailed implementation manners

[0019] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] Embodiment 1, referring to Figures 1 - 4 , which is an embodiment of the present invention, providing a vegetation risk hidden danger detection method for sparse point cloud segmentation. As Figure 1 shown, it includes: S100, collecting image data from different angles of the monitoring area, generating sparse point cloud data, and preprocessing the sparse point cloud data; S200, based on the preprocessed sparse point cloud data, performing point cloud segmentation on the sparse point cloud data through a neural network, dividing the monitoring area according to the area type, and obtaining the target area; S300, based on the target area, analyzing the spatial relationship between the vegetation and the power equipment in the target area, and performing risk assessment to obtain the first risk assessment result; S400, combining multi-modal data, optimizing the first risk assessment result, evaluating the risk level according to the optimization result, and generating a warning signal.

[0021] In a preferred implementation manner, collecting image data from different angles of the monitoring area and generating sparse point cloud data includes: Setting the height and angle of image acquisition, and performing image acquisition of different angles and different vegetation types according to the preset acquisition frequency and acquisition quantity; Based on the collected images, comparing the image pairs at different angles in the same scene through a disparity matching algorithm to obtain the disparity map in the images; Based on the disparity map, calculating the corresponding depth information to obtain the depth map, and mapping each pixel point in the image to the corresponding position in the three-dimensional space to generate sparse point cloud data.

[0022] Specifically, a binocular vision device is used to collect vegetation data in the transmission line corridor area. After image processing and feature extraction, sparse point cloud data of the vegetation is generated. Among them, the binocular vision device uses two synchronized high-resolution binocular vision cameras, which are installed on the unmanned aerial vehicle (UAV) to ensure that the two cameras can simultaneously capture the target scene from different angles. The cameras are configured with high dynamic range (HDR) image sensors to cope with complex lighting conditions. The baseline distance of each pair of cameras is preferably set to 0.5 m. This distance can effectively obtain a more accurate disparity map and is not affected by large-scale scene occlusion.

[0023] In an alternative embodiment, as Figure 2 shown, the height and angle of image acquisition are set, and according to the preset acquisition frequency and acquisition quantity, images of different angles and different vegetation types are acquired. Among them, the height and angle of image acquisition and the preset acquisition frequency and acquisition quantity can be determined and set according to the actual scene of the monitoring area. In this embodiment, the UAV flies along the direction where power equipment is concentrated in the monitoring area. Preferably, the flight height is 20 m, the shooting angle is 45°, a group of image pairs is acquired every 10 m of flight, and each pair of images obtains a stereoscopic view of the same area from two different angles. At least 3 images are taken for each tree, and the area captured by each image should include the upper part of the tree, the trunk, and the area near the root to ensure that the complete form of the tree can be captured.

[0024] In an alternative embodiment, through the disparity matching algorithm, by comparing the image pairs taken by two cameras in the same scene, the disparity map in the image is obtained. The disparity value of each pixel point in the disparity map represents the distance difference between this pixel and the corresponding image point, indicating the position change of the object in the three-dimensional space; using the disparity map to calculate the corresponding depth information. Assuming that the baseline distance of the camera is , the focal length is , then the depth can be calculated from the disparity through the following formula: , Through this formula, the obtained depth map can help map each pixel point to the corresponding position in the three-dimensional space, thereby generating sparse point cloud data.

[0025] In another alternative embodiment, a structured light projector can be used to project a known pattern (such as stripes or dot matrix) onto the target scene, and a single or multiple cameras can be used to capture the deformation of the reflected pattern. The three-dimensional coordinates of the object surface can be calculated using geometric constraints to generate point cloud data. Alternatively, a lidar can be used to emit laser pulses towards the target scene and receive the reflected signals, and the distance to the target object can be calculated by measuring the flight time or phase difference to generate dense point cloud data.

[0026] In a preferred embodiment, the preprocessing of the sparse point cloud data includes: Performing denoising processing on the sparse point cloud data to detect and remove outliers in the point cloud data; Performing downsampling processing on the sparse point cloud data to obtain optimized sparse point cloud data, and performing visualization processing to obtain the distribution characteristics of different vegetation types in the point cloud data.

[0027] It should be noted that due to the acquisition method of the binocular vision system, the depth information has strong sparsity, that is, the three-dimensional points corresponding to each camera pixel are unevenly distributed in space. Traditional point cloud data will contain a large number of invalid or low-density points, and sparse point clouds are obtained by removing these invalid data, making the final data set more efficient and representative. Preprocessing the sparse point cloud data, including denoising, data standardization, and coordinate normalization, etc., to improve the accuracy and processing efficiency of the data.

[0028] In an alternative embodiment, the Statistical Outlier Removal (SOR) algorithm is used to perform denoising processing on the point cloud data. This algorithm calculates the mean and standard deviation of the neighboring points of each point, removes outliers, and retains the points within the expected distance range. The specific steps are as follows: For each point, calculate the distance between it and the points in its neighborhood; Calculate the average distance and standard deviation of all neighboring points; If the distance of a point is greater than a certain threshold (outliers exceed 2 times the standard deviation), then remove this point from the point cloud data.

[0029] It should be noted that this denoising method can effectively remove the error points caused by sensor noise or environmental interference (such as light changes, leaf fluttering, etc.), and ensure the quality of the final point cloud data.

[0030] In another alternative embodiment, in addition to the statistical outlier removal algorithm, the following methods can also be used to detect and remove outliers in the point cloud data, such as radius outlier removal (ROR), which judges based on the number of points in the neighborhood. For each point, calculate the number of its neighbor points within a given radius range and determine the outliers, and only the radius and the minimum number of neighbor points need to be set; it can also be conditional filtering, which filters the point cloud according to the defined conditions (such as the coordinate range of the points, the normal vector direction, etc.), and the filtering rules can be customized according to specific requirements. It can not only remove outliers, but also retain specific regions or features.

[0031] In an alternative embodiment, in order to improve the calculation efficiency and reduce the memory consumption, voxel grid filtering is used to downsample the point cloud data. The steps of the voxel grid filtering method are as follows: Define the voxel size, such as set to 0.1 meters (that is, the side length of each voxel is 0.1 meters); Divide the three-dimensional space into multiple voxel cells of equal size; For each voxel, retain one point within the voxel (usually the centroid or mean value of all points in the voxel); Delete the redundant points in other voxels and only retain the representative points in each voxel.

[0032] Exemplarily, let the size of the voxel grid be , that is, each voxel has a length in the , , axis directions. For each point cloud , calculate its voxel coordinates in the voxel grid, which is expressed as: , where represents the floor operation, and then for the points within each voxel, retain the centroid point or the average point as the representative of the voxel.

[0033] Furthermore, through the above steps, the optimized sparse point cloud data is obtained and visualized. The following is a display taking the point cloud data of some trees (such as eucalyptus, pine, camphor tree) as an example: Eucalyptus point cloud: Eucalyptus belongs to tall trees, with a relatively straight trunk and a wide crown. The point cloud data is dense in the trunk and crown areas and sparse in the far areas; Pine point cloud: The trunk of the pine tree is relatively thin and the branches are sparse. The point cloud is more dispersed, especially in the branch end areas where the point cloud is sparse; Camphor tree point cloud: Camphor trees are relatively common, with thick trunks and umbrella-shaped crowns. The point cloud is concentrated in the trunk and crown areas.

[0034] Through these point cloud visualization diagrams, the distribution characteristics of different tree species in the point cloud data can be seen. The point cloud data after these processes will be used for subsequent sparse point cloud segmentation and vegetation risk and hidden danger detection to ensure the efficiency and accuracy of the detection process.

[0035] In a preferred implementation, the sparse point cloud data is segmented by a neural network, and the monitoring area is divided according to the area type to obtain target areas including: Collect historical point cloud data of various different vegetation species, and perform primary annotation and secondary annotation on the historical point cloud data. The primary annotation is to annotate the corresponding area type in the point cloud data, and the secondary annotation is to annotate the corresponding structure type in the point cloud data; Construct a point cloud segmentation model through a neural network algorithm, and input the annotated historical point cloud data into the point cloud segmentation model for model training; Input the preprocessed sparse point cloud data into the point cloud segmentation model for point cloud segmentation, identify the area type and vegetation type in the point cloud data, determine the target area, and mark the spatial position of each vegetation in the target area.

[0036] In an alternative implementation, in order to train the sparse convolutional network, a large amount of labeled sparse point cloud data is required. For each frame of the collected point cloud data, it is manually labeled into different categories, and the annotation content includes primary annotation and secondary annotation. In this embodiment, the area types corresponding to the primary annotation can be divided into vegetation areas (areas containing vegetation), power equipment areas (power facilities areas such as transmission towers and conductors), and other areas (ground, sky, etc.). The structure types corresponding to the secondary annotation can be the structural parts of the trees in the vegetation area. The structural part annotation of the vegetation area includes the crowns, trunks, etc. of the trees. For example, the tall trunks and thin branches of pine trees, the crown width of eucalyptus trees, and the relatively regular morphology of arborvitae. The structural part annotation of the power equipment area is its morphological characteristics in the point cloud (such as the long strip shape of the conductor, the three-dimensional structure of the tower, etc.) In an alternative implementation, as Figure 3 shown, a point cloud segmentation model is constructed through a neural network algorithm. Preferably, it is constructed through the sparse point cloud segmentation network of the Sparse Convolutional Neural Network (SparseConvNet), including an input layer, a sparse convolutional layer, a pooling layer, a fully connected layer, etc.

[0037] Among them, the input layer includes: The input is sparse point cloud data, which is input into the network in sparse point cloud format. Each point contains spatial coordinates (x, y, z) and additional features (such as normal vectors, color information, etc.), and these features play an auxiliary role in subsequent segmentation tasks.

[0038] Sparse Convolutional Layers include: Use sparse convolutional layers to extract features from point cloud data. The convolutional kernel is only calculated at non-empty points to avoid redundant calculations. The convolution operation is carried out through the following formula: , where, represents the neighbor point set of point , and are the convolutional kernel and bias term of the th layer respectively, is the output feature of the th layer, represents the feature of point j at the l th layer.

[0039] Pooling Layers include: To improve computational efficiency and model robustness, use Sparse Max Pooling operation to compress the features in the local area and reduce the spatial dimension. The pooling operation compresses the space by selecting the maximum value among the neighbor point features. The formula for the pooling layer is: , Fully Connected Layers include: In the last few layers, use fully connected layers to integrate the features and output the regional type (trees, ground, power facilities, etc.) of each point.

[0040] Softmax Layers include: Finally, perform multi-classification processing through the Softmax layer to classify each point into different types.

[0041] It should be noted that sparse point cloud data is processed and analyzed through a Sparse Convolutional Neural Network (SparseConvNet) to identify vegetation in different regions (especially 9 common tree species) and their spatial relationships with power facilities. The core purpose of point cloud segmentation is to distinguish different types of objects, including trees, the ground, etc., and thus provide accurate data support for subsequent risk detection. SparseConvNet is a neural network architecture specialized for processing sparse point cloud data. It can efficiently perform convolution operations on sparse point cloud data and analyze the spatial structure in the point cloud data. Compared with traditional Convolutional Neural Networks (CNNs), SparseConvNet can process a large amount of sparse data, avoiding fully connected or dense matrix calculations, thereby improving computational efficiency and memory utilization.

[0042] In another alternative implementation, point cloud segmentation can also be performed through PointNet, which is a deep learning framework for directly processing point cloud data. By designing a network structure insensitive to the arrangement order of point clouds (such as max pooling operations), point cloud classification and segmentation tasks are achieved; point cloud segmentation can also be performed through KPConv, which is a point cloud processing algorithm based on kernel point convolution. By defining a set of fixed kernel points in the local area of the point cloud and calculating the weight relationships between these kernel points and neighboring points, efficient extraction of point cloud features is realized.

[0043] Exemplarily, the training process of SparseConvNet using labeled sparse point cloud data is as follows: Data input: The collected sparse point cloud data is input into the network. The data input includes not only three-dimensional coordinate information but also other feature information (such as the color, normal vector, etc. of the points), and this information can further improve the segmentation accuracy.

[0044] Feature extraction and segmentation: The network extracts features from the point cloud data through multiple layers of sparse convolution, gradually identifying the features in different regions. Especially when facing different tree species, the network can segment different tree species according to features such as the height, crown shape, and branch distribution of the trees. For example, for pine trees, due to their tall and relatively sparse crowns, the network can capture the differences between the trunk and branch regions through convolutional features; for eucalyptus trees, due to their thick trunks and wide crowns, the network will identify these features and perform segmentation.

[0045] The segmentation results, through network training and prediction, according to the type of the region to which each point belongs, are assigned a class label, which can be "trees", "ground", or "power facilities", etc. Through a visualization tool, the segmented point cloud can be displayed, and the regional divisions of trees, the ground, transmission towers, etc. can be clearly seen; for multiple different types of vegetation, the network can efficiently separate these tree categories and mark the spatial positions where each tree is located.

[0046] In a preferred embodiment, analyze the spatial relationship between the vegetation and power equipment in the target area and conduct a risk assessment. The first risk assessment results include: Based on the segmented point cloud data, perform 3D modeling on the power facilities, calculate the minimum distance between each piece of vegetation and the power equipment, and use it as the first indicator for risk assessment; Calculate the vertical distance between each piece of vegetation and the power equipment, and use it as the second indicator for risk assessment; Predict the future growth and crown width data of the vegetation, and use the prediction results as the third indicator for risk assessment; Combine the first indicator, the second indicator, and the third indicator to conduct a comprehensive risk assessment, obtain the first risk assessment result, and determine whether there is a risk of contact between the vegetation and the power equipment.

[0047] It should be noted that high-quality sparse point cloud data has been collected through a binocular vision system, and the SparseConvNet has been used to effectively segment the point cloud data, successfully distinguishing different regions such as trees, the ground, and transmission towers. The next task is to analyze the spatial relationship between the vegetation and the power facilities, and it is possible to evaluate and determine whether the trees may come into contact with or affect the power facilities, thus posing a threat to the safety of the transmission line.

[0048] Through the analysis of the spatial relationship, the following key objectives can be obtained: The minimum distance between the trees and the transmission line: Measure the safety distance between the trees and the transmission line; The relationship between the height of the trees and the transmission facilities: Analyze whether the height of the trees may exceed the maximum safety height of the transmission line; The growth trend of the trees and risk assessment: Predict the future impact of the trees on the power facilities based on their growth trends and current states.

[0049] In an alternative embodiment, the nearest distance between the trees and the power facilities can be calculated through a nearest neighbor search algorithm (such as KD-tree or Octree), including: Step 1: Input data From the segmentation step, the point cloud data of the trees (including the segmentation results of each type of tree) is obtained. By modeling the power facilities (such as transmission towers and conductors), the three-dimensional coordinate data and geometric shape information of the power equipment are acquired; Step 2: Distance calculation Use a KD tree (k-dimensional tree) to construct an index structure for the tree point cloud data and the power facility point cloud data. A KD tree is a spatial data structure that can efficiently perform nearest neighbor queries; For each tree point , find the nearest power facility point , and calculate its Euclidean distance , expressed as: , where and are the coordinates of the tree and the power facility point respectively. Calculate the minimum distance between each tree and the power facility as the first indicator for evaluating the risk.

[0050] In an alternative implementation, in order to analyze whether the tree is likely to affect the transmission line, in addition to calculating the minimum distance, the vertical distance between the tree and the transmission line also needs to be calculated, which can be calculated by the distance formula from a point to a line.

[0051] Let the tree point be , and the straight line equation of the transmission line be , where is a point on the line, is the direction vector of the line, is a parameter, and the vertical distance from the tree point to the transmission line is calculated by the formula: , Through this calculation, it can be determined whether the tree has a risk of contacting the transmission line in its current state.

[0052] In an alternative implementation, as Figure 4 shown, the tree is not just a static object. Considering the growth trend of the tree, it is necessary to predict whether it may pose a threat to the power facilities in the future. The growth trend of the tree can be inferred based on the current height, crown width, and growth rate of the tree.

[0053] Construct a tree growth model. According to the age and type of the tree, assuming that the changes in its height and crown width over time satisfy a certain growth law, an exponential growth model can be used to describe the growth trend of the tree, expressed as: , Among them, is the height of the tree at time , is the current height of the tree, is the annual growth rate of the tree (which can be set according to the characteristics of the tree species), is time. Through this model, the future height of the tree can be predicted, and combined with the safety distance requirements of power facilities, the impact of future trees on power facilities can be evaluated.

[0054] To predict the crown width of the tree, in addition to the height of the tree, the crown width of the tree is also an important factor affecting its spatial relationship with power facilities. The growth of the tree crown width can be predicted through a similar growth model. Assuming that the crown width of the tree also follows exponential growth, it can be expressed as: , Among them, is the crown width of the tree at time , is the current crown width, is the crown width growth rate of the tree.

[0055] In another alternative implementation, the second index can also be calculated by extracting the highest point of the tree point cloud and the ground point (determining the ground position through plane fitting or RANSAC algorithm), and calculating the vertical distance between the two; the third index can be calculated by extracting the bounding box of the tree point cloud, calculating its projected area or diameter on the horizontal plane, or by using a clustering algorithm (such as DBSCAN or region growing) to extract the crown point cloud of the tree and calculating the radius of its minimum enclosing circle or ellipse.

[0056] Then, combining the first index, the second index, and the third index, that is, the height prediction of the tree, the crown width prediction, and the distance between the tree and the power facilities, a comprehensive risk assessment is carried out. If it is predicted that the future height or crown width of the tree will exceed the set safety distance (the tree height exceeds the safety height of the transmission line, or the tree crown comes into contact with the conductor), then it can be determined that the tree is a high-risk tree and needs to be pruned or removed.

[0057] It should be noted that after separating the vegetation and other objects, in this embodiment, a spatial relationship model between the vegetation and the power facilities is further established through spatial relationship analysis. By using distance calculation and geometric models in three-dimensional space, the potential risks between the vegetation and the power facilities are identified and evaluated. During the analysis process, the growth dynamics of the vegetation, such as the growth height of trees and the trend of crown expansion, are considered to predict the possible risks of vegetation invading the power facilities in the future and issue early warnings in advance. This analysis method based on spatial relationships can accurately determine which vegetation has potential hidden dangers, especially for trees, shrubs, etc. around high-voltage transmission lines and power towers, and has strong detection capabilities.

[0058] In a preferred implementation manner, optimizing the first risk assessment result by combining multi-modal data includes: The multi-modal data includes sparse point cloud data and image data. According to the spatial information of the vegetation, the eigenvalue of the vegetation in the sparse point cloud data and the image data is calculated respectively and compared with the preset standard, and weights are assigned to the sparse point cloud data and the image data. Based on the assigned weights, the two types of data of the sparse point cloud data and the image data are fused and combined, and the spatial relationship between the vegetation and the power equipment is further calculated.

[0059] It should be noted that the above steps have obtained the sparse point cloud data of the trees and the power facilities through the binocular vision device, segmented the point cloud through SparseConvNet, and analyzed the spatial relationship between the trees and the power facilities. In order to better identify and predict the potential risks between the trees and the power facilities, a multi-modal data fusion technology is adopted to comprehensively analyze different data sources (image data, point cloud data, climate data, etc.) to improve the accuracy and reliability of the risk assessment of the trees and the power facilities. Finally, accurate risk warnings are realized through intelligent algorithms. The core goal of multi-modal data fusion is to integrate multiple data from different sensors (such as binocular vision images, sparse point cloud data, climate information, etc.), make full use of the advantages of each data source, improve the accuracy and comprehensiveness of the tree risk assessment. By fusing different types of data, the model can better predict the risks of trees, identify potential hidden dangers and provide intelligent warnings.

[0060] In an alternative implementation manner, the multi-modal data fusion mainly includes point cloud data, that is, the spatial information of the trees and the power facilities extracted and segmented through the above steps; and image data, that is, the tree images obtained through the binocular vision system to help identify the growth state and morphology of the trees. These data modalities need to be combined together for fusion to obtain more accurate spatial analysis results. The data fusion algorithms include: The key to data fusion lies in how to combine point cloud data and image data to generate a unified model. A weighted fusion strategy is adopted, that is, various types of data are weighted according to the weights of different data sources to obtain a more accurate 3D model.

[0061] For the spatial information of each tree, features such as position and size in the point cloud data and image data are calculated respectively, and then compared with the preset standards to assign weights and merge the two types of data. If the accuracy of the point cloud data is higher, a higher weight is given to the point cloud data. Among them, the preset standards can be the spatial accuracy of the point cloud data, the texture information of the image data, etc.

[0062] The fused data can be calculated through the following weighted formula: , where, is the fused data, and are the point cloud data and image data respectively, and are their weights.

[0063] It should be noted that by combining multi-modal data (such as binocular vision images, point cloud data, and possibly other sensor data), the detection accuracy is further improved through a data fusion model. The stereo image information of binocular vision and sparse point cloud data are automatically fused through deep learning algorithms to enhance the ability to identify vegetation risks. Especially in complex scenarios (such as dense vegetation, occlusion, complex perspectives, etc.), more stable and accurate detection results can be provided; during the fusion process, an adaptive fusion strategy is adopted to weight different modal data according to the quality and characteristics of different data sources, improve the sensitivity of the model to vegetation, and effectively filter out irrelevant or highly interfering data. By fusing the two types of data, a 3D spatial relationship model between trees and power facilities can be established more accurately. The point cloud data provides more accurate spatial information, while the image data can make up for the texture loss and recognition accuracy problems of the point cloud data.

[0064] In a preferred embodiment, evaluating the risk level and generating a warning signal includes: If there is a risk of contact with vegetation, a risk report will be generated and reported. Evaluate the risk level according to the optimization results, classify the risks into high-risk, medium-risk, low-risk, etc. levels, and take corresponding response measures.

[0065] Specifically, by real-time monitoring the spatial relationship between trees and power facilities, combined with the prediction of the growth trend of trees, potential risk hazards can be discovered in advance and a warning notice can be issued.

[0066] Based on the results of multi-modal data fusion, a risk assessment model is established. This model combines factors such as the growth trend of trees, the distance from power facilities, and the tree species, and is used to predict the possible impact of trees on power facilities in the future. The prediction of the tree growth trend uses the exponential growth model used in step S300 to predict the future height and crown width of the tree. According to the future growth of the tree and the spatial relationship with the power facilities, the potential threat of the tree to the power facilities is evaluated. If the future height or crown width of the tree will approach or exceed the safety distance of the transmission facilities, a risk alarm will be triggered.

[0067] If the growth trend of the tree predicts that its height or crown width will exceed the safety distance of the power facilities at a certain future time point, and the current minimum distance is already very small, then the tree is determined to be a high risk. If the tree is far from the power facilities and the growth prediction does not pose a threat, it is determined to be a low risk.

[0068] Input the multi-modal data (fused point cloud and visual data) into the intelligent early warning system, calculate the spatial relationship between the tree and the power facilities according to the data, and make predictions in combination with the growth trend; if there is a potential contact risk for the tree, the system will generate a risk report and push it to the devices of relevant management personnel. According to the evaluation results, the system will classify the risks into levels such as high risk, medium risk, and low risk, and take corresponding response measures.

[0069] It should be noted that on the basis of vegetation segmentation and spatial relationship analysis, in this embodiment, by establishing a risk assessment model, a potential risk score is given to the vegetation in each area, which can quantify the risk degree between the vegetation and the power facilities. For example, evaluate the distance between the growth height of the vegetation and the power equipment, predict possible tree obstacle hazards, and generate a risk level report; according to the evaluation results, the system can issue real-time warnings, prompt possible vegetation intrusion or other risks, and take corresponding management measures according to different risk levels, such as regular pruning, removing dangerous vegetation, etc.

[0070] The present invention integrates multiple binocular vision devices and sensors to collect all-weather real-time data, and performs data processing and analysis through an edge computing or cloud computing platform. The system can continuously monitor large-scale transmission line corridors, timely obtain the growth dynamics of vegetation, quickly process data and generate risk warning reports. In areas such as high-voltage transmission line corridors and mountainous areas where it is difficult to conduct manual inspections, the system can replace manual inspections and timely discover potential vegetation risks, improving the safety guarantee ability of power facilities; with the long-term operation of the system, it can improve the accuracy and intelligent level of the model through continuously accumulated and feedback data. Especially in complex environments, the system can automatically adjust according to different vegetation types, terrain and environmental changes, optimize the detection accuracy and real-time performance, and reduce false alarms and missed alarms.

[0071] The beneficial effects of the present invention are as follows: (1) Precise and efficient sparse point cloud segmentation Improvement over traditional methods: Compared with the point cloud generation methods that are highly dependent on devices such as lidar and have high costs, using a binocular vision device to generate sparse point cloud data significantly reduces hardware dependence and operating costs; Efficient segmentation: Using SparseConvNet to accurately segment sparse point cloud data can extract the three-dimensional spatial features of trees (such as tree trunks and crown widths) even in the case of sparse data, ensuring the efficiency and accuracy of data processing; Adapt to complex environments: Even if there is noise or occlusion in the sparse point cloud, SparseConvNet can still achieve high-precision segmentation of different tree species and is suitable for the complex and changeable natural environment of transmission corridors.

[0072] (2) Accurate spatial relationship analysis and risk assessment Dynamic monitoring: By accurately calculating the spatial relationship between trees and transmission facilities (such as the minimum distance and vertical distance), the potential risks between trees and power facilities can be dynamically monitored in real time; Growth prediction and trend analysis: Combining with the tree growth model, it can predict the changes in the height and crown width of trees in the future for a period of time, providing a scientific basis for the early prediction of potential risks; Identification of high-risk tree species: This patented technology can effectively identify high-risk tree species and put forward improvement suggestions in advance to avoid the expansion of potential risks.

[0073] (3) Innovative multi-modal data fusion technology Enhance data reliability: By weighted fusion of the image data generated by binocular vision and the sparse point cloud data, the texture features and spatial features are integrated, improving the reliability and accuracy of the detection results; Adapt to diverse scenarios: The multi-modal data fusion technology has low dependence on device environments and data types and can adapt to different transmission line scenarios, including vegetation-dense areas, open areas, etc.; Reduce false alarms and missed detections: The comprehensive analysis of the fused data significantly reduces the false alarms and missed detections caused by the processing of traditional single-sensor data, improving the intelligence level of the system.

[0074] (4) Efficient intelligent warning Real-time warning mechanism: Build an intelligent warning mechanism. According to the multi-modal fusion data and risk assessment model, it can generate multi-level warning information (such as high risk, medium risk, low risk) in real time and push it to the management platform; High warning accuracy: It can issue accurate warnings for trees with different risk levels, providing decision-making support for the management of transmission lines; Active protection function: By predicting in advance and issuing early warnings, potential hazards of trees to transmission facilities are avoided, and power interruption incidents caused by tree obstacles are significantly reduced.

[0075] (5) Low cost and high availability Reduction of hardware investment: Compared with expensive devices such as lidar, the patented technology can generate high-precision point cloud data only relying on a binocular vision device, greatly reducing the hardware cost; Wide applicability: The binocular vision device is portable and suitable for rapid deployment in the inspection of transmission lines, especially suitable for large-scale coverage detection of transmission line corridors; Saving manpower and time: The intelligent detection and warning system reduces the dependence on manual inspection, and greatly improves the efficiency and accuracy of transmission line inspection.

[0076] (6) Improving the safety management level of transmission lines Advance amount for hidden danger elimination: It can predict the future threat trend through risk assessment before the occurrence of hidden dangers, realizing the early prediction and timely treatment of hidden dangers; Reducing power outage risk: By effectively identifying high-risk tree species and putting forward treatment suggestions, the tripping and power outage incidents of power lines caused by tree obstacles are greatly reduced; Balancing environmental protection and safety: On the basis of accurately identifying high-risk vegetation, unnecessary felling is avoided, and scientific management of vegetation resources is realized.

[0077] The present invention uses SparseConvNet (sparse convolutional neural network) to process sparse point cloud data. Combining with the depth information provided by the binocular vision device, it can more efficiently identify and segment vegetation data, and at the same time accurately evaluate its spatial relationship with the transmission line. Since SparseConvNet can efficiently process sparse data and avoid the calculation of fully dense data in traditional convolutional neural networks, it has significant advantages in processing large-scale point cloud data. By combining the disparity map provided by the binocular vision system, more accurate three-dimensional reconstruction can be carried out, and the vegetation status in the transmission line corridor can be detected in real time, providing more accurate data support for risk and hidden danger assessment.

[0078] This technology can not only improve the accuracy of vegetation risk and hidden danger detection, but also effectively shorten the detection time and improve work efficiency. Especially in complex environments, such as different vegetation types, terrain changes and lighting differences, etc., the sparse point cloud segmentation technology based on SparseConvNet can maintain high robustness and ensure stable operation in various environments. The binocular vision system effectively makes up for the defect that traditional two-dimensional image processing methods cannot accurately capture three-dimensional space information by providing rich depth information, and further improves the analysis accuracy of the spatial relationship between vegetation and transmission lines.

[0079] In summary, the fusion method of sparse point cloud segmentation and binocular vision device based on SparseConvNet can greatly improve the accuracy of vegetation risk detection in the transmission line corridor while ensuring high efficiency, providing a more reliable technical guarantee for the safe operation of power facilities. The proposed new technology not only solves the problems of low computational efficiency and low accuracy in the existing technology, but also provides effective technical support for the intelligent and automated inspection of the power industry.

[0080] Example 2. This example provides a vegetation risk detection system for sparse point cloud segmentation, including: An acquisition and processing module, which is used to acquire image data from different angles of the monitoring area, generate sparse point cloud data, and preprocess the sparse point cloud data; A segmentation module, which is used to perform point cloud segmentation on the sparse point cloud data based on the preprocessed sparse point cloud data through a neural network, divide the monitoring area according to the area type, and obtain the target area; An evaluation module, which is used to analyze the spatial relationship between the vegetation and power equipment in the target area based on the target area and perform risk assessment to obtain the first risk assessment result; An output module, which is used to optimize the first risk assessment result by combining multi-modal data, evaluate the risk level according to the optimization result, and generate a warning signal.

[0081] The previous example is a schematic solution of a vegetation risk detection method for sparse point cloud segmentation. It should be noted that the technical solution of the vegetation risk detection system for sparse point cloud segmentation belongs to the same concept as the technical solution of the above-mentioned vegetation risk detection method for sparse point cloud segmentation. For the details not described in the technical solution of the vegetation risk detection system for sparse point cloud segmentation in this example, reference can be made to the description of the technical solution of the vegetation risk detection method for sparse point cloud segmentation.

[0082] This example also provides a computer device suitable for the situation of vegetation risk detection for sparse point cloud segmentation, including: A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the vegetation risk detection method for sparse point cloud segmentation as proposed in the above example.

[0083] This example also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the vegetation risk detection method for sparse point cloud segmentation as proposed in the above example.

[0084] The storage medium proposed in this embodiment and the method for detecting vegetation risk hazards for sparse point cloud segmentation proposed in the above embodiment belong to the same inventive concept. Technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0085] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disc of a computer, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0086] Embodiment 3, referring to Table 1, is an embodiment of the present invention, providing a method for detecting vegetation risk hazards for sparse point cloud segmentation. In order to verify its beneficial effects, scientific demonstrations are carried out through economic benefit calculations and simulation experiments.

[0087] When detecting the risk hazards of vegetation in the transmission line corridor, here the vegetation in the transmission line corridor takes 9 common trees (eucalyptus, camphor tree, paper mulberry, cyclobalanopsis glauca, tung tree, arborvitae, pine tree, fir tree, bamboo) as an example. Using binocular vision, 10 kilometers of the area along the transmission line corridor is selected for data collection. The vegetation types in the area include 9 common trees such as eucalyptus, camphor tree, paper mulberry, cyclobalanopsis glauca, tung tree, arborvitae, pine tree, fir tree, bamboo. The target trees are distributed at different positions of the transmission line according to their types and sizes, ensuring that multiple tree types can be covered simultaneously in the same scene for real environment testing.

[0088] Under the condition of sufficient sunlight during the day, a binocular camera device installed on a drone was used for data collection. The drone flew along the transmission line corridor at a flight altitude of 20 meters and a shooting angle of 45°, ensuring complete coverage of the viewing angle. The flight path was set as a fixed transmission line corridor to ensure a wide and unobstructed viewing angle coverage. The binocular vision system collected a set of image pairs every 10 meters of flight. Each pair of images obtained stereoscopic views of the same area from two different angles. At least 3 images were taken of each tree, and the area captured by each image should include the upper part of the tree, the trunk, and the area near the root to ensure that the complete form of the tree could be captured. Each set of images obtained strong contrast texture information within the overlapping area of the fields of view of the two cameras for parallax calculation.

[0089] Among them, in order to train the sparse convolutional network, a large amount of labeled sparse point cloud data is required. The following is the preparation process of the dataset in the experiment: The point cloud data of 9 types of trees collected by the above binocular vision device was used, and the corresponding areas of each type of tree were labeled. The labeling content included the tree crown, trunk, and other objects (ground, transmission tower, etc.) of the tree. For each frame of the collected point cloud data, it was manually labeled into different categories, and the labeling content included: Tree area: The tree area was labeled through professional tools or manual methods. The tree area was calibrated according to its shape characteristics (trunk, tree crown, etc.). In particular, different parts of the tree, such as the tall trunk and thin branches of pine trees, the crown width of eucalyptus trees, and the relatively regular shape of Platycladus orientalis, etc., all needed to be specially labeled.

[0090] Power facility area: The labeling included power facility areas such as transmission towers and conductors. For power facilities, accurate labeling was required according to their morphological characteristics in the point cloud (such as the long strip shape of conductors and the three-dimensional structure of towers, etc.).

[0091] Finally, the risk assessment analysis and results are shown in the data of Table 1.

[0092] Table 1: Risk assessment table , The above are the experimental results and their analysis of 9 common types of trees based on the actual collected data. The following analysis is obtained according to the above risk assessment table: Eucalyptus: When the current height of the eucalyptus is 12 meters, its minimum distance is 4.5 meters, which is relatively close to the power facilities. And the tree will grow rapidly to 16 meters within the next 5 years, and the crown width will also increase. Therefore, it is rated as a high-risk tree. It is expected that its height will exceed the safety height of the power facilities and needs to be pruned regularly.

[0093] Camphor tree: The current height of the camphor tree is 8 meters, the crown width is 4 meters, the minimum distance is 5.2 meters, and the vertical distance is 4.5 meters, meeting the safety distance requirements. Moreover, its growth in the next 5 years will not pose a threat to power facilities, so it is rated as low risk.

[0094] Paper mulberry tree: The current height of the paper mulberry tree is 7 meters, the crown width is relatively small, the minimum distance is 3.8 meters, and there are certain potential safety hazards. It is estimated that its height will increase to 9 meters in the next 5 years. Although it will not come into contact with power facilities, due to its relatively close minimum distance, it is still rated as medium risk.

[0095] Cyclobalanopsis glauca: The current height of the Cyclobalanopsis glauca is 10 meters, the crown width is 5 meters, the minimum distance is 6.0 meters, and the vertical distance is relatively large. The growth of the tree will not affect power facilities. It is estimated that its height will be 13 meters after 5 years and is rated as low risk.

[0096] Tung tree: The current height of the tung tree is 6 meters, the crown width is relatively small, the minimum distance is 4.0 meters, and it is expected to increase to 8 meters in the next 5 years, with the crown width increasing to 4.5 meters. Therefore, it belongs to low risk.

[0097] Biota orientalis: The current height of the Biota orientalis is relatively low (5 meters), the tree crown is small, the minimum distance is relatively large (7.0 meters), and it grows slowly. It is estimated that it will only grow 1 meter after 5 years, and the tree will not pose a threat to power facilities, so it is rated as low risk.

[0098] Pine tree: The current height of the pine tree is 15 meters, the crown width is relatively large, but the vertical distance from the power facilities is 2.5 meters, the safety distance is large, and it is expected that its height and crown width will not exceed the safety height of the power facilities significantly, so it is rated as low risk.

[0099] China fir: The current height of the China fir is 13 meters, the crown width is 6.5 meters, the minimum distance is 4.3 meters, and the vertical distance is relatively large. It is estimated that its growth will not pose a threat to power facilities, so it is rated as low risk.

[0100] Phyllostachys heterocycla: The current height of the Phyllostachys heterocycla is relatively small (4 meters), the minimum distance from the power facilities is 5.5 meters, the vertical distance is relatively large, and it grows slowly. It will not pose a threat to power facilities in the next 5 years and is rated as low risk.

[0101] The following are the segmentation accuracy and potential hazard detection results: Vegetation segmentation accuracy: In the experiment, the segmentation accuracy reached over 90%. Especially for common tree species such as pine trees, China firs, and eucalyptus trees, the segmentation effect was better. In a complex background, the system could also effectively identify vegetation from sparse point clouds.

[0102] Risk potential hazard detection results. Through the analysis of the spatial relationships of 9 common trees, the following summary was obtained from the detection results: High-risk trees: Only eucalyptus trees belong to high-risk trees. Because they grow relatively fast and are relatively tall, it is predicted that they will exceed the safe height of the transmission line within the next 5 years, and there is a possibility of contact with power facilities. High-risk warning. Regular pruning or other control measures are required.

[0103] Medium-risk trees: Broussonetia papyrifera belongs to medium-risk trees due to its relatively close minimum distance and future growth potential. Medium-risk warning. It is recommended to monitor its growth in the follow-up.

[0104] Low-risk trees: Most trees (such as camphor trees, tung trees, pine trees, fir trees, etc.) have a small growth range, a relatively far distance, or a low growth rate, and their growth in the next 5 years will not affect power facilities. They are low-risk trees. There is no warning, and they will not pose a direct threat to power facilities.

[0105] In summary, through the analysis of spatial relationships, the potential risks between 9 common trees (eucalyptus, camphor tree, Broussonetia papyrifera, Cyclobalanopsis glauca, tung tree, Platycladus orientalis, pine tree, fir tree, and moso bamboo) and power facilities can be effectively evaluated. Based on these analysis results, corresponding preventive measures (such as pruning and removing high-risk trees, etc.) can be taken to ensure the safe operation of the transmission line.

[0106] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A vegetation risk and hidden danger detection method for sparse point cloud segmentation, characterized in that Including: Collecting image data from different angles of the monitoring area, generating sparse point cloud data, and preprocessing the sparse point cloud data; Based on the preprocessed sparse point cloud data, performing point cloud segmentation on the sparse point cloud data through a neural network, dividing the monitoring area according to the area type to obtain target areas; Based on the target areas, analyzing the spatial relationship between the vegetation and power equipment within the target areas and performing risk assessment to obtain a first risk assessment result; Combining multi-modal data to optimize the first risk assessment result, evaluating the risk level according to the optimization result, and generating a warning signal; Analyzing the spatial relationship between the vegetation and power equipment within the target areas and performing risk assessment to obtain a first risk assessment result includes: Based on the segmented point cloud data, performing three-dimensional modeling on the power facilities, calculating the minimum distance between each piece of vegetation and the power equipment as the first index for risk assessment; Calculating the vertical distance between each piece of vegetation and the power equipment as the second index for risk assessment; Predicting the future growth and crown data of the vegetation and using the prediction result as the third index for risk assessment; Combining the first index, the second index, and the third index to perform comprehensive risk assessment to obtain a first risk assessment result and determining whether there is a risk of contact between the vegetation and the power equipment.

2. The vegetation risk and hidden danger detection method for sparse point cloud segmentation according to claim 1, characterized in that, Collecting image data from different angles of the monitoring area and generating sparse point cloud data includes: Setting the height and angle of image collection, and performing image collection of different angles and different vegetation types according to the preset collection frequency and collection quantity; Based on the collected images, comparing image pairs of different angles in the same scene through a disparity matching algorithm to obtain a disparity map in the images; Based on the disparity map, calculating the corresponding depth information to obtain a depth map, mapping each pixel point in the image to the corresponding position in the three-dimensional space, and generating sparse point cloud data.

3. The vegetation risk and hidden danger detection method for sparse point cloud segmentation according to claim 2, characterized in that, Preprocessing the sparse point cloud data includes: Performing denoising processing on the sparse point cloud data to detect and remove outliers in the point cloud data; Performing downsampling processing on the sparse point cloud data to obtain optimized sparse point cloud data, and performing visualization processing to obtain the distribution characteristics of different vegetation types in the point cloud data.

4. A vegetation risk and hidden danger detection method for sparse point cloud segmentation according to claim 1, characterized in that, Performing point cloud segmentation on the sparse point cloud data through a neural network, and dividing the monitoring area according to the area type to obtain target areas includes: Collecting historical point cloud data of multiple different vegetation species, and performing primary annotation and secondary annotation on the historical point cloud data. The primary annotation is to annotate the corresponding area type in the point cloud data, and the secondary annotation is to annotate the corresponding structure type in the point cloud data; Constructing a point cloud segmentation model through a neural network algorithm, and inputting the annotated historical point cloud data into the point cloud segmentation model for model training; Inputting the preprocessed sparse point cloud data into the point cloud segmentation model for point cloud segmentation, identifying the area type and vegetation type in the point cloud data, determining the target areas, and marking the spatial position of each piece of vegetation within the target areas.

5. A method for detecting vegetation risk hazards in sparse point cloud segmentation according to claim 1, characterized in that Combining multi-modal data to optimize the first risk assessment result includes: The multimodal data includes sparse point cloud data and image data. According to the spatial information of the vegetation, the eigenvalue of the vegetation in the sparse point cloud data and the image data is calculated respectively, and compared with the preset standard to assign weights to the sparse point cloud data and the image data. Based on the assigned weights, the two types of data, namely the sparse point cloud data and the image data, are fused and combined, and the spatial relationship between the vegetation and the power equipment is further calculated.

6. The vegetation risk and hidden danger detection method for sparse point cloud segmentation according to claim 5, wherein, Evaluate the risk level and generate warning signals, including: If there is a risk of contact between the vegetation and the power equipment, a risk report will be generated and reported. Evaluate the risk level according to the optimization result, classify the risk into high risk, medium risk, low risk and other levels, and take corresponding response measures.

7. A vegetation risk and hidden danger detection system for sparse point cloud segmentation, applying a vegetation risk and hidden danger detection method for sparse point cloud segmentation according to any one of claims 1 to 6, characterized in that, Including, A collection and processing module for collecting image data from different angles of the monitoring area, generating sparse point cloud data, and preprocessing the sparse point cloud data. A segmentation module for performing point cloud segmentation on the sparse point cloud data based on the preprocessed sparse point cloud data through a neural network, dividing the monitoring area according to the area type to obtain the target area. An evaluation module for analyzing the spatial relationship between the vegetation and the power equipment in the target area based on the target area, and performing risk assessment to obtain the first risk assessment result. An output module for optimizing the first risk assessment result by combining multimodal data, evaluating the risk level according to the optimization result, and generating warning signals. Analyzing the spatial relationship between the vegetation and the power equipment in the target area and performing risk assessment to obtain the first risk assessment result includes: Based on the segmented point cloud data, a three-dimensional model of the power facilities is built, and the minimum distance between each vegetation and the power equipment is calculated as the first index for risk assessment. Calculate the vertical distance between each vegetation and the power equipment as the second index for risk assessment. Predict the future growth and crown width data of the vegetation, and use the prediction result as the third index for risk assessment. Combining the first index, the second index and the third index, perform comprehensive risk assessment to obtain the first risk assessment result, and judge whether there is a risk of contact between the vegetation and the power equipment.

8. A computer device, characterized in that, Including: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for detecting potential vegetation risks by sparse point cloud segmentation according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method for detecting potential vegetation risks by sparse point cloud segmentation according to any one of claims 1 to 6 are implemented.

Images