FAST station site dangerous rock identification method based on unmanned aerial vehicle intelligent inspection

Abstract

The invention discloses a FAST site dangerous rock identification method based on unmanned aerial vehicle intelligent routing inspection, which comprises the following steps: S1, performing point cloud downsampling by using unmanned aerial vehicle routing inspection, obtaining rock wall radar data near a FAST site, and obtaining dangerous rock point cloud data; s2, determining the voxel size and the voxel center of the dangerous rock point cloud data; s3, extracting the features of the dangerous rock target, and extracting the feature vector of each voxel as the shape information of the dangerous rock target; and S4, candidate dangerous rock target detection: extracting a normal vector of the point cloud as a feature vector to describe shape information of the point cloud, and scanning point cloud data by using a sliding window method to obtain all possible candidate dangerous rock bodies. The method adopts intelligent identification and automatic determination, is an efficient and convenient dangerous rock identification method, is safe and efficient, is controllable in cost, has important engineering application value, and has important significance for stable and reliable operation and scientific achievement output of a 500-meter-caliber spherical radio telescope FAST of a national important scientific and technological infrastructure.

Description

Technical Field

[0001] The present invention relates to a method for identifying dangerous rocks at a FAST site based on intelligent inspection by unmanned aerial vehicles (UAVs). Background Art

[0002] The Five-hundred-meter Aperture Spherical radio Telescope (FAST), built on a natural karst depression in Guizhou Province, is the world's largest single-aperture radio telescope, enabling high-precision astronomical observations over a large sky area. It pioneered a new model for giant telescope construction, surpassing the 100-meter engineering limit of traditional telescopes. Its enormous scale, ultra-high precision requirements, and unique operating methods created unprecedented technical challenges.

[0003] The FAST site covers a large area and is located in the Dawodang depression in Jinke Village, Kedu Town, Pingtang County, Qiannan Prefecture, Guizhou Province. It is a karst landform with a rugged surface and numerous rocks. The surface often has stone buds, stone forests, peak forests, karst gullies, funnels, sinkholes, karst depressions and other depressions. There are 5 higher peaks around the FAST site. The highest peak is located on the southeast side of the depression, with a peak elevation of 1201.20m. The maximum terrain height difference is 360.30m.

[0004] The eastern steep slopes and cliffs of the FAST site are widely distributed with structures composed of multiple groups of rock structure surfaces. Under the influence of gravity, earthquakes, water bodies, weathering and other inducing factors, they change into unstable, understable or extreme equilibrium dangerous rock masses. They exist on high and steep slopes and cliffs and collapse due to instability and movement. The steep terrain is the geomorphological feature of the development of dangerous rocks. The main modes of their destruction are sliding dangerous rocks and toppling dangerous rocks.

[0005] The instability and fall of dangerous rocks in karst terrain are characterized by suddenness, high speed, and unpredictability. Under the weight of the dangerous rock mass, fractures in the main structural surface, fissure water pressure or extreme weather conditions, the stability of local dangerous rock units will be affected. The movement on the slope will also affect other locally unstable dangerous rock units, causing them to suddenly accelerate and fall or jump from the high and steep slopes around the site, colliding with the main structure of the site. The collision and friction cause the coating of the steel structure to be destroyed, accelerating the rusting of metals exposed to the air, or the collision directly causes irreparable damage to components, such as deformation, dents, fractures, and falling off of reflective panels, posing a serious threat to the safe and stable operation of FAST. Summary of the Invention

[0006] In response to the problems existing in the prior art, the purpose of the present invention is to provide a dangerous rock identification method at the FAST site based on drone intelligent inspection, which has the characteristics of being fast, efficient, low-cost, and ground-controlled.

[0007] To achieve the above objectives, the present invention provides a dangerous rock identification method at the FAST site based on UAV intelligent inspection, which specifically includes:

[0008] S1. Use drone inspections to perform point cloud downsampling, obtain radar data of rock walls near the FAST site, and obtain point cloud data of dangerous rocks;

[0009] S2. Determine the voxel size and voxel center of the dangerous rock point cloud data, and determine the voxel center point based on a spherical region search method;

[0010] S3. Extract the features of the dangerous rock target. For each voxel, extract its feature vector as the shape information of the dangerous rock target;

[0011] S4. Detection of candidate dangerous rock targets: voxelize the collected point cloud data, use the feature extraction method to extract the normal vector of the point cloud as the feature vector to describe the shape information of the point cloud, and use the sliding window method to scan the point cloud data to obtain all possible candidate dangerous rock bodies.

[0012] Furthermore, in step S1, a voxel-based downsampling method is adopted to perform point cloud downsampling.

[0013] Furthermore, in step S2, a spherical region is constructed with the center point of each voxel as the sphere center and the voxel size as the radius, and then point cloud data is searched within the region to determine the actual center point of the voxel.

[0014] Furthermore, in step S3, the point cloud normal vector is used as the feature vector, and for each point, its k nearest neighbor points are selected, and the k neighbor points are fitted to a plane, and then the normal vector of the plane is calculated as the normal vector of the point.

[0015] Furthermore, in step S4, a window of fixed size is slid in the point cloud data, and then the point cloud data in the window is feature extracted and classified; in the classification process, a support vector machine classifier is used, and the feature vector is used as input to classify the dangerous rock targets, and the classification results are matched with the original point cloud data to obtain the location and size information of the dangerous rock targets.

[0016] Furthermore, in step S1, a laser radar is used to collect dangerous rock data by changing the rotation mode of the laser radar body to realize a non-repetitive scanning scheme; the repeated scanning only covers the horizontal field of view angle range of 70.4 degrees multiplied by 77.2 degrees.

[0017] Furthermore, the echo mode of the laser radar is set to double echo, and the sampling frequency is selected to be 240KHz.

[0018] Furthermore, the steps of voxelizing the dangerous rock point cloud data are as follows:

[0019] 1) Meshing: First, the dangerous rock point cloud data needs to be converted into mesh data. The software package Cloud Compare is used to perform meshing operations to convert the dangerous rock point cloud data into triangular mesh data.

[0020] 2) Voxelization: Voxelize the grid data using the Voxel Grid voxelization algorithm and the Open3D software package to divide the three-dimensional space into several cubic voxels of equal size;

[0021] 3) Reconstruction: After voxelization, the voxel point cloud data needs to be reconstructed. For each voxel, the mean of all points inside the voxel is used as the center point of the voxel. The reconstructed voxel point cloud data retains the local structural information of the original point cloud data and is adjusted by the size of the voxels to achieve different accuracy requirements.

[0022] 4) Export: Export the reconstructed voxel point cloud data as a point cloud file in .bin format. The exported voxel point cloud data is used for subsequent dangerous rock detection and segmentation tasks.

[0023] Furthermore, the steps for calculating the normal vector of the dangerous rock point cloud data are as follows:

[0024] 1) Select the normal vector calculation method: Select the curvature-based normal vector calculation method;

[0025] 2) Select the normal vector calculation window size: When performing normal vector calculation, you need to select a neighborhood window to calculate the normal vector of the point cloud data. The size of the neighborhood window is selected based on the density and characteristics of the point cloud data. The selected neighborhood window size is one-tenth of the number of point cloud data points.

[0026] 3) Calculate the normal vector: For each point, calculate the normal vector through the points in its neighborhood window, and use the curvature estimation method and curvature direction estimation method to calculate the curvature-based normal vector;

[0027] 4) Normal vector filtering: After calculating the normal vector, there will be some erroneous normal vectors or outliers. To remove these outliers, the normal vector is smoothed using filtering methods including Gaussian filtering and median filtering.

[0028] 5) Export: Save the calculated normal vector as one of the attributes of the point cloud data, and save the point cloud data as a point cloud file in .bin format. The calculated normal vector can be used for subsequent dangerous rock detection and segmentation tasks.

[0029] Furthermore, the curvature calculation step includes:

[0030] 1) Calculate the normal vector of the dangerous rock point cloud data: Before calculating the curvature, calculate the normal vector of the dangerous rock point cloud data first;

[0031] 2) Calculate the covariance matrix of the dangerous rock point cloud data: Calculate the covariance matrix of each point, and calculate the curvature of the dangerous rock point cloud data in the normal vector direction and the tangential direction through the covariance matrix;

[0032] 3) Calculate the principal curvature and mean curvature of the point cloud data: By calculating the covariance matrix of the dangerous rock point cloud data, the principal curvature and mean curvature of the point cloud data are obtained; the principal curvature is the maximum and minimum values that describe the curvature change of the point cloud data surface in two orthogonal directions, and the mean curvature is the average value of the principal curvatures.

[0033] The present invention uses unmanned intelligent inspection equipment to realize intelligent identification and automatic judgment of dangerous rock masses. It is an efficient and convenient method for identifying dangerous rocks. It can discover problems in time and notify the operation and maintenance team for maintenance in time. It is safe, efficient, cost-controlled, and has important engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart of the present invention;

[0035] Figure 2 This is the flow chart for dangerous rock point cloud data detection;

[0036] Figure 3 This is the flow chart for preprocessing dangerous rock point cloud data;

[0037] Figure 4 This is the effect diagram of voxelization operation on dangerous rock point cloud data;

[0038] Figure 5 The effect diagram of removing outliers from dangerous rock point cloud data;

[0039] Figure 6 The effect diagram of removing outliers from dangerous rock point cloud data;

[0040] Figure 7 This is the flow chart for feature extraction of dangerous rock point cloud data. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0043] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0044] The following is combined with Figure 1-Figure 7 The specific embodiments of the present invention are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0045] The present invention provides a FAST site dangerous rock identification method based on UAV intelligent inspection, which is divided into four steps: point cloud downsampling, determining voxel size and voxel center, feature extraction and candidate dangerous rock target detection. Figure 1 Shown is a flow chart of the present invention.

[0046] The specific steps are:

[0047] (1) First, downsample the point cloud. Since the scale of point cloud data is usually very large, processing point cloud data requires a lot of computing and storage resources. Therefore, before extracting dangerous rock features, the point cloud data needs to be downsampled. Common downsampling methods include random sampling, uniform sampling, voxel sampling, and distance-based sampling. Since the morphology of dangerous rock targets and backgrounds is relatively similar and relatively small crack targets need to be detected, a voxel-based downsampling method is adopted to reduce the density of the point cloud and improve computational efficiency.

[0048] (2) Determine the voxel size and voxel center. The determination of the voxel size directly affects the detection effect of dangerous rock targets. When determining the voxel size, the resolution of the point cloud data and the scale of the dangerous rock target need to be considered. In the FAST dangerous rock target detection, the appropriate voxel size is selected for voxelization operation based on the resolution of the point cloud data and the scale of the dangerous rock target. The determination of the voxel center point is also one of the key steps in the detection of dangerous rock targets. In the FAST dangerous rock target detection, a method based on spherical area search is used to determine the voxel center point. Specifically, the center point of each voxel is used as the sphere center and the voxel size is used as the radius to construct a spherical area. Then, the point cloud data is searched in the area to determine the actual center point of the voxel.

[0049] (3) Extract the features of the dangerous rock target. For each voxel, extract its feature vector as the shape information of the dangerous rock target, and usually use the point cloud normal vector as the feature vector. When estimating the point cloud normal vector, a method based on nearest neighbor search can be used to calculate the normal vector of each point. For each point, select its k nearest neighbor points and use these points to calculate the normal vector of the point. A method based on plane fitting can be used to calculate the normal vector, that is, for each point, fit all points in its neighborhood to a plane, and then calculate the normal vector of the plane as the normal vector of the point.

[0050] (4) Detection of candidate dangerous rock targets. When detecting candidate dangerous rock targets, the collected point cloud data is first voxelized, and then the normal vector of the point cloud is extracted as a feature vector using a feature extraction method to describe the shape information of the point cloud. Next, the point cloud data is scanned using a sliding window method to obtain all possible candidate dangerous rock targets. Specifically, a fixed-size window is slid in the point cloud data, and then the point cloud data within the window is feature extracted and classified. In the classification process, a support vector machine (SVM) classifier is used to classify dangerous rock targets, that is, a feature vector-based classification method is used, the feature vector is used as input, and the SVM algorithm is used for classification. The training data of the classifier consists of manually labeled dangerous rock and non-dangerous rock samples. Finally, the classification results are matched with the original point cloud data to obtain the location and size information of the dangerous rock targets. The point cloud data visualization method can be used to display the detection results of dangerous rock targets, and the results can be evaluated and analyzed.

[0051] During the drone inspection process, the laser radar carried by the drone is used to obtain rock wall radar data near the FAST site and obtain dangerous rock point cloud data.

[0052] LiDAR achieves two different scanning modes: repetitive scanning and non-repetitive scanning by varying the rotational motion of the LiDAR body. Repetitive scanning only covers a horizontal field of view (FOV) of 70.4 degrees by 4.5 degrees. In mobile mapping, the measured object is only scanned for a very short period of time. Because the inertial navigation accuracy drift is minimal in this short period, the model is relatively more accurate. However, the vertical FOV is very small, and there is almost no elevation information in this direction. If elevation information is required, a route must be planned in at least two directions to compensate for the loss of vertical FOV. This is often used in scenes with high precision requirements and relatively gentle scenes without extreme fluctuations to ensure point cloud accuracy. Non-repetitive scanning can quickly cover the entire FOV (70.4 degrees by 77.2 degrees), scanning the facade in all directions, and obtaining good facade information in a single scan. However, in mobile mapping, objects will be scanned by radars at different locations at different times, so the accuracy consistency of the inertial navigation system is relatively dependent on it. If the inertial navigation system's accuracy drifts over time, the model accuracy will decrease (objects will become blurred or ghosted, the point cloud will become thicker, and linear targets will become coarser). It is often used in scenarios with relatively low accuracy requirements, high efficiency requirements, and the need for complete facade information, such as urban 3D modeling, complex stereo structure modeling, and emergency rapid mapping. Considering that the actual shooting scene at the FAST site is located on a steep facade, while ensuring full coverage of the scanning field of view and the accuracy of model reconstruction, a non-repetitive scanning solution was used to collect dangerous rock data.

[0053] The sampling frequency refers to the number of laser beams that the laser can emit per unit time, which affects the density of the reconstructed point cloud. Under the same conditions, the higher the frequency, the more detection points there are and the higher the operating efficiency. The sampling frequency of the lidar is related to the echo mode used. The echo mode can be selected as single echo, double echo or triple echo. When single echo and double echo are selected, the maximum frequency can be selected as 240KHz (that is, 240,000 points are sent per second). When triple echo is selected, the maximum sampling frequency can be selected as 160KHz. Theoretically, the maximum number of points that can be received per second for double echo and triple echo is 480,000. In the actual operation of dangerous rock collection at the FAST site, the penetration generated by the triple echo is too strong, and the noise generated by shrubs and dead grass will be too dense. The number of points of the second echo and the third echo accounts for a very small proportion. Preliminary experiments have shown that when the echo mode is set to double echo and the sampling frequency is selected as 240KHz, the number of points obtained is the largest.

[0054] During data collection, the drone used in this invention generates a high-density point cloud with realistic colors in real time. Combined with a high-precision integrated navigation system, data accuracy is guaranteed. Post-processing of the point cloud achieves centimeter-level accuracy, while simultaneously capturing information such as latitude, longitude, altitude, reflectivity, RGB values, and azimuth for the entire target. Once the acquisition parameters and flight path are set, point cloud data collection can begin.

[0055] Based on the data collected by drones and lidar, the original lidar data is reconstructed into point clouds and preprocessed. The target areas of dangerous rocks of interest are selected, and then the areas with targets are finely screened. Finally, the point cloud annotation tool Cloud Compare is used to annotate them and establish a dangerous rock point cloud database.

[0056] The dangerous rock point cloud dataset uses LiDAR sensors to collect location information of rocks or vegetation in the entire area. The difficulty of detection depends mainly on the following factors: the resolution and density of the point cloud data, the coverage and complexity of the vegetation, the shape and color of the dangerous rock, the influence of light and shadow, and the noise and interference in the point cloud data. Therefore, the dangerous rock point cloud data detection process needs to be designed as follows: Figure 2 shown.

[0057] Dangerous rock detection algorithm, including:

[0058] 1. Data preprocessing

[0059] Since there is a lot of noise and vegetation background in the FAST dangerous rock point cloud data, it is necessary to preprocess the original point cloud data to clean the interference in the original point cloud data to improve the accuracy of subsequent dangerous rock detection and segmentation. The preprocessing steps include voxelization, outlier removal based on point cloud data filtering, and vegetation noise removal. The specific processing process is as follows: Figure 3 shown.

[0060] 1.1 Voxelization Operation

[0061] Voxelization is the process of converting continuous geometric shapes (such as point clouds, triangular meshes, etc.) into discrete three-dimensional voxel representations. Voxelization is an important step in many three-dimensional graphics processing algorithms, such as three-dimensional reconstruction, CAD modeling, virtual reality, etc. In dangerous rock detection and segmentation, voxelization can convert point cloud data into three-dimensional voxel meshes such as Figure 4 As shown, it is convenient to perform volume calculation, neighborhood analysis, and convolution operations. Voxelization can convert continuous geometric shapes into discrete three-dimensional voxel representations, which is convenient for subsequent volume calculation, neighborhood analysis, and convolution operations.

[0062] The specific voxelization operation is as follows:

[0063] (1) Determine voxel size and boundaries. First, determine the size and boundary range of the voxel. Generally, the size of the voxel is determined by the resolution of the point cloud and the computing resources. Therefore, the voxel boundary is calculated based on the maximum and minimum coordinates of the point cloud data.

[0064] (2) Construct a voxel grid. After determining the size and boundaries of the voxel, map the point cloud data onto the voxel grid. Specifically, by traversing each point in the point cloud data, calculate the coordinates of the voxel where it is located, and mark the voxel as an existing point set. If multiple points are mapped to the same voxel, the points are filtered by calculating the average value of the point set;

[0065] (3) Voxel grid compression. Since voxel grids are typically much larger than point cloud data, they need to be compressed. Common compression methods include Run-Length Encoding (RLE) and Octree. This project uses RLE compression, which encodes consecutive repeated voxel data to effectively reduce the storage space of the voxel grid.

[0066] 1.2. Removing outliers based on point cloud data filtering

[0067] Outlier Removal is an important method for preprocessing point cloud data. It can remove isolated points or noise points in point cloud data that do not conform to the actual geometric shape, so as to improve the quality and accuracy of point cloud data. Common outlier removal methods include statistical methods, distance-based methods, and topological structure-based methods. In order to remove outliers such as branches and noise in dangerous rock point cloud data, a common distance-based outlier removal method, statistical filtering, is used. Statistical filtering is an outlier removal method based on the local geometric features of point cloud data. The effect of removing outlier point cloud blocks is as follows: Figure 5 Through statistical filtering and other methods, outliers and noise points in point cloud data can be removed to improve the quality and accuracy of point cloud data.

[0068] The specific steps are as follows:

[0069] (1) Determine the neighborhood size and distance threshold of the point cloud data. First, determine the neighborhood size and distance threshold of each point. The neighborhood size is usually a cube or sphere voxel, and the distance threshold determines which points in the point cloud data are considered outliers.

[0070] (2) Calculate the distance threshold and average distance of each point. For each point, calculate the average distance and standard deviation of the points in its neighborhood. Using these two values, calculate the distance threshold. The distance threshold can be considered as a multiple of the standard deviation related to the average distance. Take 2 times the standard deviation as the distance threshold;

[0071] (3) Mark outliers. For each point, compare it with the points in the neighborhood. If its distance exceeds the distance threshold, it is marked as an outlier. The coordinates of the outlier are saved in a list for subsequent processing.

[0072] (4) Remove outliers. All points marked as outliers are removed from the point cloud data.

[0073] 1.3. Remove vegetation noise based on RGB color information

[0074] Removing vegetation noise is an important step in preprocessing point cloud data, which can improve the accuracy of subsequent dangerous rock detection and segmentation. Common methods for removing vegetation noise include methods based on RGB color information, methods based on morphological operations, methods based on vegetation height, etc. This paper adopts a vegetation noise removal method based on RGB color information to filter vegetation noise, and uses the RGB color difference between vegetation and ground for segmentation. Figure 6 shown.

[0075] The specific steps are as follows:

[0076] (1) Extract RGB color information. For each point in the point cloud data, use the Open3D software package to extract its corresponding RGB color information;

[0077] (2) Filter vegetation points. Use RGB color information to filter out vegetation points in the point cloud data. Adjust the screening threshold according to the actual situation of the point cloud data. The RGB color value of vegetation points is relatively high, usually in the range of (100, 100, 100) to (255, 255, 255);

[0078] (3) Mark vegetation points. Mark the selected vegetation points as 1 and the remaining points as 0;

[0079] (4) Perform morphological operations. For the marked vegetation points, use the software package OpenCV to perform morphological operations (such as dilation, erosion, etc.) to remove small vegetation areas and noise;

[0080] (5) Remove vegetation points. Remove points marked as vegetation points from the point cloud data.

[0081] 2. Point cloud data feature extraction

[0082] In the task of dangerous rock detection and segmentation, feature extraction of point cloud data is a very important step. Common point cloud features include point normal vector, curvature, surface roughness, etc. The specific processing flow is as follows: Figure 7 shown.

[0083] 2.1 Voxelization Settings

[0084] Voxelization discretizes point cloud data into voxel point cloud data, converting it into a regular voxel grid. In dangerous rock detection and segmentation tasks, voxelization can effectively reduce the amount of point cloud data and improve computational efficiency. It also preserves the local structural information of the point cloud data, facilitating subsequent feature extraction and analysis. The detailed steps for voxelizing dangerous rock point cloud data are as follows:

[0085] (1) Meshing. First, the dangerous rock point cloud data needs to be converted into mesh data. The software package CloudCompare is used to perform meshing operations to convert the dangerous rock point cloud data into triangular mesh data.

[0086] (2) Voxelization. Voxelize the grid data using the Voxel Grid voxelization algorithm and the Open3D software package to divide the three-dimensional space into several cubic voxels of equal size.

[0087] (3) Reconstruction. After voxelization, the voxel point cloud data needs to be reconstructed. For each voxel, the mean of all points inside the voxel is used as the center point of the voxel. The reconstructed voxel point cloud data retains the local structural information of the original point cloud data and is adjusted by the size of the voxel to achieve different accuracy requirements;

[0088] (4) Export. Finally, the reconstructed voxel point cloud data is exported as a point cloud file in .bin format. The exported voxel point cloud data is used for subsequent dangerous rock detection and segmentation tasks.

[0089] 2.2 Normal vector calculation

[0090] In dangerous rock detection and segmentation tasks, calculating the normal vector of point cloud data is a feature extraction method. The normal vector of point cloud data can describe the geometric characteristics of the point cloud surface and is the basis for subsequent feature extraction and classification tasks. The following are the detailed steps for calculating the normal vector of dangerous rock point cloud data:

[0091] (1) Select the normal vector calculation method. Common normal vector calculation methods include neighborhood-based normal vector calculation, curvature-based normal vector calculation, least squares-based normal vector calculation, etc. By comprehensively considering the characteristics of dangerous rock point cloud data and calculation efficiency and other factors, the curvature-based normal vector calculation method is selected;

[0092] (2) Select the normal vector calculation window size. When performing normal vector calculation, you need to select a neighborhood window to calculate the normal vector of the point cloud data. The size of the neighborhood window can be selected according to the density and characteristics of the point cloud data. The selected neighborhood window size is one tenth of the number of point cloud data points.

[0093] (3) Calculate the normal vector. For each point, calculate the normal vector through the points in its neighborhood window. Use the curvature estimation method and the curvature direction estimation method to calculate the curvature-based normal vector;

[0094] (4) Normal vector filtering. After calculating the normal vector, there may be some erroneous normal vectors or outliers. In order to remove these outliers, the normal vector is smoothed using filtering methods such as Gaussian filtering and median filtering.

[0095] (5) Export. Finally, the calculated normal vector is saved as one of the attributes of the point cloud data, and the point cloud data is saved as a point cloud file in .bin format. The calculated normal vector can be used for subsequent dangerous rock detection and segmentation tasks.

[0096] 2.3 Curvature calculation

[0097] Curvature is an indicator that describes the change in curvature of the point cloud data surface.

[47] For dangerous rock detection, the curvature of the rock surface varies greatly. Curvature can be used to describe the shape and changes of the rock surface, so it is a very important feature. The following steps are usually used to calculate the curvature of point cloud data:

[0098] (1) Calculate the normal vector of the dangerous rock point cloud data. Before calculating the curvature, it is necessary to calculate the normal vector of the dangerous rock point cloud data. The calculation of the normal vector can be done by using methods such as nearest neighbor search and plane fitting;

[0099] (2) Calculate the covariance matrix of the dangerous rock point cloud data. When calculating the curvature, it is necessary to calculate the covariance matrix of each point. The covariance matrix can be used to describe the curvature change of the point cloud data surface. Specifically, the covariance matrix can calculate the curvature of the dangerous rock point cloud data in the normal vector direction and the tangential direction;

[0100] (3) Calculate the principal curvature and mean curvature of the point cloud data. By calculating the covariance matrix of the dangerous rock point cloud data, the principal curvature and mean curvature of the point cloud data can be obtained. The principal curvature describes the maximum and minimum values of the curvature change of the point cloud data surface in two orthogonal directions, and the mean curvature is the average value of the principal curvatures.

[0101] 2.4 Surface roughness calculation

[0102] Calculate surface roughness in dangerous rock detection and segmentation tasks

[48] It is a relatively effective feature extraction method that can be used to distinguish between different surface materials such as dangerous rocks and vegetation. The following are the detailed steps for calculating surface roughness of dangerous rock point cloud data:

[0103] (1) Calculate the normal vector. Before calculating the surface roughness, you need to calculate the normal vector of the point cloud data. Use the method mentioned in 2.3 to calculate the normal vector;

[0104] (2) Calculate the curvature of the point cloud data. Curvature can describe the curvature size and curvature direction of the point cloud surface at a certain point. The curvature of the point cloud data is obtained by calculating the normal vector and the curvature tensor, where the curvature tensor is a symmetric matrix calculated using the points in the neighborhood window of the point cloud data;

[0105] (3) Calculate the surface roughness of the point cloud data. The surface roughness of the point cloud data at a certain point is obtained by calculating the ratio between the principal curvature values of the curvature tensor. Generally, the larger the surface roughness value, the less smooth the surface at that point is, and the more likely it is a dangerous rock.

[0106] (4) Surface roughness filtering. After calculating the surface roughness, there may be some erroneous roughness values or outliers. In order to remove these outliers, the surface roughness is smoothed using filtering methods including Gaussian filtering and median filtering.

[0107] (5) Export. Finally, the calculated surface roughness is saved as one of the attributes of the point cloud data, and the point cloud data is saved as a point cloud file in .bin format. The calculated surface roughness is used for subsequent dangerous rock detection and segmentation tasks.

[0108] 3. Detection and segmentation of candidate dangerous rock targets

[0109] In the dangerous rock point cloud object detection phase, the VoteNet point cloud object detection algorithm is used based on the previously extracted features to detect and segment possible dangerous rock point cloud blocks. The entire network can be divided into two parts: one is the voting mechanism based on point cloud data; the other is the voting-based object localization and classification, which uses voting to locate and classify objects. In dangerous rock detection, the network processes point cloud data to generate candidate dangerous rock objects, classify them, and locate them.

[0110] 3.1 Voting mechanism based on point cloud data

[0111] Starting with an N×3 input point cloud, where each point has a 3D coordinate, the goal is to generate M votes, each with a 3D coordinate and a high-dimensional feature vector. This process consists of two main steps: point cloud feature learning via a backbone network and Hough voting learned from seed points. In the context of dangerous rock detection, this method can identify and locate dangerous rocks by learning features and votes from point clouds.

[0112] (1) Point cloud feature learning. Generating accurate votes requires geometric reasoning and context. Instead of relying on handcrafted features, we use the recently proposed deep network PointNet++ on point clouds as the backbone network.

[50] Perform point feature learning. The backbone network has multiple setting abstraction layers and feature propagation (upsampling) layers with skip connections, outputting a subset of input points with X, Y, and Z-dimensional feature vectors. The result is M seed points of dimension (3 + C). Each seed point generates a vote.

[0113] (2) Hough voting with deep networks. Compared to traditional Hough voting, where votes (offsets of local keypoints) are determined by lookups in a precomputed codebook, we generate votes using a deep network-based voting module, which is both more efficient (no KNN lookup required) and more accurate because it is jointly trained with the rest of the pipeline, and the shared voting module generates votes from each seed independently.

[0114] 3.2 Voting-based Target Positioning and Classification

[0115] Voting generates canonical "convergence points" between different parts of an object, which can be used to aggregate contextual information. After clustering these votes, their features can be aggregated to generate object proposals and perform classification. For the dangerous rock detection task, this step can be understood as clustering all voting points and using the cluster center as the proposed point for the dangerous rock object. These proposed points are then classified to determine whether they are true dangerous rock objects.

[0116] (1) Clustering of ballots by sampling and grouping. Although there are many ways to cluster ballots, we choose a simple strategy, which is clustering by uniform sampling and spatial neighbor grouping. Specifically, in a set of ballots In the example, we use the i}Sampling the farthest point in three-dimensional Euclidean space, a subset of K votes {v ik}, where k = 1,…,K. Then, by finding the distance from each v ik The 3D position nearest neighbor votes are used to form K clusters: C k ={v i (k) |‖v i ―v ik ‖≤r}, where k=1,…,K. Although simple, this clustering technique is easy to integrate into an end-to-end pipeline and performs well in practice for dangerous rock detection.

[0117] (2) Generate proposals and classifications from vote clusters. Since a vote cluster is essentially a set of high-dimensional points, a general point set learning network can be used to aggregate votes in order to generate object proposals. Compared with the backward tracking step of the traditional Hough Voting method, this process allows for the proposal of general boundaries from partial observations while predicting other parameters such as direction and category. In the implementation, a shared PointNet is used to aggregate and propose votes in a cluster. Given a vote cluster C = {w i}, where i = 1,…,n and its cluster center w j , where w i =[z i ;h i ], It's the ballot position. is the ballot feature. To make the local ballot geometry available, the ballot position is transformed into the local normalized coordinate system z′ i =(z i ―z j ) / r. Then, through a process similar to the PointNet module, the input set of the cluster is passed to the object proposal p(C) that generates the cluster. The formula is:

[0118]

[0119] The votes in each cluster are processed independently by an MLP1, then max-pooled (channel-wise) into a single feature vector and passed to MLP2, where information from different votes is further combined. The proposal p is represented as a multi-dimensional vector, which includes the object score, bounding box parameters (center, orientation and scale parameterization) and semantic classification score.

[0120] (3) Loss function setting. In the voting-based object localization and classification stage, objectivity, bounding box estimation and semantic classification losses are used. The objectivity scores of the supervised votes include those that are close to the actual object center (within 0.3 meters) or far away from any center (more than 0.6 meters). For the proposals generated by these votes, they are regarded as positive or negative proposals. The objectivity predictions of other proposals are not penalized. Cross-entropy loss is used to supervise objectivity, and the loss value is divided by the number of proposals that are not ignored in the batch. For positive proposals, bounding box estimation and category prediction are further supervised, and the closest actual bounding box is used. Specifically, the box loss is decomposed into center regression, offset angle estimation and box size estimation. For semantic classification, the standard cross-entropy loss is used. In all regressions of the detection loss, Huber (smooth L1) loss is used.

[0121] 4. Evaluation of candidate dangerous rock detection results

[0122] In order to objectively evaluate the detection performance of the model, common evaluation indicators, recognition accuracy, recall rate and single-frame detection time, are used as evaluation indicators for FAST candidate dangerous rock target detection.

[0123] 4.1 Recognition Accuracy and Recall

[0124] The accuracy of the model is measured by the proportion of correctly identified anomalies of a certain type to all actually labeled anomalies of that type. When calculating this, all samples are divided into four categories based on the prediction results (see Table 1): true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). TP represents the number of samples that are both predicted and actual anomalies of that type; FP represents the number of samples that are predicted to be anomalies of that type but are not; TN represents the number of samples that are neither predicted nor actual anomalies of that type; and FN represents the number of samples that are not predicted to be anomalies of that type but are actually anomalies of that type.

[0125] The calculation formula for recognition accuracy and recall rate is:

[0126] Accuracy = (TP + TN) / (TP + TN + FP + FN)

[0127] Recall = (TP) / (TP+FN)

[0128] Table 1: Possible classifications of test results:

[0129]

[0130] 4.2 Single frame detection time

[0131] It is used to measure the detection speed of the model, indicating the time required for the model to detect each frame of the image. The shorter the single-frame detection time, the faster the model's detection speed. The calculation formula for single-frame detection time is:

[0132] Detection time = end time - start time

[0133] The start time indicates the time when the model starts detection, and the end time indicates the time when the model completes detection.

[0134] FAST has been in operation for seven years and is subject to numerous impacts, including extreme weather conditions and natural disasters (such as rockfall). Structural damage caused by rockfall is a common structural anomaly. Damage to the parabolic structure of the reflector units affects electromagnetic wave reflection, thereby reducing the efficiency of weak signal collection. Corrosion of the steel frame, panel dents, and damage to the actuators and lower cables are largely caused by rockfall from the steep slopes surrounding the site. In May 2017, heavy rain and extreme weather caused a large rock to roll down the slope, directly striking the lower cables beneath the reflector. The lower cables impacted the node disk, damaging seven reflector units near that node. The total cost of repairing the seven reflector panels reached 1.1778 million yuan, and the maintenance work took five months to complete, severely impacting normal astronomical observations. This demonstrates the importance of detecting and preventing rockfall. Promptly detecting and intervening in potentially dangerous rockfalls on the surrounding steep slopes can effectively prevent damage to reflector panels, node disks, steel frames, and other components.

[0135] The area below the FAST site, approximately 260,000 square meters, has been eroded by years of rain, creating multiple unstable slopes and dangerous rock masses within the site area. These pose a direct threat to the safety of the reflecting surface and the safe operation of the FAST telescope. The present invention utilizes unmanned intelligent inspection equipment to intelligently identify and automatically determine dangerous rock masses. This method is an efficient and convenient method for identifying dangerous rock masses, enabling timely detection of problems and notification to the operation and maintenance team for maintenance. This method is safe, efficient, and cost-effective. The present invention's method for identifying dangerous rock masses at the FAST site, based on drone intelligent inspections, has important engineering application value.

Claims

1. A method for identifying dangerous rocks at the FAST site based on intelligent inspection by unmanned aerial vehicles, characterized in that: The dangerous rock identification method specifically includes: S1. Use drone inspections to perform point cloud downsampling, obtain radar data of rock walls near the FAST site, and obtain point cloud data of dangerous rocks; S2. Determine the voxel size and voxel center of the dangerous rock point cloud data, and determine the voxel center point based on a spherical region search method; S3. Extract the features of the dangerous rock target. For each voxel, extract its feature vector as the shape information of the dangerous rock target; S4. Detection of candidate dangerous rock targets: voxelize the collected point cloud data, use the feature extraction method to extract the normal vector of the point cloud as the feature vector to describe the shape information of the point cloud, and use the sliding window method to scan the point cloud data to obtain all possible candidate dangerous rock bodies.

2. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: In step S1, a voxel-based downsampling method is adopted to perform point cloud downsampling.

3. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: In step S2, a spherical region is constructed with the center point of each voxel as the sphere center and the voxel size as the radius, and then point cloud data is searched within the region to determine the actual center point of the voxel.

4. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: In step S3, the point cloud normal vector is used as the feature vector. For each point, its k nearest neighbor points are selected, the k neighbor points are fitted onto a plane, and then the normal vector of the plane is calculated as the normal vector of the point.

5. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: In step S4, a window of fixed size is slid in the point cloud data, and then feature extraction and classification are performed on the point cloud data in the window; during the classification process, a support vector machine classifier is used to classify dangerous rock targets with the feature vector as input, and the classification results are matched with the original point cloud data to obtain the location and size information of the dangerous rock targets.

6. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: In step S1, a laser radar is used to collect dangerous rock data by changing the rotation mode of the laser radar body to realize a non-repetitive scanning scheme; the repeated scanning only covers the horizontal field of view angle range of 70.4 degrees multiplied by 77.2 degrees.

7. The method for identifying dangerous rocks at the FAST site based on UAV intelligent inspection according to claim 6 is characterized in that: The echo mode of the laser radar is set to double echo, and the sampling frequency is selected to be 240KHz.

8. The FAST site dangerous rock identification method based on UAV intelligent inspection according to claim 1 is characterized in that: The steps of voxelizing the dangerous rock point cloud data are as follows: 1) Meshing: First, the dangerous rock point cloud data needs to be converted into mesh data. The software package Cloud Compare is used to perform meshing operations to convert the dangerous rock point cloud data into triangular mesh data. 2) Voxelization: Voxelize the grid data using the Voxel Grid voxelization algorithm and the Open3D software package to divide the three-dimensional space into several cubic voxels of equal size; 3) Reconstruction: After voxelization, the voxel point cloud data needs to be reconstructed. For each voxel, the mean of all points inside the voxel is used as the center point of the voxel. The reconstructed voxel point cloud data retains the local structural information of the original point cloud data and is adjusted by the size of the voxels to achieve different accuracy requirements. 4) Export: Export the reconstructed voxel point cloud data as a point cloud file in .bin format. The exported voxel point cloud data is used for subsequent dangerous rock detection and segmentation tasks.

9. The method for identifying dangerous rocks at the FAST site based on UAV intelligent inspection according to claim 1 is characterized in that: The steps for calculating the normal vector of the dangerous rock point cloud data are as follows: 1) Select the normal vector calculation method: Select the curvature-based normal vector calculation method; 2) Select the normal vector calculation window size: When performing normal vector calculation, you need to select a neighborhood window to calculate the normal vector of the point cloud data. The size of the neighborhood window is selected based on the density and characteristics of the point cloud data. The selected neighborhood window size is one-tenth of the number of point cloud data points. 3) Calculate the normal vector: For each point, calculate the normal vector through the points in its neighborhood window, and use the curvature estimation method and curvature direction estimation method to calculate the curvature-based normal vector; 4) Normal vector filtering: After calculating the normal vector, there will be some erroneous normal vectors or outliers. To remove these outliers, the normal vector is smoothed using filtering methods including Gaussian filtering and median filtering. 5) Export: Save the calculated normal vector as one of the attributes of the point cloud data, and save the point cloud data as a point cloud file in .bin format. The calculated normal vector is used for subsequent dangerous rock detection and segmentation tasks.

10. The method for identifying dangerous rocks at the FAST site based on intelligent inspection by unmanned aerial vehicles according to claim 9, characterized in that: The curvature calculation step includes: 1) Calculate the normal vector of the dangerous rock point cloud data: Before calculating the curvature, calculate the normal vector of the dangerous rock point cloud data first; 2) Calculate the covariance matrix of the dangerous rock point cloud data: Calculate the covariance matrix of each point, and calculate the curvature of the dangerous rock point cloud data in the normal vector direction and the tangential direction through the covariance matrix; 3) Calculate the principal curvature and mean curvature of the point cloud data: By calculating the covariance matrix of the dangerous rock point cloud data, the principal curvature and mean curvature of the point cloud data are obtained; The principal curvature describes the maximum and minimum values of the curvature change of the point cloud data surface in two orthogonal directions, and the mean curvature is the average value of the principal curvatures.

Images