Anchoring and spraying robot tunnel arch surface re-spraying area identification and positioning method

Abstract

The application provides a tunnel arch surface re-spraying area identification and positioning method of an anchor spraying robot, which comprises the following steps: a robot guide rail drives a laser radar to collect point cloud data of a sprayed arch surface after initial spraying of a tunnel; a computer accurately identifies and positions a missed spraying area according to the point cloud data, and adjusts the pose of a robot mechanical arm end spraying port according to position information to realize positioning and supplementary spraying of the re-spraying area; in the identification and positioning process, the computer filters out invalid ground points and outliers according to the characteristics of a target object, then uses a region growing algorithm based on local features to cluster and segment the missed spraying area in the tunnel arch surface point cloud and perform area screening, verifies the segmentation effect by extracting boundary information of the missed spraying area, and calculates the centroid coordinates of each re-spraying area to realize accurate positioning of the re-spraying area. The method can make the positioning result more accurate and stable when the robot re-sprays, and can reduce the work cost and improve the work efficiency in actual application.

Description

Technical Field

[0001] This invention relates to the field of 3D visual region recognition and positioning technology, specifically to a method for recognizing and locating the re-spraying area on the tunnel arch surface of an anchor spraying robot. Background Technology

[0002] With the rapid advancement of the integration of informatization and industrialization technologies, intelligent shotcrete robots have been widely used in industries such as automotive parts, furniture and home appliances, and building materials. To improve the efficiency of shotcrete construction on tunnel arch surfaces and prevent injuries from shotcrete rebounds and environmental dust, the use of intelligent shotcrete robots to replace traditional manual shotcrete has become an urgent need in practical engineering applications. As international research on intelligent mobile robot technologies continues to advance, the application areas of mobile robots are gradually expanding, and they have now become an important indicator of a country's level of informatization, social civilization, and technological innovation. Shotcrete robot application, as a new interdisciplinary field, involves multiple core technologies such as sensor technology, artificial intelligence, machine learning, and intelligent control, and has high scientific research and industrial application value.

[0003] Due to the poor lighting and high dust levels in tunnel environments, traditional visual sensors cannot meet the requirements of practical engineering applications. LiDAR, however, is less susceptible to interference from lighting and other environmental factors and has been widely used in complex industrial environments. Currently, there is limited research both domestically and internationally on the application of shotcrete robots in tunnel environments. To achieve automated control of shotcrete quality on tunnel arch surfaces using shotcrete robots, a method is needed to efficiently identify and locate areas with missed spraying or where the shotcrete quality does not meet engineering requirements, guiding the robot to complete targeted re-spraying. This will lay a solid foundation for unmanned intelligent shotcreting in future tunnel environments. Summary of the Invention

[0004] In response to the limitations of traditional visual sensors in existing tunnel environments due to poor lighting and high dust levels, which cannot meet the requirements of practical engineering applications, and in order to achieve automatic control of the quality of shotcrete on the tunnel arch surface by the anchor spraying robot, a method is needed to efficiently identify and locate areas with missed spraying or shotcrete quality that do not meet engineering requirements to guide the robot to complete the positioning and re-spraying. This invention provides a method for identifying and locating the re-spraying area on the tunnel arch surface of the anchor spraying robot. This method can accurately identify the re-spraying area and quickly, accurately, and stably complete the positioning and re-spraying of the re-spraying area.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A method for identifying and locating the re-spraying area of ​​a tunnel arch surface using a shotcrete robot is disclosed. This method employs a lidar to collect point cloud data of the sprayed arch surface after the initial spraying. The lidar is rotated 90° towards the Z-axis and installed in a protective device on the base of the shotcrete robot's robotic arm. The lidar is extended and retracted to achieve horizontal scanning even when partially obscured, preventing residual rebound slurry from the initial spraying from affecting the lidar's acquisition of environmental information during re-spraying area identification and location. The relative positions of the lidar and the robotic arm are fixed, both moving along the shotcrete robot's guide rail to expand the working range.

[0007] The method includes the following steps:

[0008] S1. Initialization: The robotic arm's axes return to their original state. The lidar extends from the protective device and moves along the guide rail without being obstructed by the robotic arm to collect point cloud data of the tunnel arch surface after initial spraying and transmit the data to the computer.

[0009] S2. The computer uses a filter to remove invalid ground points and outliers from the point cloud data to obtain valid point cloud target objects;

[0010] S3. Extract key points from the acquired valid point cloud and estimate the surface normal of the point cloud. Then, use FPFH descriptor features to describe the local geometric features of the point cloud. Then, use a region growing algorithm based on local features to cluster and segment the leaked spray area in the tunnel arch point cloud to obtain the preliminary re-spraying area segmentation object.

[0011] S4. Perform region filtering on the initially obtained re-spraying areas to extract the final re-spraying target objects;

[0012] S5. Extract boundary information based on the edge features of the re-sprayed area to verify the segmentation effect, and then calculate the centroid coordinates of each re-sprayed area and pass them to the computer as the positioning coordinates of the re-sprayed area.

[0013] S6. The computer adjusts the position of the spray nozzle at the end of the robotic arm according to the positioning coordinates to perform positioning and re-spraying of the re-spraying area, thereby completing a round of positioning and re-spraying of the collected object.

[0014] S7. The anchor spraying robot moves forward and repeats steps S1 to S6 to start the next round of re-spraying.

[0015] Furthermore, in step S1, the lidar only extends from the protective device when it receives a command from the computer to collect point cloud data, and retracts into the protective device after completing data collection. The lidar also establishes communication with the computer via an IP address.

[0016] Furthermore, in step S2, the computer uses a filter to remove invalid and outlier points on the ground to obtain valid point cloud target objects. Specific steps include:

[0017] S21. Input the initial point cloud dataset P. First, use the removeNaNFromPointCloud function in the PCL point cloud library to remove invalid points with a value of NAN from the point cloud.

[0018] S22. Then, introduce a ground point cloud filter and select any point P in the 3D point cloud. i (x i ,y i ,z i Set the radius R of the circular domain, calculate the slope value t between the point and its domain points. If the slope value t is less than the given slope threshold T, it is a ground point; otherwise, it is a tunnel arch point cloud object. Filter out the ground points according to the calculation results to obtain the point cloud dataset P'.

[0019] S23. Input P' into the statistical filter, set the neighborhood value k and the standard deviation threshold τ, and when the average distance d between a point and its neighborhood points... i When a point is located outside the range (μ-σ*τ, μ+σ*τ), it is denoted as an outlier; when P'={p i (x i ,y i ,z i When |1≤i≤n}, calculate the average distance d between it and all three-dimensional neighborhood points. i The global distance mean μ and standard deviation σ are calculated using the following formulas:

[0020]

[0021] Where m represents the number of point pairs, n represents the number of points in the point cloud, and d represents the distance between two points; using the above calculation formula, the average distance d i Points outside (μ±σ*τ) are filtered out;

[0022] S24. Output the valid point cloud target object.

[0023] Furthermore, in step S3, key points are extracted from the acquired effective point cloud and the surface normals of the point cloud are estimated. Then, the FPFH descriptor feature is used to describe the local geometric features of the point cloud. Finally, a region growing algorithm based on local features is used to cluster and segment the missed spray areas in the tunnel arch point cloud, thereby obtaining a preliminary segmentation object of the re-sprayed area. The specific steps include:

[0024] S31. Read in point cloud data, establish a kd-tree data structure to obtain the topological structure of the point cloud, divide the data in three-dimensional space, use it to make decisions on the branches of the tree, and quickly retrieve the domain information of points in the point cloud.

[0025] S32. Introduce an internal morphological signature algorithm to extract key points of the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud;

[0026] S33. Principal component analysis algorithm is used to estimate the surface normal of the point cloud to obtain the geometric features of the point cloud;

[0027] S34. Use FPFH to describe the local features of the point cloud geometry;

[0028] S35. After the above feature extraction is completed, the region growth algorithm based on local features is used to cluster and segment the missed spraying areas in the point cloud of the tunnel arch surface, thereby obtaining the preliminary segmentation objects of the re-spraying area.

[0029] Furthermore, step S32 introduces an internal morphological signature algorithm, which extracts key points from the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud. Specifically, this includes the following steps:

[0030] S321. Let the point cloud P = {p i (x i ,y i ,z i For any point p, |1≤i≤n} i Establish a local coordinate system and set the search radius p. r Determine p i For the center of the ball, p j For all points within the radius sphere, the covariance matrix is ​​constructed using the following formula:

[0031]

[0032] Among them, w ij The weights are calculated using the following formula:

[0033]

[0034] S322. Calculate the eigenvalues ​​{λ} of the covariance matrix using the moving least squares method. i1 ,λ i2 ,λ i3 And sort them in descending order;

[0035] S323. Set thresholds θ1 and θ2. If both conditions are met... and Then p i Use key points as the starting point, iterate until all key points are found.

[0036] Furthermore, in step S33, principal component analysis is used to estimate the surface normal of the point cloud to obtain the geometric features of the point cloud, specifically including the following steps:

[0037] S331, any point p in the point cloud i (x,y,z) and the set of points p within its circular radius R. k (1≤k≤m) can define a small local plane, and each two-dimensional plane can be defined by a point p on the plane. i and past point p i normal vector Let p be an example, using the moving least squares method. i and p k The local plane is fitted using the following formula:

[0038]

[0039] in, Let d be the normal vector of plane P, and d be the distance from plane P to the origin.

[0040] S332. Calculate p according to the principal component analysis algorithm. k The covariance matrix C∈R 3×3 Given the eigenvalues ​​and eigenvectors, we can obtain...

[0041]

[0042] Where, ξ i Point p i The weight is usually set to 1. For the domain point set p k The centroid is calculated using the following formula:

[0043]

[0044] S333. Calculate the eigenvalues ​​λ0, λ1, λ2 and eigenvectors of the covariance matrix C using the least squares method. Its λ and One-to-one correspondence; if 0 ≤ λ0 ≤ λ1 ≤ λ2, then the eigenvector corresponding to the smallest eigenvalue λ0 is... The point p can be approximated i normal at the location have to or in so It can be represented by a set of angle combinations (φ, θ):

[0045]

[0046] Where, n z The z-coordinate of the normal is represented by n. y The y-coordinate of the normal is represented by n. x Represents the x-coordinate of the normal;

[0047] S334, Calculate point p i The curvature τ at that point pi :

[0048]

[0049] S335. Since the observation points for the laser point cloud sampling values ​​are all the center v of the lidar coordinate system... p (0,0,0), to obtain optimized normal results, all normals should point towards the viewpoint, i.e., normals... The following conditions should be met:

[0050]

[0051] Therefore, when but otherwise This ensures the consistency of the normal vector direction of the point cloud surface and accurately extracts the geometric features of the point cloud.

[0052] Furthermore, step S34 uses FPFH descriptive eigenvalues ​​to perform local feature description of the point cloud's geometry, specifically including the following steps:

[0053] S341. There exist two points p within a circular region of radius R. s and p t First, with p s normal As the coordinate axis u, then u and point p s Point P t The coordinate axis v is obtained by taking the outer product of the unit vectors in the direction, and then the coordinate axis w is obtained by taking the outer product of the coordinate axis u and the coordinate axis v. s A local coordinate system uvw with the origin;

[0054] S342. Translate the coordinate system uvw to point p. t Let point p t normal The cosine of the line between the line and the coordinate axis v is α, and the normal line is... With point p s p t The cosine of the angle between the connecting lines is φ, and the normal is... Projection in plane uw If the angle between point p and the coordinate axis u is θ, then point p... qThe formula for calculating a triple <α,φ,θ> between itself and other points in its neighborhood is as follows:

[0055]

[0056] Where d=||p t -p s || represents p t and p s The distance between them;

[0057] S343. After completing the calculations in steps S341 to S342, denote them as SPFH, i.e., Simple Point Feature Histogram. Then, recalculate the neighborhood of the remaining points, calculating the Simple Point Feature Histogram value for each point to calculate p. q The Fast Point Feature Histogram (FPFH) is calculated using the following formula:

[0058]

[0059] Where, ω i For weights, point p is typically chosen. q Point P i The distance between them, where K represents the number of points in the neighborhood.

[0060] Furthermore, after the feature extraction in step S35 is completed, a region growing algorithm based on local features is used to cluster and segment the missed spraying areas in the point cloud of the tunnel arch surface, thereby obtaining preliminary segmented objects of the re-spraying areas. Specifically, this includes the following steps:

[0061] S351. Select seed points and sort them according to the curvature values ​​of the points in the point cloud. Select the point with the smallest curvature as the initial seed point.

[0062] S352, using the seed point p with the minimum curvature. q =(x i ,y i ,z i R is the center of the spherical region. m Let p be the radius. q The normal and curvature are compared with other sampling points within the radius. If a point has the lowest curvature and its normal direction is close enough, it is used as a new seed point to redefine its respective spherical domain. Otherwise, it is not used as a seed point. This process is repeated to grow in parallel in three-dimensional space.

[0063] S353. Growth stops when the spherical neighborhood of each new seed point no longer contains any unclustered points. At this point, the output data structure is an array composed of clustering results.

[0064] Furthermore, in step S4, the initially obtained re-spraying area is filtered to extract the final re-spraying target object, specifically including the following steps:

[0065] S41. Introduce the depth information of the laser radar to the sprayed surface before and after spraying. Based on the curvature estimation, combine the depth information to determine whether the area has changed. If the depth information and curvature information remain unchanged, it is judged as a missed spraying area. If the depth information is less than the depth before spraying and the curvature increases, it indicates that the area is not a missed spraying area. This enables the screening of the re-spraying area and obtains the accurate re-spraying area.

[0066] S42. The final clustering information output by the algorithm only includes the identified re-sprayed regions, which are stored in an array.

[0067] Furthermore, in step S5, boundary information is extracted based on the edge features of the re-sprayed area to verify the segmentation effect, and then the centroid coordinates of each re-sprayed area are calculated and used as the positioning coordinates of the re-sprayed area to be transmitted to the computer. Specifically, this includes the following steps:

[0068] S51. Extract boundary information from the segmented regions stored in the array;

[0069] S52. Calculate the centroid coordinates of the i-th re-spraying region in the point cloud of the tunnel arch surface.

[0070]

[0071] Where k represents the number of points, p(x i ,y i ,z i () represents the three-dimensional coordinates of point p in the point cloud;

[0072] S53. Set the centroid coordinates of each re-spraying area. The data is transmitted to the computer to achieve precise positioning of the re-spraying area;

[0073] S54. Quantitative analysis methods are used to evaluate the recognition accuracy and algorithm time.

[0074] Compared with existing technologies, the method for identifying and locating the re-spraying area on the tunnel arch surface provided by this invention first uses a laser radar driven by the anchor-spraying robot's guide rail to collect point cloud data of the arch surface after the initial spraying of the tunnel. Then, the computer accurately identifies and locates the missed spraying areas using the point cloud data, and adjusts the pose of the spray nozzle at the end of the anchor-spraying robot's robotic arm based on the location information to achieve re-spraying of the re-spraying area. During the identification and location process, the computer filters out invalid points and outliers on the ground based on the characteristics of the target object, thereby increasing the accuracy of re-spraying area identification and location. Then, surface normal estimation is performed on the point cloud, and local feature description is performed using FPFH to obtain the geometric features of the sprayed surface. To better... This method identifies areas with uneven shotcrete thickness and uses a region growing algorithm to cluster and segment the re-spraying areas, then filters these regions. By extracting the boundary information of the re-spraying areas and calculating their centroid coordinates, these coordinates are transmitted to the computer as the positioning coordinates of the re-spraying areas, achieving precise positioning of the re-spraying areas. The computer adjusts the end pose of the shotcrete nozzle of the anchor shotcrete robot's robotic arm based on the centroid coordinates to accurately, quickly, and stably complete the positioning and re-spraying of the re-spraying areas. The identification and positioning of the re-spraying areas by the anchor shotcrete robot are both automatically controlled by the computer, making the shotcrete work more intelligent. Therefore, this method can make the positioning results of the anchor shotcrete robot more accurate and stable during re-spraying, and can reduce working costs and improve work efficiency in practical applications. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the process for identifying and locating the re-spraying area on the tunnel arch surface using an anchor spraying robot, provided by the present invention.

[0076] Figure 2 This is a schematic diagram of the anchor spraying robot component structure in this invention.

[0077] Figure 3 This is a schematic diagram of the internal structure of the laser radar protection device provided by the present invention.

[0078] Figure 4 This is a schematic diagram of the extended and retracted states of the lidar provided by the present invention, wherein Figure (a) is a schematic diagram of the extended state and Figure (b) is a schematic diagram of the retracted state.

[0079] Figure 5 This is a schematic diagram of the horizontal scanning range of the lidar provided by the present invention.

[0080] Figure 6 This is a schematic diagram of point cloud surface normal estimation and normal direction consistency adjustment provided by the present invention.

[0081] Figure 7 This is a schematic diagram of the FPFH feature description principle provided by the present invention, wherein Figure (a) is the FPFH calculation area and Figure (b) is the deviation of the normals between two points.

[0082] In the diagram, 1 is the robot's mobile platform; 2 is the guide rail; 3 is the robotic arm; 4 is the spray nozzle; 5 is the protective device; 51 is the protective cover; 52 is the hydraulic cylinder; 53 is the enclosed piston; and 6 is the lidar. Detailed Implementation

[0083] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0084] Please refer to Figure 1 As shown, this invention provides a method for identifying and locating the re-spraying area of ​​the tunnel arch surface using a shotcrete robot. In this method, a lidar is used to collect point cloud data of the sprayed arch surface after the initial spraying of the tunnel. The lidar is rotated 90° in the Z-axis direction and installed in a protective device on the base of the shotcrete robot's robotic arm. The horizontal scanning is achieved after the lidar is partially blocked by the extension and retraction operation, preventing the residual rebound slurry after the initial spraying from affecting the environmental information collected by the lidar during the identification and location of the re-spraying area. The relative positions of the lidar and the robotic arm are fixed, and both move with the guide rail of the shotcrete robot to expand the working range.

[0085] The method includes the following steps:

[0086] S1. Initialization: The robotic arm's axes return to their original state. The lidar extends from the protective device and moves along the guide rail without being obstructed by the robotic arm to collect point cloud data of the tunnel arch surface after initial spraying and transmit the data to the computer.

[0087] S2. The computer uses a filter to remove invalid ground points and outliers from the point cloud data to obtain valid point cloud target objects;

[0088] S3. Extract key points from the acquired valid point cloud and estimate the surface normal of the point cloud. Then, use FPFH descriptor features to describe the local geometric features of the point cloud. Then, use a region growing algorithm based on local features to cluster and segment the leaked spray area in the tunnel arch point cloud to obtain the preliminary re-spraying area segmentation object.

[0089] S4. Perform region filtering on the initially obtained re-spraying areas to extract the final re-spraying target objects;

[0090] S5. Extract boundary information based on the edge features of the re-sprayed area to verify the segmentation effect, and then calculate the centroid coordinates of each re-sprayed area and pass them to the computer as the positioning coordinates of the re-sprayed area.

[0091] S6. The computer adjusts the position of the spray nozzle at the end of the robotic arm according to the positioning coordinates to perform positioning and re-spraying of the re-spraying area, thereby completing a round of positioning and re-spraying of the collected object.

[0092] S7. The anchor spraying robot moves forward and repeats steps S1 to S6 to start the next round of re-spraying.

[0093] For a specific embodiment, please refer to Figure 2 As shown, the specific structure of the anchor spraying robot is existing technology well known to those skilled in the art. It mainly consists of multiple components such as a robot mobile platform 1, guide rail 2, robotic arm 3, and spraying nozzle 4. A lidar 6 is installed in the protective device 5 on the base of the robotic arm of the anchor spraying robot, and the lidar simulates construction in the tunnel. Two spraying modes are designed in the entire anchor spraying robot spraying program control process: initial spraying mode and re-spraying mode. The initial spraying mode constructs the tunnel arch surface according to the initially planned spraying path, without judging the quality of the sprayed arch surface; while the re-spraying mode requires 3D visual recognition of the sprayed surface after the initial spraying to detect areas with missed spraying or where the spraying thickness does not meet engineering requirements, and then performs targeted re-spraying. This invention mainly studies the re-spraying mode. In a specific implementation, the lidar uses an existing LR-16FIS-C1 16-line explosion-proof lidar as the data acquisition unit to collect point cloud data of the sprayed arch surface after the initial spraying of the tunnel. This explosion-proof lidar has a vertical field of view of (-15° to +15°), a horizontal field of view of 360°, and a measurement range of 0.05m to 120m. The measurement data is based on the radar coordinate origin. Please refer to... Figures 3 to 5 As shown, in order to improve the efficiency of the lidar, a sealing method is used to protect the lidar. Then, the lidar is rotated 90° in the Z-axis direction and installed in the protective device 5 on the base of the anchor spraying robot's robotic arm. After receiving the command to collect data, the lidar is extended and retracted to achieve a horizontal 238.14° scan to obtain point cloud after partial obstruction. This prevents the residual rebound slurry after the initial spraying from affecting the lidar's acquisition of environmental information during the identification and positioning of the re-spraying area. The relative positions of the lidar 6 and the robotic arm 3 are fixed. Both follow the guide rail of the anchor spraying robot to expand the working range. The guide rail 2 is 1.6m long. Specifically, the protective device 5 includes a protective cover 51, a hydraulic cylinder 52, and a sealing piston 53. The protective cover 51 is a cylindrical structure, and an end cap is fixedly connected to the right end face of the cylindrical protective cover 51. The hydraulic cylinder 52 is fixedly sleeved in the middle of the end cap. The sealing piston 53 is slidably fitted with the inner wall of the protective cover 51 and fixedly connected to the piston rod of the hydraulic cylinder 52. The lidar 6 is fixedly connected to the sealing piston 53. The lidar 6 extends outward and retracts inward under the action of the piston rod of the hydraulic cylinder 52 via the sealing piston 53. In this embodiment, the operating system in the computer is Ubuntu 16.04, the development platform is ROS, and the open-source library is the PCL point cloud library based on C++.

[0094] As a specific embodiment, to prevent the rebound of the sprayed concrete from affecting the lidar, in step S1, the lidar only extends from the protective device when it receives a command from the computer to collect point cloud data, and retracts into the protective device after data acquisition is completed. The lidar establishes communication with the computer via IP address, and the robotic arm base rail is mounted on the anchor spraying robot's mobile platform. Furthermore, before identifying and locating the re-spraying area, due to the inconsistency between the lidar's base coordinates and the robotic arm base's base coordinates caused by the installation method, the base coordinates of the point cloud data are converted after data acquisition to ensure consistency with the robotic arm base's base coordinates to meet practical application requirements.

[0095] As a specific embodiment, in step S2, due to the complex tunnel environment and high dust levels, invalid ground points and outliers are easily generated. To improve the accuracy of point cloud recognition and enhance the efficiency of the algorithm, a ground point cloud filter and a statistical filter are introduced in this stage to filter out invalid ground points and outliers in the point cloud data, ensuring that valid point cloud target objects are obtained. Therefore, in step S2, the computer uses filters to filter out invalid ground points and outliers to obtain valid point cloud target objects. The specific steps include:

[0096] S21. Input the initial point cloud dataset P. First, use the removeNaNFromPointCloud function in the PCL point cloud library to remove invalid points with a value of NAN from the point cloud.

[0097] S22. Then, introduce a ground point cloud filter and select any point P in the 3D point cloud. i (x i ,y i ,z i Set the radius R of the circular domain, calculate the slope value t between the point and its domain points. If the slope value t is less than the given slope threshold T, it is a ground point; otherwise, it is a tunnel arch point cloud object. Filter out the ground points according to the calculation results to obtain the point cloud dataset P'.

[0098] S23. Input P' into the statistical filter, set the neighborhood value k and the standard deviation threshold τ, and when the average distance d between a point and its neighborhood points... i When a point is located outside the range (μ-σ*τ, μ+σ*τ), it is denoted as an outlier; when P'={p i (x i ,y i ,z i When |1≤i≤n}, calculate the average distance d between it and all three-dimensional neighborhood points. i The global distance mean μ and standard deviation σ are calculated using the following formulas:

[0099]

[0100] Where m represents the number of point pairs, n represents the number of points in the point cloud, and d represents the distance between two points; using the above calculation formula, the average distance d i Points outside (μ±σ*τ) are filtered out;

[0101] S24. Output the valid point cloud target object.

[0102] By adopting the technical solution in the above embodiments, invalid ground points and outliers in the original point cloud are filtered out, thereby accelerating the speed of subsequent data processing, improving the accuracy of re-spraying area identification and positioning to a certain extent, and effectively improving the algorithm's computing speed.

[0103] In a specific embodiment, step S3 involves extracting key points from the acquired effective point cloud and estimating the surface normal of the point cloud. Then, the FPFH descriptor feature is used to describe the local geometric features of the point cloud. Finally, a region growing algorithm based on local features is used to cluster and segment the missed spray areas in the tunnel arch point cloud, thereby obtaining a preliminary segmentation object for the re-sprayed region. The specific steps include:

[0104] S31. Read in point cloud data, establish a kd-tree data structure to obtain the topological structure of the point cloud, divide the data in three-dimensional space, use it to make decisions on the branches of the tree, and quickly retrieve the domain information of points in the point cloud.

[0105] S32. Introduce the Intrinsic Shape Signatures (ISS) algorithm to extract key points of the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud.

[0106] S33. Use Principal Component Analysis (PCA) algorithm to estimate the surface normal of the point cloud to obtain the geometric features of the point cloud;

[0107] S34. Use FPFH (Fast Point Feature Histograms) to describe the local features of the point cloud geometry using feature pairs.

[0108] S35. After the above feature extraction is completed, the region growth algorithm based on local features is used to cluster and segment the missed spraying areas in the point cloud of the tunnel arch surface, thereby obtaining the preliminary segmentation objects of the re-spraying area.

[0109] Using the technical solutions in the above embodiments, in view of the irregular distribution of data after point cloud preprocessing, kd-tree is used to establish the topological relationship between point clouds, which speeds up the search speed between neighboring points; then, key points are extracted, geometric features are estimated and local features are described for the missed spray areas after the initial spraying of the tunnel arch surface. At the same time, in order to solve the problem of uneven spray thickness, a region growing algorithm is used to grow some regions, which basically realizes the clustering and segmentation of the missed spray areas.

[0110] As a specific embodiment, step S32 introduces an internal morphological signature algorithm, which extracts key points of the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud. The specific steps include:

[0111] S321. Let the point cloud P = {p i (x i ,y i ,z i For any point p, |1≤i≤n} i Establish a local coordinate system and set the search radius p. r Determine p i For the center of the ball, p j For all points within the radius sphere, the covariance matrix is ​​constructed using the following formula:

[0112]

[0113] Among them, w ij The weights are calculated using the following formula:

[0114]

[0115] S322. Calculate the eigenvalues ​​{λ} of the covariance matrix using the moving least squares method. i1 ,λ i2 ,λ i3 And sort them in descending order;

[0116] S323. Set thresholds θ1 and θ2. If both conditions are met... and Then p i Use key points as the starting point, iterate until all key points are found.

[0117] In a specific embodiment, step S33 uses principal component analysis to estimate the surface normal of the point cloud to obtain the geometric features of the point cloud, specifically including the following steps:

[0118] S331, any point p in the point cloud i (x,y,z) and the set of points p within its circular radius R. k(1≤k≤m) can define a small local plane, and each two-dimensional plane can be defined by a point p on the plane. i and past point p i normal vector Let p be an example, using the moving least squares method. i and p k The local plane is fitted using the following formula:

[0119]

[0120] in, Let d be the normal vector of plane P, and d be the distance from plane P to the origin.

[0121] S332. Calculate p according to the principal component analysis algorithm. k The covariance matrix C∈R 3×3 Given the eigenvalues ​​and eigenvectors, we can obtain...

[0122]

[0123] Where, ξ i Point p i The weight is usually set to 1. For the domain point set p k The centroid is calculated using the following formula:

[0124]

[0125] S333. Calculate the eigenvalues ​​λ0, λ1, λ2 and eigenvectors of the covariance matrix C using the least squares method. Its λ and One-to-one correspondence; if 0 ≤ λ0 ≤ λ1 ≤ λ2, then the eigenvector corresponding to the smallest eigenvalue λ0 is... The point p can be approximated i normal at the location have to or in so It can be represented by a set of angle combinations (φ, θ):

[0126]

[0127] Where, n z The z-coordinate of the normal is represented by n. y The y-coordinate of the normal is represented by n. x Represents the x-coordinate of the normal;

[0128] S334, Calculate point p i The curvature τ at that point pi :

[0129]

[0130] S335. After obtaining the above calculation results, the result obtained at this time is... Direction is ambiguous and difficult to determine; therefore, a viewpoint-centered normal consistency adjustment is adopted, specifically as follows: Figure 6 As shown. Since the observation points for the laser point cloud sampling values ​​are all at the center v of the lidar coordinate system. p (0,0,0), to obtain optimized normal results, all normals should point towards the viewpoint, i.e., normals... The following conditions should be met:

[0131]

[0132] Therefore, when but otherwise This ensures the consistency of the normal vector direction of the point cloud surface and accurately extracts the geometric features of the point cloud.

[0133] As a specific implementation, after extracting the point cloud normals, in order to better describe the relationship between the sampled point and its neighborhood points, the FPFH descriptor feature is used to calculate the sampled point p. q The fast point feature histogram, the principle is as follows: Figure 7 As shown, Figure (a) shows the computational region of FPFH, and Figure (b) is a schematic diagram of the calculation of the deviation between the normals of two points. Specifically, step S34 uses FPFH descriptive features to perform local feature description of the geometry of the point cloud, including the following steps:

[0134] S341. There exist two points p within a circular region of radius R. s and p t First, with p s normal As the coordinate axis u, then u and point p s Point P t The coordinate axis v is obtained by taking the outer product of the unit vectors in the direction, and then the coordinate axis w is obtained by taking the outer product of the coordinate axis u and the coordinate axis v. s A local coordinate system uvw with the origin;

[0135] S342. Translate the coordinate system uvw to point p. t Let point p t normal The cosine of the line between the line and the coordinate axis v is α, and the normal line is... With point p s p t The cosine of the angle between the connecting lines is φ, and the normal is... Projection in plane uw If the angle between point p and the coordinate axis u is θ, then point p... q The formula for calculating a triple <α,φ,θ> between itself and other points in its neighborhood is as follows:

[0136]

[0137] Where d=||p t -p s || represents p t and p s The distance between them;

[0138] S343. After completing the calculations in steps S341 to S342, denote them as SPFH, i.e., Simple Point Feature Histogram. Then, recalculate the neighborhood of the remaining points, calculating the Simple Point Feature Histogram value for each point to calculate p. q The Fast Point Feature Histogram (FPFH) is calculated using the following formula:

[0139]

[0140] Where, ω i For weights, point p is typically chosen. q Point P i The distance between them, where K represents the number of points in the neighborhood.

[0141] As a specific embodiment, after obtaining the local features of the point cloud data, based on the curvature changes of the sprayed and unsprayed surfaces, the point cloud data stored in the kd-tree is segmented into regions according to the idea of ​​the region growing algorithm, and connected regions with the same unsprayed features are segmented out; specifically, after the above feature extraction is completed in step S35, the unsprayed regions in the tunnel arch surface point cloud are clustered and segmented using a region growing algorithm based on local features, thereby obtaining the preliminary resprayed region segmentation object, including the following steps:

[0142] S351. Select seed points and sort them according to the curvature values ​​of the points in the point cloud. Select the point with the smallest curvature as the initial seed point.

[0143] S352, using the seed point p with the minimum curvature. q =(x i ,y i ,z i R is the center of the spherical region. m Let p be the radius. q The normal and curvature are compared with other sampling points within the radius. If a point has the lowest curvature and its normal direction is close enough, it is used as a new seed point to redefine its respective spherical domain. Otherwise, it is not used as a seed point. This process is repeated to grow in parallel in three-dimensional space.

[0144] S353. Growth stops when the spherical neighborhood of each new seed point no longer contains any unclustered points. At this point, the output data structure is an array composed of clustering results.

[0145] In a specific embodiment, step S4 involves filtering the initially obtained re-spraying areas to extract the final re-spraying target objects, specifically including the following steps:

[0146] S41. Since the point cloud of the tunnel arch is a non-planar model, it is not possible to directly filter regions based on changes in curvature values. Therefore, the depth information of the laser radar to the sprayed surface before and after shotcreting is introduced. Based on curvature estimation, the depth information is combined to determine whether the region has changed. If the depth and curvature information remain unchanged, it is judged as a missed spraying region. If the depth information is less than the depth before shotcreting and the curvature increases, it indicates that the region is not a missed spraying region. This enables the screening of re-spraying regions and obtains accurate re-spraying regions.

[0147] S42. The final clustering information output by the algorithm only includes the identified re-sprayed regions, which are stored in an array.

[0148] In a specific embodiment, step S5 involves extracting boundary information based on the edge features of the segmented re-spraying regions to verify the segmentation effect, and then calculating the centroid coordinates of each re-spraying region, which are then used as the positioning coordinates of the re-spraying regions and transmitted to the computer. The steps include:

[0149] S51. Extract boundary information from the segmented regions stored in the array. Specifically, for each sampling point, use the nearest neighbor search algorithm to search the kd-tree to quickly retrieve the k-neighbor points of the point, and then use the least squares method to fit the tangent plane. Project the sampling point and the k-neighbor points onto the tangent plane, and define a vector with the projection point of the sampling point as the starting point and the projection point of the neighbor points as the ending point. Randomly select a vector as the reference vector, and take the cross product of the reference vector and the normal of the tangent plane to obtain the vector v. Then calculate the angle between the remaining vector and the reference vector and the vector v. Finally, sort the obtained angle values ​​to obtain the maximum angle. When the maximum angle is greater than the angle threshold, the point is the boundary point, thus obtaining the boundary information of the re-sprayed region.

[0150] S52. Calculate the centroid coordinates of the i-th re-spraying region in the point cloud of the tunnel arch surface.

[0151]

[0152] Where k represents the number of points, p(x i ,y i ,z i () represents the three-dimensional coordinates of point p in the point cloud;

[0153] S53. Set the centroid coordinates of each re-spraying area. The data is transmitted to the computer to achieve precise positioning of the re-spraying area;

[0154] S54. Quantitative analysis methods are used to evaluate the algorithm's recognition accuracy and processing time; specifically, the first evaluation indicator is the recognition accuracy R. c The following formula is used for calculation:

[0155]

[0156] Where, N s N represents the number of correctly segmented point cloud subsets, and N represents the total number of targets.

[0157] The second evaluation metric, algorithm execution time T, is obtained by recording the running time of the algorithm.

[0158] Using the technical solution in the above embodiments, in order to verify the accuracy of the segmented object of the re-spraying area and achieve accurate re-spraying area positioning, the boundary information of the segmented area was extracted, and the centroid coordinates of the re-spraying area were calculated. Here, the centroid is the positioning center of the identified re-spraying area. Finally, the effect of the algorithm was evaluated.

[0159] Compared with existing technologies, the method for identifying and locating the re-spraying area on the tunnel arch surface provided by this invention first uses a laser radar driven by the anchor-spraying robot's guide rail to collect point cloud data of the arch surface after the initial spraying of the tunnel. Then, the computer accurately identifies and locates the missed spraying areas using the point cloud data, and adjusts the pose of the spray nozzle at the end of the anchor-spraying robot's robotic arm based on the location information to achieve re-spraying of the re-spraying area. During the identification and location process, the computer filters out invalid points and outliers on the ground based on the characteristics of the target object, thereby increasing the accuracy of re-spraying area identification and location. Then, surface normal estimation is performed on the point cloud, and local feature description is performed using FPFH to obtain the geometric features of the sprayed surface. To better... This method identifies areas with uneven shotcrete thickness and uses a region growing algorithm to cluster and segment the re-spraying areas, then filters these regions. By extracting the boundary information of the re-spraying areas and calculating their centroid coordinates, these coordinates are transmitted to the computer as the positioning coordinates of the re-spraying areas, achieving precise positioning of the re-spraying areas. The computer adjusts the end pose of the shotcrete nozzle of the anchor shotcrete robot's robotic arm based on the centroid coordinates to accurately, quickly, and stably complete the positioning and re-spraying of the re-spraying areas. The identification and positioning of the re-spraying areas by the anchor shotcrete robot are both automatically controlled by the computer, making the shotcrete work more intelligent. Therefore, this method can make the positioning results of the anchor shotcrete robot more accurate and stable during re-spraying, and can reduce working costs and improve work efficiency in practical applications.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying and locating the re-spraying area on the tunnel arch surface using a spraying robot, characterized in that, In this method, a lidar is used to collect point cloud data of the arch surface to be sprayed after the initial spraying of the tunnel. The lidar is rotated 90° in the Z-axis direction and installed in the protective device of the anchor spraying robot's robotic arm base. The horizontal scanning after partial obstruction of the lidar is achieved by extending and retracting the lidar. The relative positions of the lidar and the robotic arm are fixed, and both move with the anchor spraying robot's guide rail to expand the working range. The method includes the following steps: S1. Initialization: The robotic arm's axes return to their original state. The lidar extends from the protective device and moves along the guide rail without being obstructed by the robotic arm to collect point cloud data of the tunnel's arch surface after initial spraying, and transmits the data to the computer. Furthermore, the lidar only extends from the protective device when it receives a command from the computer to collect point cloud data, and retracts into the protective device after completing data collection. The lidar also establishes communication with the computer via IP address. S2. The computer uses a filter to remove invalid ground points and outliers from the point cloud data to obtain valid point cloud target objects; S3. Extract key points from the acquired valid point cloud and estimate the surface normal of the point cloud. Then, use FPFH descriptor features to describe the local geometric features of the point cloud. Then, use a region growing algorithm based on local features to cluster and segment the leaked spray area in the tunnel arch point cloud to obtain the preliminary re-spraying area segmentation object. S4. Perform region filtering on the initially obtained re-spraying areas to extract the final re-spraying target objects; S5. Extract boundary information based on the edge features of the re-sprayed area to verify the segmentation effect, and then calculate the centroid coordinates of each re-sprayed area and pass them to the computer as the positioning coordinates of the re-sprayed area. S6. The computer adjusts the position of the spray nozzle at the end of the robotic arm according to the positioning coordinates to perform positioning and re-spraying of the re-spraying area, thereby completing a round of positioning and re-spraying of the collected object. S7. The anchor spraying robot moves forward and repeats steps S1-S6 to begin the next round of re-spraying; among which... Step S2 specifically includes the following steps: S21. Input the initial point cloud dataset. First, use the removeNaNFromPointCloud function in the PCL point cloud library to remove invalid points with a value of NAN from the point cloud; S22. Then, introduce a ground point cloud filter and select any point in the 3D point cloud. Set the radius of the circular neighborhood. Calculate the slope between this point and its neighboring points. If the slope value All are less than the given slope threshold If the ground points are not found, they are considered ground points; otherwise, they are considered point cloud objects on the tunnel arch surface. Ground points are filtered out based on the calculation results to obtain the point cloud dataset. ; S23, will As input to the statistical filter, set the neighborhood value. and standard deviation multiple threshold When the average distance between a point and its neighboring points lie in When outside the range, it is recorded as an outlier; when At that time, calculate the average distance between it and all its three-dimensional neighbors. and global average distance and standard deviation The calculation formula is as follows: in, Indicates the number of point pairs. Indicates the number of points in the point cloud. This represents the distance between two points; using the above formula, the average distance... exist Points other than those specified are filtered out. S24. Output the valid point cloud target object; Step S3 specifically includes the following steps: S31. Read in point cloud data, establish a kd-tree data structure to obtain the topological structure of the point cloud, divide the data in three-dimensional space, use it to make decisions on the branches of the tree, and quickly retrieve the neighborhood information of points in the point cloud. S32. Introduce an internal morphological signature algorithm to extract key points of the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud. S33. Principal component analysis algorithm is used to estimate the surface normal of the point cloud to obtain the geometric features of the point cloud; S34. Use FPFH to describe the local features of the point cloud geometry; S35. After the above feature extraction is completed, a region growing algorithm based on local features is used to cluster and segment the missed spraying areas in the point cloud of the tunnel arch surface, thereby obtaining preliminary segmented objects of the re-spraying areas; this step further includes: S351. Select seed points and sort them according to the curvature values ​​of the points in the point cloud. Select the point with the smallest curvature as the initial seed point. S352, using the seed point with the minimum curvature. The center of the spherical neighborhood, For radius, The normal and curvature are compared with other sampling points within the radius. If a point has the lowest curvature and its normal direction is close enough, it is used as a new seed point to redefine its respective spherical neighborhood. Otherwise, it is not used as a seed point. This process is repeated to grow in parallel in three-dimensional space. S353. Growth stops when the spherical neighborhood of each new seed point no longer contains any unclustered points. At this point, the output data structure is an array composed of clustering results. Step S4 specifically includes the following steps: S41. Introduce the depth information of the laser radar to the sprayed surface before and after spraying. Based on the curvature estimation, combine the depth information to determine whether the area has changed. If the depth information and curvature information remain unchanged, it is judged as a missed spraying area. If the depth information is less than the depth before spraying and the curvature increases, it indicates that the area is not a missed spraying area. This enables the screening of the re-spraying area and obtains the accurate re-spraying area. S42. The final clustering information output by the algorithm only includes the identified re-spraying regions, which are stored in an array; Step S5 specifically includes the following steps: S51. Extract boundary information from the segmented regions stored in the array; S52, Calculate the first point cloud of the tunnel arch surface. The centroid coordinates of the re-spraying area : in, Indicates the number of points. In the point cloud The three-dimensional coordinates of a point; S53. Set the centroid coordinates of each re-spraying area. The data is transmitted to the computer to achieve precise positioning of the re-spraying area; S54. Quantitative analysis methods are used to evaluate the recognition accuracy and algorithm time.

2. The method for identifying and locating the re-spraying area on the tunnel arch surface using a spraying robot according to claim 1, characterized in that, Step S32 introduces an internal morphological signature algorithm, which extracts key points from the point cloud by analyzing the neighborhood weighted covariance matrix of a point in the point cloud. Specifically, the steps include: S321, Point Cloud For any point Establish a local coordinate system and set the search radius. ,Sure For the center of the ball, For all points within the radius sphere, the covariance matrix is ​​constructed using the following formula: in, The weights are calculated using the following formula: S322. Calculate the eigenvalues ​​of the covariance matrix using the moving least squares method. And sort in descending order; S323, Set threshold and If both conditions are met and ,but Use key points as the starting point, iterate until all key points are found.

3. The method for identifying and locating the re-spraying area on the tunnel arch surface using a spraying robot according to claim 1, characterized in that, Step S33 uses principal component analysis to estimate the surface normals of the point cloud to obtain the geometric features of the point cloud, specifically including the following steps: S331, Any point in the point cloud and its circular neighborhood radius Neighborhood point set A small local plane can be defined, and each two-dimensional plane can be defined using a point on the plane. and over the point normal vector To represent, the moving least squares method is used to... and The local plane is fitted using the following formula: in, For plane The normal vector, For plane Distance to the origin; S332. Calculate based on principal component analysis algorithm covariance matrix Eigenvalues ​​and eigenvectors: in, Point The weight is set to 1. For neighborhood point set The centroid is calculated using the following formula: S333, Calculate the covariance matrix using the least squares method. eigenvalues and eigenvectors ,That and One-to-one correspondence; if Then the smallest eigenvalue Corresponding feature vector Approximate representation of points normal at the location ,have to or ,in ,so A set of angles can be used. express: in, Representing the normal coordinate, Representing the normal coordinate, Representing the normal coordinate; S334, Calculation Point curvature at : S335. Since the observation points of the laser point cloud sampling values ​​are all at the center of the lidar coordinate system... To obtain optimized normal results, all normals should point towards the viewpoint, i.e., normals... The following conditions should be met: Therefore, when ,but ,otherwise This ensures the consistency of the normal vector direction of the point cloud surface and accurately extracts the geometric features of the point cloud.

4. The method for identifying and locating the re-spraying area on the tunnel arch surface using a spraying robot according to claim 1, characterized in that, Step S34 uses FPFH descriptive features to perform local feature description of the point cloud's geometry, specifically including the following steps: S341, There exists a circular neighborhood radius. Two points inside and First of all normal As coordinate axes ,Then With point Time The coordinate axes are obtained by taking the outer product of the unit vectors of the directions. Then use the coordinate axes with coordinate axes The coordinate axes are obtained by performing the outer product. , to obtain Local coordinate system with origin ; S342, coordinate system Translate to point Set up points normal with coordinate axes The cosine value between normal With point , The cosine of the angle between the lines is normal In plane projection with coordinate axes The included angle is Then point A triple between itself and other points in its neighborhood The calculation formula is as follows: in, express and The distance between them; S343. After completing the calculations in steps S341~S342, denote them as SPFH, i.e., Simple Point Feature Histogram. Then, recalculate the neighborhood of the remaining points, calculating the Simple Point Feature Histogram value for each point, and so on. The Fast Point Feature Histogram (FPFH) is calculated using the following formula: in, For weights, points are generally selected. Time Distance between them This represents the number of points in the neighborhood.

Images