A segment automatic troweling visual guidance method and system

Abstract

This invention provides a visually guided method and system for automated pipe segment finishing, belonging to the field of automated pipe segment technology. The invention acquires images and point cloud data of standard pipe segments, obtains their geometric features through the images, and records the movement path of the trowel to define a standard path. Next, it analyzes the image of the pipe segment to be finished for preliminary coarse matching, and then performs precise matching using the point cloud data. Based on the center of the point cloud of the pipe segment to be finished after registration, it generates a cleaning trajectory and connects them to form the actual cleaning path. It analyzes the height information of the standard pipe segment point cloud and the actual height information of the pipe segment to be finished, optimizing the contact surface to minimize height error. Finally, it statistically analyzes uneven points through image processing, comprehensively analyzes path smoothness, height error, and the proportion of uneven points, and generates a pass index to evaluate the finishing quality in real time. This invention achieves a fully automated pipe segment finishing process, significantly improving construction efficiency and quality, reducing costs, and has broad application prospects.

Description

Technical Field

[0001] This invention relates to the field of automated tunnel segment technology, specifically to a visually guided method and system for automated tunnel segment finishing. Background Technology

[0002] In the field of tunnel segment finishing, existing technologies have significant shortcomings. Traditional finishing tools are mostly 304 stainless steel or high-manganese steel trowels, whose shapes are difficult to adapt to the complex surfaces of tunnel segments. Finishing requires multiple operations and is highly dependent on manual skills. Even slight mistakes can lead to uneven finishing and defects, requiring repeated corrections. Using steel rulers to assist finishing is also unsatisfactory, as it is labor-intensive, inefficient, has low automation, is highly dependent on manual operation, and performs poorly in terms of precision control, failing to meet the high standards required for tunnel segment finishing.

[0003] Currently, the application of intelligent technology in tunnel segment finishing is severely lacking, with labor-intensive production still dominating. Manual finishing is inefficient and costly. At least two workers are needed per finishing station to maintain production line rhythm. Workers are prone to fatigue from prolonged repetitive tasks, affecting finishing quality, increasing subsequent repair work, and the high cost of hiring finishing workers, a skilled trade. With rising raw material costs, tunnel segment manufacturing costs remain high, weakening enterprise competitiveness. The lack of intelligent equipment makes real-time monitoring of the finishing process impossible, compromising tunnel segment quality and severely hindering industry development.

[0004] Therefore, in today's fiercely competitive cost environment, developing an intelligent surface finishing system for pipe segments to reduce production costs is an essential path for enterprises to reduce costs, increase efficiency, and enhance competitiveness.

[0005] The existing technology has the following shortcomings:

[0006] Traditional tunnel segment finishing processes rely heavily on manual labor, resulting in low efficiency, inconsistent quality, and high labor costs. During manual finishing, workers often struggle to maintain a smooth trowel path, leading to uneven finishing and even rough surfaces, which negatively impacts subsequent construction and the overall performance of the tunnel segments. Furthermore, manual operation has limitations in height control and geometric feature recognition, making high-precision construction difficult, especially in complex construction environments, where errors and defects are easily introduced.

[0007] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide a visually guided method and system for automated surface finishing of pipe segments, in order to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A visually guided method for automated surface finishing of tunnel segments, comprising the following steps:

[0011] Step 1: Collect images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path.

[0012] Step 2: Collect images and point cloud data of the pipe segment to be coated, analyze the images of the pipe segment to be coated to determine the feature points of the pipe segment to be coated, perform preliminary coarse matching of the pipe segment to be coated with the standard pipe segment through the feature points, and then perform precise matching through the point cloud data;

[0013] Step 3: Based on the center of the point cloud of the tube segment to be cleaned after registration, obtain the cleaning trajectory by transforming the centers of adjacent point clouds, and then connect the cleaning trajectories to obtain the actual cleaning path.

[0014] Step 4: Analyze the point cloud data of the standard tube segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error between the standard tube segment and the trowel path, and minimize the contact surface through an optimization algorithm;

[0015] Step 5: Real-time acquisition of images of the pipe segments to be plastered after construction, conversion of images into grayscale images, statistical analysis of uneven points in the grayscale images, comprehensive analysis of path smoothness, total height error and uneven point ratio to obtain the current plastering qualification index and determine whether the current plastering is qualified.

[0016] Furthermore, the specific data collected regarding the standard tunnel segment includes:

[0017] The geometric features include the coordinate information of the edges and corners of the standard pipe segment. The edge information of the standard pipe segment is obtained using an edge detection algorithm, and the corners of the standard pipe segment are determined using a corner detection algorithm.

[0018] Furthermore, the smoothness of the computation path specifically includes:

[0019] By manually operating the robot, the movement path of the spatula is recorded, and the path coordinates are stored as a point set in the following format: , Indicates the first pixel is in shaft and Position on the axis Given the total number of pixels, the formula for calculating the smoothness of a standard path is:

[0020]

[0021] in, For the smoothness of the standard path, For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinate of each pixel For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinates of each pixel, and .

[0022] Furthermore, the preliminary rough matching of the tube segments to be coated specifically includes:

[0023] Edge detection and corner detection algorithms are used to obtain feature points of the pipe segment to be smoothed, namely, points on the edges and corners. For each feature point... Calculate the eigenvector , To define the dimensions of the descriptors, Euclidean distance between descriptors is used to match feature points. Matched feature point pairs are calculated. Let the features of the pipe segment to be smoothed be... The characteristics of standard tunnel segments are: The formula for calculating the distance between feature points is:

[0024]

[0025] in, For the first section of the tube to be coated The first characteristic point and the standard segment The distance between feature points For the first section of the tube to be coated Feature vectors of feature points For standard tunnel segments Feature vectors of 1 feature point;

[0026] The coarse registration uses RANSAC (Random Sample Consensus Algorithm) to select interior points, removes exterior points, and calculates the coarse registration transformation matrix based on the interior points, as shown in the following formula:

[0027]

[0028] in, The transformation matrix for coarse registration. For rotation matrix, It is a translation vector;

[0029] Furthermore, the precise matching using point cloud data specifically includes:

[0030] Collect point cloud data of the pipe segment to be coated and record it as follows: , The total number of point cloud data for the tunnel segment to be coated is [number], and the standard tunnel segment point cloud data record is [number]. , Given the total number of standard tunnel segment point cloud data, the standard tunnel segment point cloud, the point cloud of the tunnel segment to be smoothed, and the initial transformation matrix are used as input. For each point in the point cloud, the nearest point pair is found, and the optimal transformation between the point pairs is calculated. Let the registered point cloud be... The least squares method is used to optimize the transformation, and the formula is expressed as:

[0031]

[0032] in, This is the update transformation matrix calculated based on the point cloud matching results after each iteration. For any pair of points, This is the current transformation matrix;

[0033] The updated transformation matrix is ​​applied to obtain the registered point cloud, which is then used to update the point cloud of the tube segment to be smoothed, denoted as follows: The root mean square error of the current registration result is calculated using the following formula:

[0034]

[0035] in, This is the root mean square error of the current registration result;

[0036] when If the iteration stops, then stop. This is the preset threshold.

[0037] Furthermore, obtaining the actual cleaning path specifically includes:

[0038] The formula for calculating the center of the point cloud of the tube segment to be smoothed is as follows:

[0039]

[0040] in, The center of the point cloud for the tube segment to be coated. For the first section of the tube to be coated The coordinates of each point, i.e. ;

[0041] The transformation between adjacent centers is calculated to obtain the cleaning trajectory, as shown in the following formula:

[0042]

[0043] in, To describe the sweep trajectory, describe the transformation from the previous geometric center to the current geometric center. It is the geometric center of the previous point cloud;

[0044] The actual cleaning path is generated based on the calculated cleaning trajectory to guide the cleaning robot, specifically including:

[0045]

[0046] in, For the actual cleaning path, Indicates the first cleaning trajectory. This indicates the second cleaning trajectory. Indicates the first A cleaning trajectory, The number of path points;

[0047] Furthermore, the total distance error for obtaining the standard tube segment and trowel path specifically includes:

[0048] Sensors are installed on the trowel to collect real-time height data of the trowel's position during its movement. The real-time trowel height data is compared with the height data of a standard pipe segment to obtain the total height error.

[0049]

[0050] in, This represents the total height error. For standard tunnel segments in position height, This represents coordinate information. , This represents the number of points included in the error calculation.

[0051] The objective function is defined as the total height error. The contact surface is minimized by optimizing the objective function using the least squares method. The height model is defined as follows:

[0052]

[0053] in, These are the parameters to be optimized.

[0054] The total height error is calculated as the loss function:

[0055]

[0056] Calculate the loss function with respect to the parameters and The partial derivatives are used to update the parameters using gradient descent:

[0057]

[0058]

[0059] in, This is the learning rate.

[0060] Furthermore, the determination of whether the current surface finishing is qualified specifically includes:

[0061] Real-time acquisition of images of the pipe segments to be plastered after construction, and conversion of the images into grayscale images.

[0062] Next, convert the grayscale image to a binary image using the following formula:

[0063]

[0064] in, These are the pixel values ​​of a binary image. The set pixel threshold;

[0065] The number of uneven points in the image is counted, and the proportion of uneven points in the entire point cloud to be smoothed is obtained using the following formula:

[0066]

[0067] in, The proportion of uneven points, This represents the number of uneven points in the image.

[0068] The formula used to obtain the qualification index is:

[0069]

[0070] in, The passing index, To ensure the smoothness of the path of the pipe segment to be coated, For the smoothness of the standard path, The maximum allowable value for the total height error. These are the weighting coefficients for each item. ,and ;

[0071] The pass index is compared with the preset pass threshold. If the pass index is higher than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a qualified state, and construction can be stopped. If the pass index is lower than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a unqualified state, and plastering continues. The result is then fed back to optimize the point cloud matching step.

[0072] The present invention also provides an automated surface finishing visual guidance system for tunnel segments, which is used to implement the above-mentioned automated surface finishing visual guidance method for tunnel segments, comprising:

[0073] The standard tube segment feature acquisition module is used to acquire images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path.

[0074] The module for analyzing the pipe segments to be coated is used to collect images and point cloud data of the pipe segments to be coated, analyze the images of the pipe segments to be coated to determine the feature points of the pipe segments to be coated, perform preliminary coarse matching between the pipe segments to be coated and standard pipe segments through the feature points, and then perform precise matching through the point cloud data.

[0075] The cleaning trajectory generation module is used to obtain the cleaning trajectory by transforming the centers of adjacent point clouds based on the center of the point cloud of the pipe segment to be cleaned after registration. The cleaning trajectories are then connected in series to obtain the actual cleaning path.

[0076] The height error analysis module is used to analyze the point cloud data of the standard pipe segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error of the standard pipe segment and the trowel path, and minimize the contact surface through an optimization algorithm.

[0077] The plastering quality assessment module is used to collect images of the pipe segments after plastering in real time, convert the images into grayscale images, count the uneven points in the grayscale images, and comprehensively analyze the smoothness of the path, the total height error, and the proportion of uneven points to obtain the current plastering qualification index and determine whether the current plastering is qualified.

[0078] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0079] Compared to surface finishing techniques using ordinary trowels and steel rulers, this invention offers the following advantages: First, it significantly improves efficiency. Traditional trowel-assisted surface finishing relies heavily on manual operation, is cumbersome, and slow. This invention allows for the completion of more tunnel segments within the same timeframe, greatly increasing production efficiency and meeting the demands of large-scale production. Second, it significantly improves surface finishing quality. Ordinary trowels, due to their shape and operating limitations, are prone to uneven finishing and defects. Using steel rulers also makes it difficult to guarantee accuracy. This invention, however, can control the surface flatness error of the tunnel segments within an acceptable range, reducing subsequent repair work and improving product reliability and stability. Third, it reduces long-term operating costs. Traditional finishing methods rely on a large number of skilled workers, resulting in high labor costs. While the intelligent finishing system of this invention requires some initial investment, in the long run, the machine replaces a large amount of manual labor, reducing labor costs. Furthermore, this invention avoids the safety risks associated with workers performing repetitive tasks for extended periods, which can lead to fatigue and operational errors. This invention can provide timely feedback on the construction status, ensuring that the finishing meets the predetermined standards, reducing the need for manual intervention, thereby effectively reducing construction costs and improving overall construction efficiency. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0081] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0083] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0084] Example:

[0085] Please see Figure 1 The present invention provides a technical solution:

[0086] A visually guided method for automated surface finishing of tunnel segments, comprising the following steps:

[0087] Step 1: Collect images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path.

[0088] In this embodiment, the collection of relevant data for standard tunnel segments specifically includes:

[0089] The geometric features include the coordinate information of the edges and corners of the standard pipe segment. The edge information of the standard pipe segment is obtained using the Canny edge detection algorithm, and the corners of the standard pipe segment are determined using the Harris corner detection algorithm. The edge information and corner information help to determine the coordinate information of feature points on the standard pipe segment in the subsequent process.

[0090] Edge detection algorithms specifically include:

[0091] Calculate the image in and The gradient in the direction is used to obtain the edge strength and direction. Non-edge points are suppressed along the gradient direction, while edge points are retained. Edge classification is performed using a double threshold. Finally, the final edge is determined by connecting strong and weak edges. The edge detection algorithm can be expressed by the following formula:

[0092]

[0093] in, To identify location The edge strength value indicates whether a point is an edge. An image of a standard tube segment. The gradient of the image in the horizontal direction represents the change in image brightness along the x-axis. The gradient of the image in the vertical direction represents the rate of change of image brightness along the y-axis. These gradient values ​​are used to determine the edge strength and direction of the image. If the gradient value at a point is large, it indicates that the point is an edge. The system uses low and high thresholds. The low threshold is used to initially determine whether a point might be an edge, while the high threshold is used to determine the edge's strength. Only when the edge strength exceeds this value will it be identified as an edge. These two thresholds are used for edge connection and filtering to reduce the impact of noise on edge detection. The edge detection process consists of three stages: Strong edges: Strength values ​​higher than... Points, weak edges: intensity values ​​at and Points between, not edges: intensity values ​​lower than point.

[0094] Calculating corner points specifically includes:

[0095] For each pixel Calculate the gradient information of its surrounding region and construct a second-order matrix according to the following formula:

[0096]

[0097]

[0098] in, It is a second-order matrix;

[0099] The formula for calculating the response function is:

[0100]

[0101] in, It is a matrix The determinant of the matrix represents the strength of the corner point. It is a matrix The trace (the sum of the diagonal elements) represents the total change in a local region of the image. It is an empirical parameter, typically ranging from 0.04 to 0.06, used to adjust the sensitivity of the response function.

[0102] in:

[0103]

[0104]

[0105] Perform the above calculations on each pixel in the image to obtain the response function. Determine the threshold and retain only Points with values ​​greater than the threshold are designated as corner points.

[0106] Step 2: Collect images and point cloud data of the pipe segment to be coated, analyze the images of the pipe segment to be coated to determine the feature points of the pipe segment to be coated, perform preliminary coarse matching of the pipe segment to be coated with the standard pipe segment through the feature points, and then perform precise matching through the point cloud data;

[0107] In this embodiment, the smoothness of the calculation path specifically includes:

[0108] By manually operating the robot, the movement path of the spatula is recorded, and the path coordinates are stored as a point set in the following format: , Indicates the first pixel is in shaft and Position on the axis Given the total number of pixels, the formula for calculating the smoothness of a standard path is:

[0109]

[0110] in, For the smoothness of the standard path, For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinate of each pixel For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinates of each pixel, and .

[0111] Step 3: Based on the center of the point cloud of the tube segment to be cleaned after registration, obtain the cleaning trajectory by transforming the centers of adjacent point clouds, and then connect the cleaning trajectories to obtain the actual cleaning path.

[0112] In this embodiment, the preliminary rough matching of the tube segments to be coated specifically includes:

[0113] Edge detection and corner detection algorithms are used to obtain feature points of the pipe segment to be smoothed, namely, points on the edges and corners. For each feature point... Calculate the eigenvector , To determine the dimension of the descriptor, specific methods for calculating feature vectors include SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), and ORB (Oriented Fast Feature Transform). This embodiment uses the SIFT method. First, Gaussian differences are used to find feature points in the image to be smoothed. For each feature point, the principal direction is calculated based on the direction and magnitude of the local image gradient. A 16×16 pixel window is defined around the feature point, dividing it into 4×4 sub-regions. For each sub-region, a histogram of gradient directions is calculated. Each histogram contains 8 directions. Therefore, the feature descriptor of a key point can be represented as:

[0114]

[0115] in, The gradient histogram for each sub-region has a dimension of 8 (number of directions) × 16 (number of sub-regions) = 128;

[0116] Feature points are matched using Euclidean distance between descriptors, and the matched feature point pairs are calculated. Let the feature of the segment to be smoothed be... The characteristics of standard tunnel segments are: The formula for calculating the distance between feature points is:

[0117]

[0118] in, For the first section of the tube to be coated The first characteristic point and the standard segment The distance between feature points For the first section of the tube to be coated Feature vectors of feature points For standard tunnel segments Feature vectors of 1 feature point;

[0119] The coarse registration uses RANSAC (Random Sample Consensus Algorithm) to select interior points, removes exterior points, and calculates the coarse registration transformation matrix based on the interior points, as shown in the following formula:

[0120]

[0121] in, The transformation matrix for coarse registration. For rotation matrix, It is a translation vector;

[0122] choose For the matched feature points, calculate the centroids of the point clouds of the segment to be smoothed and the standard segment. The formula for calculating the centroid of the point cloud of the segment to be smoothed is:

[0123]

[0124] in, Let be the centroid of the point cloud of the tube segment to be smoothed. The centroid is the average position of all matching feature points and represents the geometric center of the point cloud. The first section of the tube to be coated A point cloud of individual points, , The total number of matching feature points, and , For the tube segments to be coated For the matched feature points respectively in Mean at position;

[0125] The formula for calculating the centroid of the point cloud of a standard tunnel segment is:

[0126]

[0127] in, The centroid of the point cloud for the standard tube segment. For the first standard tunnel segment A point cloud of individual points, , , For standard tunnel segments For the matched feature points respectively in Mean at position;

[0128] Decenter each point in the point cloud (i.e., subtract the centroid) to obtain a new point cloud:

[0129]

[0130] in, This is the new point cloud for the tube segment to be smoothed after decentralization. This represents the new point cloud for the standard pipe segment after decentralization.

[0131] The formula for constructing the covariance matrix is:

[0132]

[0133] in, For the covariance matrix, the operators are... Represents the outer product of vectors. For the first standard segment after decentralization Point cloud;

[0134] The formula for singular value decomposition of the covariance matrix is:

[0135]

[0136] in, and It is an orthogonal matrix representing the rotation relationships of the point cloud. It is a diagonal matrix containing singular values;

[0137] Here, we need to calculate the matrix first. transpose and The product of, i.e. Find the matrix by solving the eigenvalue problem. The eigenvalues ​​and eigenvectors, i.e., solving the equation , identity matrix For eigenvalues, The eigenvectors are arranged in columns into a matrix. Ensure that the eigenvalues ​​are arranged in descending order; similarly, calculate the matrix. and The product of the transposes of, i.e. ,right Perform the same eigenvalue decomposition, i.e. ,here It is an eigenvalue, The eigenvectors are arranged in columns into a matrix. Similarly, arrange them in descending order of eigenvalues; calculate the singular values, which are... or The square roots of the eigenvalues ​​are used to arrange these singular values ​​in descending order to form a diagonal matrix. The elements on the diagonal are , These are the eigenvalues.

[0138] The rotation matrix is ​​calculated using the results of singular value decomposition, as follows:

[0139]

[0140] To ensure the positive definiteness of the rotation matrix (and avoid non-rotation), it can be adjusted using the following formula:

[0141]

[0142] The translation vector is calculated using the difference between the centroids, as shown in the following formula:

[0143]

[0144] Among them, the translation vector This indicates the amount of translation required to move from the centroid of the rotated surface to the centroid of the standard segment.

[0145] In this embodiment, the precise matching using point cloud data specifically includes:

[0146] Collect point cloud data of the pipe segment to be coated and record it as follows: , The total number of point cloud data for the tunnel segment to be coated is [number], and the standard tunnel segment point cloud data record is [number]. , Given the total number of standard tunnel segment point cloud data, the standard tunnel segment point cloud, the point cloud of the tunnel segment to be smoothed, and the initial transformation matrix are used as input. For each point in the point cloud, the nearest point pair is found, and the optimal transformation between the point pairs is calculated. Let the registered point cloud be... The least squares method is used to optimize the transformation, and the formula is expressed as:

[0147]

[0148] in, This is the update transformation matrix calculated based on the point cloud matching results after each iteration. For any pair of points, This is the current transformation matrix;

[0149] The updated transformation matrix is ​​applied to obtain the registered point cloud, which is then used to update the point cloud of the tube segment to be smoothed, denoted as follows: The root mean square error of the current registration result is calculated using the following formula:

[0150]

[0151] in, This is the root mean square error of the current registration result;

[0152] when If the iteration stops, then stop. This is the preset threshold.

[0153] In this embodiment, obtaining the actual cleaning path specifically includes:

[0154] The formula for calculating the center of the point cloud of the tube segment to be smoothed is as follows:

[0155]

[0156] in, The center of the point cloud for the tube segment to be coated. For the first section of the tube to be coated The coordinates of each point, i.e. ;

[0157] The transformation between adjacent centers is calculated to obtain the cleaning trajectory, as shown in the following formula:

[0158]

[0159] in, To describe the sweep trajectory, describe the transformation from the previous geometric center to the current geometric center. It is the geometric center of the previous point cloud;

[0160] The actual cleaning path is generated based on the calculated cleaning trajectory to guide the cleaning robot, specifically including:

[0161]

[0162] in, For the actual cleaning path, Indicates the first cleaning trajectory. This indicates the second cleaning trajectory. Indicates the first A cleaning trajectory, The number of path points;

[0163] Step 4: Analyze the point cloud data of the standard tube segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error between the standard tube segment and the trowel path, and minimize the contact surface through an optimization algorithm;

[0164] In this embodiment, obtaining the total height error of the standard tube segment and the trowel path specifically includes:

[0165] A laser rangefinder sensor is installed on the trowel. The sensor collects the height data of the trowel's position in real time during the trowel's movement. The real-time monitored trowel height data is compared with the height data of a standard pipe segment to obtain the total height error.

[0166]

[0167] in, The total height error represents the deviation between the trowel path and the target height of the standard segment. For standard tunnel segments in position height, This represents coordinate information. , This represents the number of points included in the error calculation.

[0168] The objective function is defined as the total height error. The contact surface is minimized by optimizing the objective function using the least squares method. The height model is defined as follows:

[0169]

[0170] in, These are the parameters to be optimized.

[0171] The total height error is calculated as the loss function:

[0172]

[0173] Calculate the loss function with respect to the parameters and The partial derivatives are used to update the parameters using gradient descent:

[0174]

[0175]

[0176] in, The learning rate can be initially set to 0.01 or 0.001, and then adjusted based on performance during training.

[0177] Step 5: Real-time acquisition of images of the pipe segments to be plastered after construction, conversion of images into grayscale images, statistical analysis of uneven points in the grayscale images, comprehensive analysis of path smoothness, total height error and uneven point ratio to obtain the current plastering qualification index and determine whether the current plastering is qualified.

[0178] In this embodiment, determining whether the current surface finishing is qualified specifically includes:

[0179] Real-time acquisition of images of the pipe segments to be plastered after construction, and conversion of the images into grayscale images.

[0180] The color value of each pixel in the real-time image of the tube segment to be smoothed is represented as:

[0181]

[0182] in, The color value for each pixel, Indicates the location is The intensity of the red channel at that location, Indicates the location is The strength of the green channel at that location, Indicates the location is The intensity of the blue channel at that location;

[0183] A color image is converted to a grayscale image using a grayscale conversion formula. The grayscale conversion formula is as follows:

[0184]

[0185] in, For the location at The grayscale value at that location ranges from 1 to 2. Between (for 8-bit images);

[0186] Next, convert the grayscale image to a binary image using the following formula:

[0187]

[0188] in, These are the pixel values ​​of the binary image, where 1 represents an uneven point and 0 represents a smooth point. The set pixel threshold is used to distinguish between flat and uneven points;

[0189] The number of uneven points in the image is counted, and the proportion of uneven points in the entire point cloud to be smoothed is obtained using the following formula:

[0190]

[0191] in, The proportion of uneven points, This represents the number of uneven points in the image.

[0192] The formula used to obtain the qualification index is:

[0193]

[0194] in, The passing index, To ensure the smoothness of the path of the pipe segment to be coated, For the smoothness of the standard path, The maximum allowable value for the total height error. These are the weighting coefficients for each item. ,and ;

[0195] The pass index is compared with the preset pass threshold. If the pass index is higher than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a qualified state, and construction can be stopped. If the pass index is lower than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a unqualified state, and plastering continues. The result is then fed back to optimize the point cloud matching step.

[0196] The path smoothness difference item compares the smoothness of the segments to be coated. Smoothness of standard segments This reflects the smoothness of the path during the smoothing process. Path smoothness directly affects the smoothing effect; a smooth path can reduce local unevenness and defects. The value of this item is within... Between. If If the value is 1, then the item is 1, indicating a perfect match; if A larger value indicates that this item is close to 0, reflecting the degree of non-compliance. The height error item is used to assess the height difference between the segment to be coated and the standard segment. Smaller errors mean the spatula stays closer to the intended path during the smearing process, thus improving the quality of the smear. Normalization ensures this is also achieved. If the error is zero, this term is 1; if the error reaches its maximum value... At that time, this item is 0. Different industries and project types may have different requirements for flatness and error. For concrete plastering, the maximum allowable height error can be set to 5mm. If higher flatness requirements are needed, the maximum value can be reduced to 3mm or less. The unevenness point ratio item evaluates the plastering quality by the proportion R of uneven points in the entire point cloud to be plastered. The smaller the proportion of uneven points, the better the plastering quality, and vice versa. Therefore, this item is also... Between. If (If there are no uneven points), then this item is 1; if The larger the value, the closer this value is to 0, indicating a decrease in quality. The preset acceptable threshold can be set to 0.8, or adjusted according to the required smoothness of the surface for this application.

[0197] The number or proportion of uneven points directly determines the quality of the troweling process. The main purpose of troweling is to achieve a smooth surface; a large number of uneven points indicates a poor troweling effect. Therefore, giving this factor the highest weight is reasonable. The presence of uneven points is the most intuitive impression for users and is usually the easiest problem to spot during quality inspection. The smoothness of the trowel path has a significant impact on the uniformity and overall quality of the troweling process, but its impact may be relatively indirect compared to the number of uneven points. A smooth path allows for better control of the trowel, but if the final result has many uneven points, the impact of smoothness will be weakened. Therefore, although path smoothness is still important, its weight can be set to the second highest. Height error reflects how close the trowel is to the standard path. Although it affects the troweling effect, in practical applications, the impact of height error may be smaller than that of smoothness and uneven points. In actual operation, path error may not directly cause unevenness, but it still needs to be considered; therefore, setting the weight of height error to the lowest is reasonable. Ensure This enables the qualification index The value in This allows for direct assessment of eligibility.

[0198] Please see Figure 2 The present invention also provides an automated surface finishing visual guidance system for pipe segments, which is used to implement the above-mentioned automated surface finishing visual guidance method for pipe segments, comprising:

[0199] The standard tube segment feature acquisition module is used to acquire images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path.

[0200] The module for analyzing the pipe segments to be coated is used to collect images and point cloud data of the pipe segments to be coated, analyze the images of the pipe segments to be coated to determine the feature points of the pipe segments to be coated, perform preliminary coarse matching between the pipe segments to be coated and standard pipe segments through the feature points, and then perform precise matching through the point cloud data.

[0201] The cleaning trajectory generation module is used to obtain the cleaning trajectory by transforming the centers of adjacent point clouds based on the center of the point cloud of the pipe segment to be cleaned after registration. The cleaning trajectories are then connected in series to obtain the actual cleaning path.

[0202] The height error analysis module is used to analyze the point cloud data of the standard pipe segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error of the standard pipe segment and the trowel path, and minimize the contact surface through an optimization algorithm.

[0203] The plastering quality assessment module is used to collect images of the pipe segments after plastering in real time, convert the images into grayscale images, count the uneven points in the grayscale images, and comprehensively analyze the smoothness of the path, the total height error, and the proportion of uneven points to obtain the current plastering qualification index and determine whether the current plastering is qualified.

[0204] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0205] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0206] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method of visual guidance for segmental concrete automatic finishing, characterized in that, The specific steps include: Step 1: Collect images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path. Step 2: Collect images and point cloud data of the pipe segment to be coated, analyze the images of the pipe segment to be coated to determine the feature points of the pipe segment to be coated, perform preliminary coarse matching of the pipe segment to be coated with the standard pipe segment through the feature points, and then perform precise matching through the point cloud data; Step 3: Based on the center of the point cloud of the tube segment to be cleaned after registration, obtain the cleaning trajectory by transforming the centers of adjacent point clouds, and then connect the cleaning trajectories to obtain the actual cleaning path. Step 4: Analyze the point cloud data of the standard tube segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error between the standard tube segment and the trowel path, and minimize the contact surface through an optimization algorithm; Step 5: Real-time acquisition of images of the pipe segments to be plastered after construction, conversion of images into grayscale images, statistical analysis of uneven points in the grayscale images, comprehensive analysis of path smoothness, total height error and uneven point ratio to obtain the current plastering qualification index and determine whether the current plastering is qualified. The total height error of obtaining the standard tube segment and trowel path specifically includes: Sensors are installed on the trowel to collect real-time height data of the trowel's position during its movement. The real-time trowel height data is compared with the height data of a standard pipe segment to obtain the total height error. ； wherein, is the total height error, is the height of the standard tube sheet at position , represents the coordinate information , is the number of points included in the error calculation; The objective function is defined as the total height error. The contact surface is minimized by optimizing the objective function using the least squares method. The height model is defined as follows: ； in, These are the parameters to be optimized. The total height error is calculated as the loss function: ； Calculate the loss function with respect to the parameters and The partial derivatives are used to update the parameters using gradient descent: ； ； in, This is the learning rate.

2. The automated visual guidance method for pipe segment finishing according to claim 1, characterized in that, The specific data collected regarding the standard tunnel segment includes: The geometric features include the coordinate information of the edges and corners of the standard pipe segment. The edge information of the standard pipe segment is obtained using an edge detection algorithm, and the corners of the standard pipe segment are determined using a corner detection algorithm.

3. The automated visual guidance method for surface finishing of tunnel segments according to claim 1, characterized in that, The smoothness of the calculation path specifically includes: By manually operating the robot, the movement path of the spatula is recorded, and the path coordinates are stored as a point set in the following format: , Indicates the first pixel is in shaft and Position on the axis Given the total number of pixels, the formula for calculating the smoothness of a standard path is: ； in, For the smoothness of the standard path, For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinate of each pixel For the first in the standard path The x-coordinate of each pixel For the first in the standard path The vertical coordinates of each pixel, and .

4. The automated visual guidance method for pipe segment finishing according to claim 1, characterized in that, The preliminary rough matching of the tube segments to be coated specifically includes: Edge detection and corner detection algorithms are used to obtain feature points of the pipe segment to be smoothed, namely, points on the edges and corners. For each feature point... Calculate the eigenvector , To define the dimensions of the descriptors, Euclidean distance between descriptors is used to match feature points. Matched feature point pairs are calculated. Let the features of the pipe segment to be smoothed be... The characteristics of standard tunnel segments are: The formula for calculating the distance between feature points is: ； in, For the first section of the tube to be coated The first characteristic point and the standard segment The distance between feature points For the first section of the tube to be coated Feature vectors of feature points For standard tunnel segments Feature vectors of 1 feature point; The coarse registration uses RANSAC (Random Sample Consensus Algorithm) to select interior points, removes exterior points, and calculates the coarse registration transformation matrix based on the interior points, as shown in the following formula: ； in, The transformation matrix for coarse registration. For rotation matrix, It is a translation vector.

5. The automated visual guidance method for surface finishing of tunnel segments according to claim 4, characterized in that, The precise matching using point cloud data specifically includes: Collect point cloud data of the pipe segment to be coated and record it as follows: , The total number of point cloud data for the tunnel segment to be coated is [number], and the standard tunnel segment point cloud data record is [number]. , Given the total number of standard tunnel segment point cloud data, the standard tunnel segment point cloud, the point cloud of the tunnel segment to be smoothed, and the initial transformation matrix are used as input. For each point in the point cloud, the nearest point pair is found, and the optimal transformation between the point pairs is calculated. Let the registered point cloud be... The least squares method is used to optimize the transformation, and the formula is expressed as: ； in, This is the update transformation matrix calculated based on the point cloud matching results after each iteration. For any pair of points, This is the current transformation matrix; The updated transformation matrix is ​​applied to obtain the registered point cloud, which is then used to update the point cloud of the tube segment to be smoothed, denoted as follows: The root mean square error of the current registration result is calculated using the following formula: ； in, This is the root mean square error of the current registration result; when If the iteration stops, then stop. This is the preset threshold.

6. The automated segment finishing visual guidance method according to claim 5, characterized in that, The specific steps for obtaining the actual cleaning path include: The formula for calculating the center of the point cloud of the tube segment to be smoothed is as follows: ； in, The center of the point cloud for the tube segment to be coated. For the first section of the tube to be coated The coordinates of each point, i.e. ; The transformation between adjacent centers is calculated to obtain the cleaning trajectory, as shown in the following formula: ； in, To describe the sweep trajectory, describe the transformation from the previous geometric center to the current geometric center. It is the geometric center of the previous point cloud; The actual cleaning path is generated based on the calculated cleaning trajectory to guide the cleaning robot, specifically including: ； in, For the actual cleaning path, Indicates the first cleaning trajectory. This indicates the second cleaning trajectory. Indicates the first A cleaning trajectory, This represents the number of path points.

7. The automated visual guidance method for surface finishing of pipe segments according to claim 1, characterized in that, The specific steps for determining whether the current finishing is satisfactory include: Real-time acquisition of images of the pipe segments to be plastered after construction, conversion of the images to grayscale, and then conversion of the grayscale images to binary images, using the following formula: ； in, These are the pixel values ​​of a binary image. The set pixel threshold; The number of uneven points in the image is counted, and the proportion of uneven points in the entire point cloud to be smoothed is obtained using the following formula: ； in, The proportion of uneven points, This represents the number of uneven points in the image. The formula used to obtain the qualification index is: ； in, The passing index, To ensure the smoothness of the path of the pipe segment to be coated, For the smoothness of the standard path, The maximum allowable value for the total height error. These are the weighting coefficients for each item. ,and ; The pass index is compared with the preset pass threshold. If the pass index is higher than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a qualified state, and construction can be stopped. If the pass index is lower than the preset pass threshold, it indicates that the construction of the current pipe segment to be plastered is in a unqualified state, and plastering continues. The result is then fed back to optimize the point cloud matching step.

8. A visual guidance system for automated surface finishing of tunnel segments, characterized in that, The automated segment finishing visual guidance system is used to implement the automated segment finishing visual guidance method according to any one of claims 1-7, comprising: The standard tube segment feature acquisition module is used to acquire images and point cloud data of standard tube segments. The images are used to obtain the geometric features of the standard tube segments. The robot is manually operated to record the movement path of the trowel and calculate the smoothness of the path. The module for analyzing the pipe segments to be coated is used to collect images and point cloud data of the pipe segments to be coated, analyze the images of the pipe segments to be coated to determine the feature points of the pipe segments to be coated, perform preliminary coarse matching between the pipe segments to be coated and standard pipe segments through the feature points, and then perform precise matching through the point cloud data. The cleaning trajectory generation module is used to obtain the cleaning trajectory by transforming the centers of adjacent point clouds based on the center of the point cloud of the pipe segment to be cleaned after registration. The cleaning trajectories are then connected in series to obtain the actual cleaning path. The height error analysis module is used to analyze the point cloud data of the standard pipe segment to obtain the height value of each point, then obtain the height value of the trowel at each point, obtain the total height error of the standard pipe segment and the trowel path, and minimize the contact surface through an optimization algorithm. The plastering quality assessment module is used to collect images of the pipe segments after plastering in real time, convert the images into grayscale images, count the uneven points in the grayscale images, and comprehensively analyze the smoothness of the path, the total height error, and the proportion of uneven points to obtain the current plastering qualification index and determine whether the current plastering is qualified.

Images